He's making me live my past emotionally.

I’m 26years old in a 13months relationship with a 42years old man. our relationship is so good i enjoy it. His previous relationship didnt end well and i used to party a lot nd i long planed to quit my habbits before we met, but emotionally no. We’re always together but he doesnt trust me at all he even used to compare me with his ex but he stoped. I luv him so much when im with him feel so happy, he’s my only boyfriend but hes making me live my past emotionally by his assumptions of me having affairs and sleeping around. what do i do?