He sponsored you to where you wanted to go and you should pay some price right? You can’t have everything you want. It’s not realistic that he is a perfect man.
Work out your own problems. Think what advantage you have, and then you can make your own decision, either keep him or leave him. No one will judge you; you are free for your own body and mind.