I think you have to go there

I think you definitely have to go there to protect yourself. He is not going to sponsor your green card application, and he can dump you any time. If you have a domestic voilation case in file, it may help you to get your green card by yourself. There is a legal system to protect the victim of domestic violence which may help you to keep your temporary green card status. Without that protection, you may be out of status soon.