The bottom line is - does he want to keep this family? If it is only a simple affair and play, make sure his finance is clean and clear to you - better under you and/or both of you control.
If this can be done, I would not suggest you to divorce - MAN is MAN, some of them never grow up until kids left home...