I think if you don't separate for more than 6 months, things shouldn't be too bad. He can try to go back first and settle down, then, you and kids be back with him.