I can tell he is a very strong, independent, and brave cat, because he was strayed and has handled a lot of stress in the wild.
You should let him go. He will be fine no matter what.
MeiBeiBei and ShaoYe need you more. They cannot live without you.