if you baby uses US passport

you have to apply for entry visa for him/her from the Chinese Consulate General near you first of all.

For either or both of you, you have to have valid travel documents to come back.