I would do so if I were you.

It is true that nobody knows when USCIS contacts you for whatever extra material. So it is better for you not to leave the US for a long period, or to have someone monitor your case in the US while you are away, so that you don't miss some important communications/dates.