Your lawyer is right.

If you your physical exam result without the RFE, it is much easier for your files to get lost and not added to your 485 files. The USCIS offices are hit with hundreds of thousands of mailings. Their employees may not care about a misplaced mail package, but you definitely do.