The safest way is to send a certified mail with return receipt to USCIS informing that your attorney no longer represents you and requseting that all notices be sent to your address.
At least if anything happens in the future, you have proof that notice has been given.
PS: not a regular vistor.