not a big deal

You don't need to change your address if your driver license's address has not been changed, and the address on your W2, pay stub should be consistent with your current address.

This is just in case they check (but I do not believe they will ever check at all, unless you are a criminal, who cares when you move where you move to? INS has much much more important things to do than investigating your address change), you have all addresses consistent (license' address, pay stub address, W2 address, etc). You will be fine.

Another thing to consider is that if you need a divorce, some states require seperate for some time first. If you decide to divorce, it might take longer time if you do not change your address now.