As long as USCIS knows nothing about it.
If your enemy wrote a letter to USCIS to make a complain about you on this issue, you would be in trouble.
If your enemy wrote a letter to USCIS to make a complain about you on this issue, you would be in trouble.