Now I understand you do not have H-1 visa

What you have is H-1 status, that is not visa. If you only have H-1 status without a valid visa, you need to bring H-1 approval notice together with your I-94 attached to it. You can surrender the white I-94 and change another white I-94.Your green I-94 attached to H-1 approval is kept by you.