Your H4 starts when CIS approve your H4 application.

Currently, you are on both H1 and H4 pending status. You can choose either. As long as the old H1 is not revoked, you can continue to work. Just write a letter to the service center including the receipt stating you intend to withdraw the application. Do the withdrawal after your receive the receipt.
The other way is to stop payment of your check. But that may hurt your credit record.