No, your new company is wrong

If your old company hasn't canceled your H1b, you can still continue working for it. It is just like someone holding two H1b working for two companies. But I am not sure if you need cancel the new H1b since you no longer plan to work for it. Good luck!