-Yes, you are right if your I-485 is pending. But you have a potential risk. Even if your GC sponsor do not withdraw the appproved I-140, but if USCIS make any decision or ask RFE for your I-485 after you change the job, your I-485 will denied if the I-485 pending is <180 days.
If you have not file your I-485 yet and you change the job/employer, your I-140 will be cancelled no matter whether it has been approved or not.