The contact with new builder is normally in favor of builder. You need a real estate attorney to review yiur contract and give you the advice:
1) if the builder could quit the contract with no compensation to you.
2) if it is worth for you to sue the builder in the court?
To be hoenst, you can not force the builder to build a house if they don't want, but you should get some compensations except that the contact has clauses to allow the builder to do so.
Once the builder breached the contract, and you agreed, then you both quit the deal. However, it has to be in written.