give up your earnest fee and hope that seller lets u go

Be realistic. If u really don't want this house because of noise, tell the seller that u are willing to give up the earnest fee. U don't have a case since there is no contingency on inspection.
Yes, sellers could sue u, but mostly they will not coz nobody wants to get into a lengthy and costly legal battle, and they might not be able to find the next buyer because of the bad image.
BTW, your agent is really aweful if he is your buyer's agent. Once u get out of your contract, u might want to complain to his boss and the realtor organization. He is really an idiot for not checking the inspection contingency for his clients.