It depend on where it is. You may do survey by yourself and it should be done ASAP (before closing).
1. because of this you lose some lot which means you lose money.the other words is you pay money for someone else which it is not good thing for you. If the property is in the big city, maybe it will be big difference for the feature.
2. When you sell you property someday, you may lose some buyers because of it.
3. The seller will not want to involove it because if there is a problem he/she will be in trouble(either lose money or unsell the property).
4. Don't think other people, think about yourself. Nobody think about you including your buy agent. all they want is to make this transaction done and get money.
5. I know some cases take legal action because of it, I couldn't say how much it will hurt you.
6. If you are Ok with it then who else will not OK?