This is a pure life style question

I would not move.

Moving from a big house is a significant undertaking.

Thinking about carpool with other kids' families who might have the same school activities? That's also a good way to build relationship. Yes you may still have to fill in from time to time as a driver.

Also, renting out a high-end house will be less profitable than a lower-end one.