it's a very complex issue. but i disagree with you on one point:

the people living in inner cities can move out freely and once they move into the towns they desire, their kids can enter new schools without any hurdle. you could argue they don't have enough money for the housing in expensive towns. maybe voucher is a good way to give parent more choices.