I test drive the older A4 model before and it was a heavy car. The accelleration was not as good as the new A3. I saw the new A4 in person and it is a much nicer car.
If you care for the driving experience, the A3 is your better bet. If you care for the space etc. go with A4.