The hissing sound is related to the small pressure vessel that is a part of the emergency breaking boost. In normal breaking, when you press lightly and slowly, the vessel let's the air out through the relief valve.
In a panic stop, when you hit the breaks fast and hard, the air pressure initiates the break modulation before escaping through the valve. That's why when you hit the breaks hard, they seem to grab.
So, in normal breaking, you 'll hear that little hiss, or swish. You don't hear it in panic breaking because of all of the cursing and yelling going on.
http://www.toyota-4runner.org/5th-gen-t4rs/88073-brake-pedal-hiss.html
