you should have asked him right away!

Instead analyzing for so many days,  I would have asked back right after he said that word. Why he was disappointed at? and what should i do to make a change.  If the boss is American, be very straight forward is the best way to know what he thinks.