because a modest person is not perceived as someone who can carry a lot of weight by being responsible for himself and everyone under him. He is saying: "Don't expect too much from me."
One cannot be a boss if he is modest, because his boss cannot afford to worry about his modesty getting in the way of discharging his responsibilities.
(Your boss's behavior has a message: I do not care what you think of me. )