The late fee is like that and I suspect it was mentioned because you may have a right reasoning on the wear and tear.
So the lesson is: they will get the money one way or the other unless you are truly flawless.
Besides, if landlord stated it correctly, it is not the first time you were late. Recall so many here are shouting "Three days and evict"? Maybe what goes around, comes around.