Did you also provide the estimate of repairing the damage from contractors?
It won't work if you just said that you kept deposit because of ..... (that was how I fighted with my old landlord)
Anyway, going to the court will not affect your credit. Trying to avoid it is what I would do.