You are wasting your time
You are wasting your time
and you are wasting mine.
If you care about hurt feelings
don't get into the business of renting.
The issue here is, after both parties of a contract already performed (you allowed him to stay, and he paid the rent), should one party have right to demand another party to modify the contract that already concluded? If you think about it, you'll find your demand is laughable.
Based on what you said, I would not sign your addendum either. And I agree with the father, I could sue you if you don't return the deposit.