you have done what you can do.if your boss already don't like you,this is just one more reason.If not,why you should worry about it.