I do not know the background so it is hard for the outsiders to give you the specific suggestions. Sounds like there are some issues out of your control and you can only try to influence to a certain degree.
You were asking how to work on your end not to care about other's opinions much. I think you can try to turn the negative comments to the positive feedback. There might be some truth in their comments that can help you to make some improvement. Think positively and show others your effort to do a better job. We all come cross the similar situation, it is part of life. Remember, sometimes you have no choice, but you always have the choice of your attitude. Tomorrow will be another day!
