I think this is a philosophical question or statement. Have

You heard of the "grandma exploits"?

Its about tricking the AI model giving you information that it refused before.

because the model was trained to be compassionate, it's really interesting,

teaching AI to be compassionate.