There is a difference between not knowing (your feeling) and not caring about it.
Your concern is more about the former than the latter, and you should.
Ask him directly.
He does not have to care, but he should know. In life, many not knows are masked by not cares, and that's not wise.
