1) The woman does not meet the man's desire level. For example, she is not pretty enough.
2) The man has someone else in his heart.