It sounds like you're male and so your sexist patient is nicer to you than to your female colleagues. The tone of this post strikes me as dismissive of their experience, like because he's not so bad to you, they must be exaggerating. You're not obligated to hate this patient on your coworkers' behalf, but maybe express some empathy for them?