They don't make that distinction against men because they're actively trying to attack men, and so want all of those 'miscommunications' and 'oh i didn't mean it that way even though thats absolutely what i said' bullshitery so they can hide how outrageously sexist they are being while pretending not to be.
Perhaps there's some truth to that, though in my circles i hear a lot more about the patriarchy than complaints about men in general.
The fallacy you are claiming they are making is the same one you made yourself when you waltzed in here and made it all about how hard things are for women.
I suppose i'm trying to defend a position that's not my own, and yeah, using "men" to describe a system created by some men to advantage all (or at least white) men in a broad way is absolutely sexist - but it's hardly the main issue here.