Just because you don’t think so doesn’t mean those things haven’t had influence on the cultural perception of it.
I understand the desire to separate yourself from those things as an individual. And I don’t know what method will work, but simply telling people the “manosphere” just doesn’t count isn’t going to change anyone’s minds.