The reason "the few" get away with behavior that ostracizes minorities is because the majority, both gamers and developers, are apathetic to it. Gaming was arbitrarily cultivated as a boys' club for lonely white nerds, for one, many of which had an axe to grind with their female peers for not hooking up with them or for not displaying interest in their hobby. Number two, online gaming launched without much in the way of tools or reinforcement techniques to truly dissuade harassment and bigoted language being strewn about and it in turn has become a hallmark, an expectation, "playful trash talking" that is endemic to the "appeal" of game culture. Critique of games from a lens beyond just measuring the effectiveness of 1s and 0s is seen as invasive, an insidious means to make games less fun by providing better avenues for storytelling and representation (and the fact that fun is inversely tied to respectful representation is the problem). Throw in the fact that gamers do not exist in a vacuum, but are also human beings inundated with the political, social, and cultural mores of wherever they live (and we currently live in a time of right-wing populist backlash), and you inevitably get shit like Gamergate which, I remind you, is a movement that gaming developers and publishers did fuck all to openly rebuke.
Harassment is not a once-in-a-blue-moon occurrence in gaming culture. It is an omnipresent systemic failure of both gamers and developers to make the space welcoming for minorities from the outset.
And trying to downplay this as just a few gamers only adds to the problem.