It ultimately comes down to the design, how fair the challenge is and how fun the game is. Most danmakus like Touhou, Dodonpachi, Deathsmiles, Guwange, etc. can be very, very difficulty on the highest difficulties, insanely so, but that kinda is the point: learning patterns, optimal strategies, as you replay through the stages a hundred times or more. That's a fair challenge. Similarly, games like Shovel Knight or The Surge have a pretty high base difficulty, but at the same time you're allowed to farm to improve yourself to great lenghts to make the game easier for you, though the game, as is, while difficult is feasible if you play it smart and are pretty good.
The problem is when the difficulty goes against the fun factor. Multiple Call Of Duty games had parts that were just unfun because they relied on bullshit mechanisms disguised as difficulty. Four places pop to mind:
- the entirety of World At War on the highest difficulty: enemies spam you with grenades with laser precision, that's just bullshit considering that leaving cover usually equals death
- the endless spawn of enemies while waiting for the chopper in Modern Warfare: you are rewarded for exploiting the game, basically, otherwise it's very much luck based
- a similar situation in the first Black Ops when you have to get down a hill between trenches: enemies never stop, on higher difficulties you need tons of luck to find a path to go through
- and another equally annoying one at the end of Black Ops III, where you need to access a door with neverending spawns of robots: it's a long battle either way, and at the end you take a leap of faith
My experience with Dark Souls in particular has been hit and miss. I think that in many cases the difficulty is very high, but also fair. However, there are situations that you honestly can not anticipate no matter how skilled or careful you are, causing you pretty much unavoidable deaths. When I first started Dark Souls, I went straight on for the skeletons. Took me hours to learn how to beat them efficiently and to get past them without problems, only to find out that path leads me to a mage that I simply can not beat with such low level items since he one shots me easily. That's what I consider unoptimal game design: nothing in the game suggested I shouldn't attempt that route, but the reward was an enemy I could only beat far later in the game, thus placing a difficulty block in front of me without any chance to foresee it.
Likewise, there are horrible examples like bullet sponge enemies. The reason certain parts of loot-based games like Borderlands, Destiny or The Division don't work too well is because the enemies are the same cannon fodders you've learnt to deal with dozens of hours earlier, but suddenly they have 100 times their regular health, so while the strategy doesn't really change it all becomes a battle of attrition, shooting from your cover for even 15 minutes if you're unlucky just to pop that guy's head off. That is difficult because his damage is also probably increased, however it is more than anything a battle of attrition, of patience. You don't need extra skills compared to the initial part of the game, you only need to concentrate for a longer amount of time. That is a bad example of difficulty.
So yes, "too difficult" can definitely be a complaint in a game, because there can be areas (or entire games, of course) where the difficulty comes for all the wrong reasons. The original Crash Bandicoot games, for example, allows you to finish levels just about how you wish, you can take your time and mind your own business. "Getting gud" is only required if you want to complete all stages 100%, therefore the difficulty doesn't really impact the enjoyment of the game itself. But if a game's difficulty is all over the place, it rewards luck rather than skill (see Mario Party games), if attrition is the main strategy, if the game mechanics are not explained properly which actually causes the difficulty to be there in the first place, etc., a high difficulty can easily be a complaint.