It bugs me a little bit too, but in saying that I don't think he has fought any truly 'regular' human dudes?I know I shouldn't get bogged down by stupid nerd power level stuff, but it keeps bothering me how Loki, an Asgardian who has fought Thor hand-to-hand and has cleanly overpowered Captain America, keeps struggling when fighting regular human dudes.
The TVA might be variants from Earth, but it would make sense for them to have enhanced strength for the job. The people in the RoxxMart were being possessed by Sylvie which you can also rationalize as magically making them more powerful. Plus Loki very visibly uninterested in those fights anyway, and just wanted to speak to Sylvie. And in this episode, while the population of Lamentis appear human...they aren't. They could very easily be much more physically powerful than actual humans.