Cthulhu may be beyond the understanding of humans (which is also the description of the God of the Abrahamic religions) but that doesn't mean that they are beyond the human definition of evil.
Those entities want to assert control over, or just completely wipe out life on earth for their own selfish ends, despite any suffering they might inflict. Now you might try and equate humans to mere bacteria compared to the infinitely more complex minds and plans of a great old one like Cthulhu, but the fact is that even lowly human draw the line on inflicting pain and suffering on an entity that can sense it. Those that choose to not draw that line are called evil. So to go down that line of reasoning you basically have to be saying that the human concept of 'evil' does not exist, so the whole question is irrelevant.
The lovecraftian entity Azathoth is not evil imo, because it is not aware of what it is doing, it is basically physics. Whereas an entity like the coenobites know exactly what they are doing, and derive pleasure from trapping people (despite them consenting at first perhaps) and then torturing them forever, even when they ask for it to stop. If a person acted like that, would they be evil? Um, yes.
Azathoth for reference:
"Azathoth, sometimes referred to as the "Blind Idiot God", is a dreaming monster whose dream is where the universe resides. Azathoth is completely unaware of anything going on in the dream; hence his title. Azathoth also shifts in his slumber, causing reality to change. He is the most powerful entity, according to Lovecraft, closely followed by his grandson Yog-Sothoth, and is the creator and ruler of the Outer Gods."
