So I've been playing 8 for a few hours and has gotten to around level 22, but I've realised that the game is very boring.
The gameplay loop consists of going to a new map, spamming arrows or magic at enemies, running away to not get hit or some status effect I can't cure yet, spam arrows some more, clean up the bodies, gather loot, go back to town, repeat. And I'm not having any fun anymore.

My biggest problem is also that the story is almost non-existent. I'm not looking for something life changing, but what I like about Oblivion, Skyrim, New Vegas, and Grimrock is the small stories a dungeon or a questline can tell. Finding scraps of journal entries adds fun flavor to the exploration. It's unrealistic that so many people are losing their journals, or ripping out pages and leaving them as bread crumbs, but the mechanic works for adding flavor to a task that can feel pretty sterile otherwise.
Are the other games any better? I've read that the M&M games are pretty light on story, but is it as bad as this game?

short answer: no
long answer: old games probably ain't for you, the focus was more on discovering new gameplay elements and doing it better than the previous games rather than story.

8 is perhaps the weakest entry of the 6-8 set, but the basic gameplay is the same in all 3. The focus in these games is on exploring the locations, murdering the monsters, and taking their stuff. There are quests (mostly text based tasks that send you somewhere to do or get something), and some story cut scenes, but 6-8 doesn't have random bits and bobs strewn everywhere like the Elder Scrolls games do. Various NPCs have things to say about the world, but there isn't nearly as much text as in Oblivion, etc.

That's too bad. I've been looking for another open world dungeon crawler, but I've realised that without any context to why I'm clearing out a dungeon and snatching all the valuables I get pretty bored.
Thinking of trying out "The Quest" some time in the future, perhaps next time it goes on sale or something.

What context there is in MM 8, much of it comes from meeting again, or hearing about what is happening to, characters from the series (including the Heroes of MM strategy games), particularly HoMM 2, MM 6, HoMM 3 and MM 7 (that is the narrative order).

There are quests that send you into dungeons (ex in MM8, exploring the Troll's Tomb or clearing Balthazar's Lair), but if you don't enjoy exploring for the sake of exploring and seeing the sites, killing the monsters, and taking their stuff, then this game may not be for you. If it helps, you can skip a lot of the overworld monsters once you have access to Flight (whether by Air spell or Dragon 'spell'), or kite them into a tight cluster and meteor shower/starburst them to death (depending on the monsters). Armageddon can also do wonders to clear an overworld map of monsters.

It is wide open. But you are kept on track through the Oracle if you start feeling lost.
A clever way to keep things moving forward.

What do you mean there's no story? A terrorist caused a large cataclysm in the middle of one of the largest towns in the region, causing portals to the elemental planes to open and spew forth dangerous creatures. It's up to you to discover the nature of the large crystal structure and restore things back to normal. Combat also gets better once you level up a bit more, so that you don't have to rely on kiting individual enemies away from a large pack and filling them with arrows.

From my recollection MM8 doesn't actually have that many characters who carry over from the other games. I think these are all of them.
Elemental lords ( HOMM3: Armageddon's Blade)
Roland Ironfist (HOMM 2, MM7, Armageddon's Blade)
Catherine Ironfist (HOMM 3, MM7)
Sandro (HOMM 3: Shadow of Death)
Thant (HOMM 3)
Sir Charles Quixote (MM7)

The wererat and dark elf factions are only in MM8 and the Trolls, Dragons, Lizard Men, and Minotaurs, which were mostly generic monsters in the previous games, got towns, quest givers, etc. The pirate faction is also new, although it's mentioned quite a bit in the other games.

The story is largely self contained. Playing the previous game will give some frame of reference but it really isn't necessary to get what's going on.