So I got the water chip and all that and thought that the game was over, but then I'm told I've got to stop the super mutants. That's fine and all but at this point in the game I am level 8 with leather armor (the metal armor gave me a lower armor class rating for some reason) and the best weapon I have is a double-barreled shotgun. The only companions I've managed to pickup so far are the dog and Ian.

I've been trying to play the game as you would without a guide (finding where to go next etc. by talking to people and such) and all the overseer tells me is to go west. So I go out into the desert and I end up stumbling upon either a pack of mutants, centaurs or floaters (or all of them together!). The laser rifle the mutants are carrying cut me in half after 2 hits. I've been in the mutant base before (got caught earlier before I reloaded, having seen it was impossible for me to get out) and there is no way I could survive in there.

How the **** would you know where to go without a guide? I mean I think I found the military base, but the mutant leader is in the cathedral. Nobody I ask about anything has the option to ask about the mutants and it seems like the only places I haven't visited are the glow, the military base, the cathedral and the brotherhood (I visited them, but they sent me off on a quest "to the south" (where I didn't see anything....)). Did I miss something vital going through the water chip quest or is the game really designed such that you can be so under-leveled and under-geared and get to the last few hours of the game, requiring you to backtrack to other areas and grind?

Old rpgs don't hold your hand like some modern ones do and they don't scale the enemies exactly so that you can beat them without too much effort no matter how strong you are.

You have to find out for yourself, if some area is too difficult for you go elsewhere first. You are allowed to run away, you are allowed to reload.

The brotherhood sent you to the glow. You should be able to do this easily at level 8 with chems to get 100% radiation resistance temporarily and a rope in your backpack.
If you do the Brotherhood quests you should end up far more powerful with even better equipment.

Metal armor is better than leather armor, the damage reduction is far more useful than AC bonus.
Armor does make the biggest difference in combat.

Yes, your main problem is that you skipped the Brotherhood of Steel quests (and possibly you skipped other quests before that, too). I will not say more, to avoid spoilers.