I attempted to find a peaceful solution to the Barrowholm situation... and then, later on, I found out what their deal is.
(I think it's rather stupid I can't turn against them right then and there, what exactly is stopping me from shooting them where they stand?)

Looking things up on the Internet (I don't mind spoilers about that particular faction at this stage), I found out it's possible to attack the Ravager when he is summoned.

How tough is he (I'm playing on Normal, non-veteran difficulty, currently I'm at level 64, Commando), and what happens if I manage to kill him (or even just attack him and die), am I still allied with the faction, or do they become hostile?

Ravager is one of the most difficult foes to fight.

https://grimdawn.gamepedia.com/Cast_Off_the_Flesh

Since I've never heard you can attack him while being allied I've no idea what may happen.

Just stay friendly this time around and then choose to go hostile with them when you move on to the next difficulty of the game.