I felt like in a Monty Python movie.

While my main hero was trying to engage the other masters, they raided my provinces from the inside about every fifth turn. The enemy masters did not take a single province from me, but these took about ten and they spread, other than Barbarian Tribes for example.

1xExecutionor
2xPikeman
1xHealer

They take down 100%:
Pikeman Guards
Foresters
Forest Pixies
Recruits

The only thing that stopped them and available at this stage of the campaign, were Forest Keeper.

My exploration warrior at home was not able to re-raid, so I had to move the main hero forth and back every the time they took another province. All my provinces were heavily guarded in the end and not because of the enemy masters lol

This only started about mid turn of the shard, did I make someone angry or is this normal?

According to Tumblr, their trigger is "infidel scum".
Joke aside, there is an event, "Inquisition asks permit to search for witches".
You can let them (-population - mood, -karma)
You can deny (+mood, +karma), inquisition would start attacks
If you have guards you can kill them and
a) hide bodies (20% that inquisition would start attacks)
b) show off heads (80% that inquisition would start attacks)

Grest, thank you.
There are sometimes many events with the more provinces you rule and I start only reading the answers, trying to stay true to a trend, i.e. no witch hunts.