Gremlion: Corruption is a multiplier to your expenses.
No expenses (solo warriors, solo mages) - no corruption.
taltamir: That doesn't mean it can't bankrupt you.
Gremlion: For example, 2 executed deputies equal to zero corruption from human province.
taltamir: You can't unconquer a province though. And I was talking about large shards. so you conquer your initial 10-15 provinces, then you conquer "lines" to strategic resources as well as having to conquer some of the AI provinces, you end up with a ton of unwanted provinces that are bankrupting you via corruption unless you are playing solo hero
I've had colossal shards fully conquered with the exception of last lord, fully guarded against inquisitors, 8 heroes (6 with army) and corruption wasn't even close to being noticeable.
Your problem, imo, is that you don't conquer enough provinces. More provinces - more sites, more events with gold or possibility to decrease corruption, more slots for economical buildings.
Corruption is capped on doubling your expenses. After reaching breakpoint (all provinces together support doubled expenses) - each new province only adds income. You don't lose anything.
On a side note - you can unconquer a province by making them unhappy (buildings, guards, pillaging) and letting them go.