I tried to fend of 80 soldiers at once at the mine, but that simply won't work. I didn't see any way of accomplishing it with my gear without resorting to underhand tactics like putting everyone in the bar and waiting for the enemies to enter one by one.
It turns out though that they like to surround the mine from the east, south and west before attacking. And since it doesn't appear as if reinforcements can move diagonally, by attacking first you can effectively isolate 20 soldiers of the counter attack at a time. Doing this I've managed to wipe out all of them except for 20 that remain to the east of the mine, at the cost of temporarily losing control of the mine. Luckily the elites drop some pretty good gear, so you can upgrade and re-arm yourself as you fight them in this way.

Huh, I've never had that problem, or that number of enemies in a counter-assault. Have you tried actively hunting down patrols before they get the chance to stack to one big mob?