Swampland: Time limits.
May it be a countdown to a bomb detonating or the Fallout thing where you have to finish the game in a set amount of time. There are also some missions in the original StarCraft like that. I can't stand that added level of stress.
To me especially the game-wide time limits (like in Fallout and System Shock) are a problem because how the heck can I know in the beginning, how much in a hurry I am? Should I beeline to the exit of each level or area as fast as I can? Or do I have time to search for secrets, fight enemies etc.?
If I am 90% done with the game and find out at that point that I don't have enough time to finish the game, what then? I guess the only option is to restart the whole game from the start, and just be quicker about it.
Now, I've understood the time-limit in Fallout is quite lax and not really a hindrance, and you can even stretch it... in that case, why is it there in the first place, if it won't really affect anything? Anyway, I guess it was removed in some patch, IIRC.
In Starcraft it was used only on certain special missions, but yeah even there it was quite irritating. I had to restart some missions from the very start, just because I barely missed the timer near the end of the level. At least I didn't have to restart the whole campaign, though.