Ultra_DTA: I'm setting up one of my older computers as an older gaming pc. I'm going to be installing Windows XP and trying to play older games for Windows, mostly from the mid 90s to early 2000s.
If you have USB ports in that machine, I don't think you want to buy older joysticks/controllers with older game port plugs.
Just buy a good USB joystick/flightstick from a store, and you are all set up for those old games which need a joystick. Thrustmaster T-16000m is a good one, Logitech Extreme Pro 3D is probably also fine (a bit cheaper).
The only reason you'd want an older gameport joystick is for DOS games (to run them in real MS-DOS on and old PC, MS-DOS never supported USB devices I think), but for DOS games you'd use DOSBox anyway, which lets you use modern USB joysticks fine with old MS-DOS games. All Windows games (Windows 9x and newer) should run fine with USB joysticks, even running them in e.g. Windows 98SE.
Also, if you see them somewhere, forget about those ancient Gravis PC GamePads:
http://en.wikipedia.org/wiki/Gravis_PC_GamePad They were more like a gimmick (I had one, but in the end used it very little), no PC game really ever needed one but were just as playable with e.g. the keyboard. If you want to play old games with a gamepad, get some newer (preferably DirectInput, but I guess XBox360 gamepad works also) gamepad instead. Analog joysticks though, they are very much needed with many flight sims, space combat games and racing games.
tl;dr:
If you have a mouse, keyboard, and an USB flightstick/joystick, then you are all set for playing old PC games (as far as PC gaming controllers go).