Another guy here who played Fallout 1 and 2 only recently, after playing 3. And, another guy chiming in with: It's not just nostalgia. These games are amazing.
To me, what really makes them great are the roleplaying options. You can literally create any sort of character you want and succeed. Some are more difficult to play than others, but that's what the game's all about. If I'm not mistaken, Fallout 1 allows you to beat the entire game without fighting anyone, if you're clever.
It was this myriad of options that really impressed me. Just about any situation, you can talk your way out of it, sneak around it, fight through it, or come up with something more cunning and situation-dependent.
The story is good. It's not great, but it's good. You can't really compare it to something like Grim Fandango or Planescape: Torment, but it's better than a lot of games out there, and if it's not life-changing it's not disappointing, either. And really with so many options for how to play your character, I'm surprised its as good as it was.
Combat, as others have said, is great as well. About as tactical as it gets in an RPG, and it becomes very tense at points. It's not so much difficult as it is unforgiving - specifically the critical hit system, where if some mook gets a lucky shot he can utterly destroy you. Still, though, to me that added to the tension, and made every battle feel "real" and threatening instead of just slogging through minions. You may be able to turn it off, or at least decrease the likelihood of it happening against you.
My one complaint was with the dialogue and characters, neither of which were particularly captivating. I loved the dialogue OPTIONS, don't get me wrong - one of the few games where saying different things actually affects what happens instead of just taking you down the same path in a different way. But the way they're delivered left me rather cold, being a bit terse and detached. Same thing with the characters. Never really came to care for them.
Still, though, those aren't nearly enough to detract from a game this brilliant.
Also, if you enjoy it, I strongly recommend checking out its spiritual successor Arcanum - different setting (fantasy world in an industrial revolution - you haven't lived until you've seen Orc bandits shooting down a blimp from makeshift fighter planes, or Ogre manservants in smoking jackets), but the gameplay, especially the roleplaying elements, are very similar.
You summed up my feelings pretty well. Excellent roleplaying games, but the dialog does have a tendency to feel a bit... Sterile, I guess?
And +1 for Arcanum. It is awesome. :)