I'm on 5th level of the dungeon and I have problems with weight management. I wear chainmail, I have some gold, keys, potions and quest items, and often I have to throw out potions or skip gold. But on one site they say "You will want to have a Strength factor of at least 25 in order to wear plate armor, and still be able to carry a fair amount of goodies." I have 25 and chainmail, and still it's bad. Am I doing something wrong?

Sounds normal to me.
You may want to get some high quality ("Unsurpassed Protection" I think is the best) Leather instead.

Be selective in what you carry.

Unless you know you're about to go spend gold (or gems), don't haul it around everywhere. Stash it somewhere central that you can easily return to.

Likewise, there's no real benefit to hauling around quest items that you won't need until later. Keep them in your stash. (Most of the talismans fall into this category: the candle, sword, and shield are the main ones you'll want to keep on you for use.)

Similarly, only carry a few wands/potions/scrolls that provide effects you might need in an emergency. The rest can be stashed until you actually want to use them. Food should be kept to one or two items (since you'll almost certainly find more as you go, and can conjure food when needed past the early stage of the game). Ditto for tools like the fishing rod.

--

I usually end up claiming the shrine near the entrance as my first "base", then move things to unused areas in the dwarf halls, the knights' chambers, and the wizard's academy as I move downward and become active in those areas.

Keep stashes of items on each floor and mark their locations on the map. That way you can carry around just the essentials and restock at your stashes when you need to.

25 strength is more then enough to run around in full plate mail and carry everything that you need around. The problem is that most people actually carry much more than they need.

Gold and jewelery is pretty much useless in UW1. You should drop some coins (maybe 50 or so) at Shak's to pay for repairs. No need to carry lots of it around. Also food: Don't carry much around. Remember: Create Food is a simple first circle spell. You need food? Create some when you need it! Also weapons: You don't need more than one. Maybe two if you're paranoid about it breaking in the middle of combat. You also don't need spare armor. Carry a light source, a few items to refuel it, a pole, a sleeping bag, maybe a few potions and wands.

That's really good advice; I'd add a few things:
1) Get the Taper of Sacrifice, a guy named Zak on the lizardmen level (lvl 3) is trading it. It's called just taper, but it's a candle that never burns out and provides decent lighting, it saves you the need to carry around spare light sources.
2) If your weapon of choice is swords get the Sword of Justice, it's a decent weapon and it never breaks, then you won't need to carry around spare weapons. You can even use it to break open doors
3) small items have weight as well, so don't carry around notes you don't need. use the map instead to write information down.
4) the same thing applies to keys as well, you won't need any keys on any leves other than the one where you got that key. It's a good idea to create a stash of keys after three levels or so.

Ultima Underworld is not just a dungeon crawler, it's a dungeon simulator. If you were thrown into a death pit in real life you wouldn't pick up every random thing you come across either.

EDIT: For the reference, I beat Underworld 1 with a strength of 19; it was annoying at times, but it was doable.