If I find a better wife, can I just leave the existing one? Are there any penalties for divorce, aside from the gold loss, or can I just change wives as much as I want?

Kids.

You can change wives to enhance your playing style as each give different bonuses and each can carry different items (armour, boots, artefacts, etc.). I can't remember though if you can regain a wife after divorcing her if you change your mind later. And then there's the question of children, each one you have will take up an equipment slot so you'd need to decide which is more important to you :)