I can't decide... :(

I assume you don't mean Patrician IV vs Port Royale 3

I liked them both a lot. Though I put more time into Patrician 3 I imagine. I wouldn't be able to explain why though or what makes one better, I like them both ^^

Well, I just mean the series' in general. I wanted some city building action and they're both made by Ascaron and they both look VERY similar. Port Royale 2 almost looks like Pirates! with city building elements (which is awesome).

Patrician because of the setting (I always choose Lubeck as home town) and because it's a tad more about open world and a bit less about fullfilling missions. Otherwise they are extremely similar.

They are NOT city building games for the most part - the heaviest focus is on trading, with the possibility to build your own businesses after some time (which, by the way, is a must if you want to have a reliable flux of goods), but that, and some optional city enhancement buildings is all that you can build.
From the newest installments (Patrician 4 and Port Royale 3), PR3 is way less frustrating (for example, you don't have to worry about an annoyed prince besieging a city). Also, from the classics, PR2 is probably the easiest one, so if you fancy a trading simulation but you're a complete newcomer, I'd suggest getting either Port Royale 2 or 3.