purpleprince: I mean, does you tagging the building with your flag have any other effect, negative or positive, than you (and your enemies) will know that you have been there before?
1) You maintain vision of a small area around it, even if something restores the fog of war (like the Cloud of Darkness).
2) It might distract incoming enemies that stop to flag it.
3) Perhaps most importantly, it gives each city with a matching dwelling a +1 creature/week growth bonus. This bonus is most apparent with the higher tier monsters. Say you have two Ramparts, one with a castle and one with a fort, both with the Dragon Cliffs. The castle will produce two dragons per week, the fort will produce one dragon per week. If you flag a dragon cliff map dwelling, then your two Ramparts will produce 3 and 2 dragons per week. If you flag 5 such map dragon cliffs, you'll be getting 7 and 6 dragons stockpiling each week. This does NOT count the dragons you can get by visiting the structure itself.
If you flag a behemoth map dwelling, then your Ramparts get no such bonus.
There are no negative effects unless you are trying to keep enemies from knowing you've been there. I can't think of a case where this will actually be useful, since there is no fog of war that doesn't also cover the terrain.
The only other possible negative is losing units in combat when taking control of such a structure, but that is more about the combat than about owning the structure.
Edit: I suppose if you DON'T want to stockpile more creatures in your towns, then that isn't actually a bonus. Also, you only get the bonus creatures in the city dwelling if the city has actually built the dwelling. Flagging dragon cliffs doesn't help a Rampart that doesn't have the Dragon Cliff building built.