I noticed a teleporter at the lower lvl of town but when i got there i was attacked by slimes who killed my party most enemies just make you unconsious, am i too low lvl for that area? Mt fighters and robber are lvl 4 and my mage and cleric 3.
If I remember correctly, then yes, you're probably too low level for that area. You were teleported to a different cave with some tough enemies in it. Instead of going there, finish exploring below the town and then just leave town and explore the outside world first. You'll eventually return to that cave that you teleported to, but you don't want to explore it until you're higher level.