So, just to be clear, am I the only one with this issue? I have seen no mention of it on the net.

youtube.com/watch?v=5QjsghyrUOA
(I can't seem to be able to post a link)

It's just always there, when I look up or down, the perspective gets screwed. It may not sound awful, but it makes my brain hurt.
I tried with DarkXL, but the issue remains.

I had this issue when playing Brutal Doom in the past, but I solved it by using Hardware rendering. I have tested a few options in the DOSBox config file, but I'm always getting the same results.

Any help would be much appreciated, I can't actually enjoy this game in this state.

That's just how it worked back in the day. The game isn't actually fully rendered in 3D, which is why FPS of that era are often referred to as "2.5D". In the original Doom you weren't even able to look up and down, whereas in Duke Nukem 3D it was the same as with Dark Forces, the perspective would "warp" when looking up or down.

The only way to "fix" this is porting it to a new engine that renders the game in full 3D, which I thought DarkXL does. But, it probably won't ever be finished.