Personally, I don't really like playing games in Windowed Mode. I guess it can be practical and allow you to do other things on the side while gaming, but I don't really want to do other things when gaming, I want to be fully immersed into the game and I find Windowed Mode immersion breaking. Still, occasionally games don't offer Fullscreen Mode, or, as I just had to discover with Noctropolis, they have issues when running in Fullscreen Mode on certain systems (in my case Win 8, despite GOG listing it as supported).

My question is this: Do you have any tips on how to make Windowed Mode less immersion breaking? Like removing desktop UI elements, running the window on a black background, maybe hiding the window frame, something like that? Or do you just not care about all this and ignore it?

I believe I never really thought about immersion breaking regarding windowed mode, since I usually find it very useful, and play a lot in windowed mode, especially old games, so I am a bit uneasy to answer your question.
In the other hand, I think it could be interesting to force mouse locking to prevent immersion breaking, as I do sometimes in DOSBox and Wine. I think mouse locking is a mandatory option for several games, those games which have map scrolling on borders, tactical or strategy games (Paradox games, for instance - games which have limited resolution of course; for the ones which can be me tweaked or the ones which can be played fullscreen, there's no question). But for few others, the ones which have a more static stance, not have mouse lock can be more useful to do something else like keep notes on a file (statistics, resources rates, etc), but again, here comes immersion breaking.

You may add custom screen resolution to that. Worked for me on windows.

I feel similar.

Go to http://e-try.com/black.htm and press F11, make you game the only windows in front of the browser.

I don't know how to get rid of window borders and titlebar though.

I think I have used DxWnd for borderless mode and some other similar tool I can't remember now.

You can shut down the desktop. That should work in Windows too. If the game's window does not fill up the screen, all you will see around it will be a black void.

some games may allow full screen but not explicitly say so in the settings.

Try alt-enter to go full screen.

Thanks for your input! It's mainly the visuals that I find immersion breaking though, not related to mouse movement. Could you specify how that works and to what end you used it? I'm not tech-savvy enough to imagine how that would solve my issue ... The description on the website only talks about turning fullscreen to windowed mode. Can you also turn windowed mode into borderless window mode? The problem I have with Noctropolis is that in fullscreen mode there is no text display, so I have to change the cfg file to run the game in a window. True, but unfortunately that doesn't solve my current problem with Noctropolis (see above). How would I do that? And what does it mean? I take it this would lead to the same outcome as toxicTom's suggestion? Cool, thanks! In addition to this, I set the taskbar to hide automatically, so now the window frame is the only distracting element on a completely black screen. I guess I can live with that.

The desktop can be shut down temporarily by pulling up the processes list and turning the program off. Sorry cannot remember now what it was called for certain but maybe explorer.exe BUT if you got an easier method, better use that instead.

I know you can hide the borders of a window but no idea how to on Windows.

Windowed Borderless Gaming - http://westechsolutions.net/sites/WindowedBorderlessGaming/download

or

Borderless-Gaming - https://github.com/Codeusa/Borderless-Gaming/releases

I use Borderless Gaming and it works on every game I have tried, can setup a custom window size too and you can make it so it does all this automatically when you start the game. I find it very useful for games that don't have borderless mode when gaming on my TV. You need to make the game run in window mode first for it to work though.

That's even better, thanks!

I own quite dated laptop (nvidia bloody optimus™) and setting custom resolution letted me to enjoy some newer games at fullscreen mode (e.g. 1024x576/600). The very first tool comes to my mind is PowerStrip, but that was long time ago.. Now it is possible to do make a custom resolution directly in a driver settings:
How to Create Custom Resolutions on Windows 7, 8 or 10
As for DxWnd, messing around with settings should help. Feel free to try Pherim solution: Anno 1701 SD: Quest for Borderless Window
There is also Borderless Gaming I've never tried.

I haven't use Windows for a while..

Thanks! I think I'll go with Borderless Gaming for now.

Yes, indeed it's explorer.exe.