Is there any list of Windowed games (GOG games) available?
Either exclusive or having graphic option to change to windowed.
I want games I can have open playing in a window while the rest of the monitor has a web browser playing videos etc.

You can try Infinity Engine games + Neverwinter Nights.

I assume most if not all DOSBox games can be windowed?

I believe most of Unity3d games have windowed options (at least every Unity games I own do).
If you use Linux, every Windows game can be windowed via Wine.
As tfishell previously said, with DOSBox too, so to tell the truth, it's been a while since I haven't played a game without the window option...

On https://pcgamingwiki.com, you can check if the specific game has native windowed mode support or not, but I don't think there is an actual list for that.

I guess it is like 95% of the games have windowed mode or you can force them to run in windowed mode.
So it is better to just the ask and check pcgamingwiki to confirm if the actual game you want to buy has windowed mode.

As I‘m just playing it, Broforce has an option to set a windowed mode.

Basically what bela555 said.