I have a rift and did realize that games that I own, like solus project and everquest, do support VR (oculus and vive).

Although I have no idea how do you run these games in VR. When I run the games, they just appear on my monitor; there is no apparent option to run them in VR, like you would do on steam.

No clue. I wanted to check whether I have any games with VR support but it seems we cannot search for that neither in library nor in the shop.

I was able to find games by searching for the VR term; there is no filter on GOG for VR games but you can still use the text search, or check on Steam.

My question is more like "how do I run a vr game"; since I see no way to trigger the game from Gog Galaxy. I guess I have to open Oculus app and then run the exe from there?

Try having Steam VR running, and see if openVR is kind enough to make the game run in VR. Often, this does the trick.
Just remember to enable "Unknown Sources" in the Oculus software first.

After I got into VR, I more or less abandoned GOG in favor of Steam. But I did buy a few GOG games with VR support:
Scanner Sombre
Everspace
Obduction
Pollen
MIND: Path to Thalamus Enhanced Edition
Descent (2019)
Sublevel Zero Redux
Doom 3: BFG Edition (require mod)