So I logged out of Galaxy but now every time I restart my pc Galaxy has an annoying log-in flashing in the taskbar.
Is there a way to stop this or is deleting Galaxy my only choice?

Open Galaxy - Open Settings - Go to GOG Galaxy Client General Settings - Under Convencience disable Auto start (uncheck the Lauch GOG Galaxy at system startup prompt)

You can un-install the galaxy client completely, since you don't need that to access your games.

It was not an autostart issue as that box was not checked. It was under the gear icon/settings/general for me. Uninstalling may be the way to go. I don't remember the reason I installed it in the first place. If it is going to annoy me then that sounds good to me.

Some free game codes in exchange for trying Galaxy out, or that it looks like the download for the game rather than for galaxy when they did that sudden switch and then hid the 'old decrepit depreciated offline downloads' in another tab entirely.