i don't have the gog version, i bought my Duke CD a long, long time ago and I use eduke32. I mention this so my next statement doesn't seem as stupid as usuall.
Isn't there a dosbox shortcut created to run the setup.exe, and not just the duke3d.exe, when the game is installed?
make a copy of the dosbox config file that runs duke3d.exe. edit it with notepad and change all instances of duke3d.exe with setup.exe.
Firstly, you must run the setup.exe within DOSBox as it isn't a native Windows program. GOG have most likely provided a shortcut that runs setup.exe within DOSBox, so this should be used.
Secondly, on Vista and 7, if you install into Program Files or Program Files (x86) then UAC and DEP may prevent the settings from being saved as these are protected folders and Windows tries to prevent files from being altered in those locations after the program installation. So if you have installed to either of these folders make sure you use the Run as administrator option (even if you are an admin user). Or just install outside of Program Files.