I wanted to set up a Duke3D Deathmatch on our home LAN. The challenge is that only one of my machines is running XP (work was throwing it out since it is somewhat obsolete, but I grabbed it to play some GOG games). My other two machines are much more up-to-date, and in fact are running various flavours of linux.
Firstly, I changed my linux dosboxDuke3D_server.conf file and added the line
IPXNET STARTSERVER 12346
just before the setup.exe line. (It's right at the end of the file). I choose a higher port number, as for some reason lower port numbers did not seem to work in linux. No idea why, but a five digit number did the trick.
Then run dosbox with the server.conf file. You should get some messages advising that the IPX server is working, and then the Duke setup will appear. Before you go to network game, go to the sound setup and choose "no card" for both the sound FX and music. Yes, it is somewhat odd, but if I do not shut the sound off, the game crashes. Note that this only occurs playing on my LAN. When playing solo, the sound (AWE32) works fine.
Go to network game, choose whatever map you want, name, etc, and then launch Duke. A screen should appear, waiting for another player to begin.
Now, over the XP box. I opened the dosboxDuke3D_client.conf, and added 192.168.0.10 12346 behind the IPXNET CONNECT command. The first group of numbers is my linux machine's LAN IP address of course; the other number is the port which corresponds to the command I added to my server .conf file. I was unable to use the GOG multiplayer app on the XP box, since I could not specify a port, just the IP address.
Now, simply run dosbox with the client.conf file, shut the sound off (see above) and go to network game.
Everything should connect and - in my case anyway - my son proceeded to kick my Duke ass! Sadly without sound, but still - better than nothing! I might try tweaking a few things, like different sound choices, or different port numbers, to see if I can get sound working.
Because I do not claim any great knowledge of networking or linux...I just wanted to share what worked for me, just in case it might work for someone else, or provide them with a pointer in the right direction.