Baldur's Gate 1:
Right-click Baldur.exe, click on the Compatibility tab, tick 'Disable desktop composition' it can take a while to load, but this should make joining and creating games work. (Will not work with multiple monitors connected)
Baldur's Gate 2:
Tick 'Use 3D Acceleration' in configuration, then it takes a while for a multiplayer game to start, but it works.
Fix for both Baldur's Gate 1 and 2:
Before creating or joining a multiplayer game, press Alt+Enter to window the game and then press those keys again once the game has been created/joined to go fullscreen again.