The maintainer (Kalanyr) has been busy with Real Life in recent days, but it looks like (from github activity) he's working on it now, so shouldn't be long.

Those of us unable to dive in and help directly just need a little more patience.

There's a partial fix up thanks to a pull request from IdamkinI.

From my testing it fixes *most* of the issue, but there's a few places where the GOG servers are behaving pathologically (allowing connections to time out rather than properly closing them). This seems to be related to the issue with some of the md5 data .xml files being empty (as it results in a similar file sometimes and sometimes in an access forbidden that isn't returning the proper response code) but it's not exactly the same, the case where the xml is just empty is handled by the parsing failing, whereas this isn't. Games I know have this issue at the moment are expeditions_rome, rimworld and stellaris , though there are probably others.

I'm currently working on a fix for that and verifying that it doesn't interfere with downloading the actual files and will fix that if it does.

I may have to put this on hold temporarily if the other project I was working on is running to schedule (I was a little ahead of schedule and had today free) but in that case I should get back to it later this week.

Thank you for the fixes.
Btw could you also try to report such problematic servers to Gog?
(I'm not sure I could explain the problem properly myself)

I intend to but it's related to specific files as best as I can tell, so having this temp fixed with a specific error message that people can report to me, will allow me to put together a list of files that need fixing.

Thanks for the update Kalanyr, very much appreciated. Today I had an update for Master of Magic Classic and was getting the xml parsing errors on your prior version of gogrepoc. With your updated version I got no errors and the download and verify went fine. Checked the manifest and the md5 was populated for the file that was downloaded "setup_master_of_magic_classic_cmfw_1.04.01_(54182).exe"

I wish it were that simple but sadly it isn't.

1) An XML that is not available now (or even most of the time) can randomly be available later if you just keep retrying.

2) An XML that is not available for you can be available for me or the other way round because we download from different CDN servers due to our different locations.

However, if you still want reports I can PM you the files that are problematic for me.

Added some additional fixes for this issue (the GOG servers are timing out when reading some of the md5 xml files)

If you see an error of the form
"Unexpected connection error fetching md5 data for ITEM. This error may be temporary. Please retry in 24 hours."
Please wait 24 hours and retry it, and if the error persists let me know what game and which specific item is having the problem, so I can report it to GOG.

There is one other rare error I'm aware of concerning broken encoding when downloading files but I need to do some more investigation before I fix it / request reports.

I have another project I have to get back to now, but I'll try and get back to this later this week.

ETA - Dropping in to note that the decode fail file appears to have been fixed, going to manually run a full update overnight tonight to see if I can find any other instances of that. If not I think, this is about as fixed as things can get for right now.

Yes, if the issue persists for you for over 24 hours let me know. That at least suggests the error is fairly wide spread throughout the CDN. Of the 5 I noticed, 2 resolved pretty quickly and seem to be fine now but the other 3 are persistently broken.

Thanks for your quick fixes. I still see some exceptions ie, for Adeptus Titanicus Dominus:
Traceback (most recent call last):
File "./gogrepoc.py", line 723, in filter_downloads
fetch_file_info(d, True,updateSession)
File "./gogrepoc.py", line 677, in fetch_file_info
md5_response = request(updateSession,tmp_md5_url)
File "./gogrepoc.py", line 230, in request
response = session.get(url, params=args,stream=stream,timeout=HTTP_TIMEOUT)
....
requests.exceptions.ContentDecodingError: ('Received response with content-encoding: gzip, but failed to decode it.', error('Error -3 while decompressing data: incorrect header check'))
09:17:15 | failed to fetch https://www.gog.com/downloads/adeptus_titanicus_dominus/en1installer2 because of non-HTTP Error

Yeah, this is the error that I mentioned before, seems like it does still exist for some files, so I'm going to have investigate precisely what data is actually being returned that causes the decompression to fail.

For me it is Dying light which is utterly fucked up.

https://drive.google.com/file/d/1xZBg4eSgN0Z-AVjM6-Ldm8QCe4OBDUIJ/view?usp=sharing

I'm running the command to create a new manifest as we speak - some files returned that "check again in 24 hours" error, I will try the command tomorrow and send you the list of files which return that error again, Kalanyr.

Do you also want the list of files or games causing different errors? (from reading the last page of forum discussion, I'm not sure either way and prefer not to spam you if you don't)

What seems to be the nastiest (or at least longest) one I have copied onto pastebin (it wasn't saved in the .log file, so I wouldn't be able to copy it afterwards) at https://pastebin.com/CCv4mbxy (I might have copied more than needed, figured too much is better than too little).

I don't want the gzip encoding error / bad header check errors yet, I need to verify if that's a python problem or a gog problem or a a gog problem they need to fix.

Otherwise yeah send through other errors that are persistent and that don't happen when you try and download the same file from the website yourself through the browser interface (I'll accept errors that do occur when you try and download them from the website for reporting, but I prefer them to be in a separate list)

That last error of yours I think is due to the connection being closed uncleanly, it's sent through a unicode sequence that isn't actually legal unicode so it can't be written to the log file (it's probably sent through part of a multi-bytte unicode character that isn't valid by itself).

Well, in good news, I think I've IDed the encoding issue, it looks like yet another kind of bizarre response to an xml request. I think it's sending through completely empty streams but marking them as gzip encoded which doesn't exactly make sense, you can't decode nothing.

I probably won't have a chance to fix this today but it's a lead at least.

You can pretty much either:
- Skip over the issue and do without for files that have weird .xml content
- If you can have a multi-location setup, try various locations and hope to hit a functional part of the CDN for that file

Okay, pushed an update that should fully restore parity with the situation before the GOG updates (ie it should work as well as the website), let me know if any downloads are still broken in exciting ways.

I've updated the logging/display a bit to more clearly indicate if an exception is handled / unhandled (a line of text before the exception dumping indicating if the exception was handled or not) and a line indicating when the dump is finished. Hopefully this should make it clearer which errors aren't a severe problem An exception without the wrapper is probably something extremely bad. I'll probably reduce the number of things that actually log detailed exceptions in the future, once we're clearer on the status of stuff.

The list of games that I've identified that have issues with their md5 data are

a_plague_tale_innocence
anomaly_korea
baldurs_gate_iii
bloodstained_ritual_of_the_night
children_of_morta
champions_of_krynn
craft_the_world
cyberpunk_2077_game
dragon_star_varnir
expeditions_rome
interstellar_space_genesis
iron_harvest_base
kingdom_come_deliverance
oriental_empires
phantom_doctrine
phoenix_point_year_one_edition
questron_ii
rimworld
simon_the_sorcerer
star_trek_25th_anniversary
star_trek_starfleet_command_iii
stellaris
stronghold_warlords
the_riftbreaker
the_universim
starpoint_gemini_3
tropico_6
warhammer_40000_gladius_relics_of_war

And something of a special case
imperator_rome

Which is completely missing files again, and those files include the installers.

graveyard_keeper is also missing a file but it's only a patch file from a very old version to a less old version that's still too old to continue the patch chain actually get up to date, so I'm not going to lose sleep over it.