Seems like it doesn't fixes it for all games, maybe GOG hasn't update all links?
09:56:44 | xml parsing error occurred trying to get md5 data for setup_bioshock_infinite_1.0.1643565_(29748).exe
09:56:46 | xml parsing error occurred trying to get md5 data for setup_bioshock_infinite_1.0.1643565_(29748)-2.bin
09:56:46 | xml parsing error occurred trying to get md5 data for setup_bioshock_infinite_1.0.1643565_(29748)-3.bin
09:56:47 | xml parsing error occurred trying to get md5 data for setup_bioshock_infinite_1.0.1643565_(29748)-5.bin
09:56:49 | xml parsing error occurred trying to get md5 data for setup_bioshock_infinite_1.0.1643565_(29748)-8.bin
09:56:50 | xml parsing error occurred trying to get md5 data for setup_bioshock_infinite_1.0.1643565_(29748)-10.bin

That's pretty much exactly what I did as well. In my code it looks like this:

// $pos = strpos($fileUrl, '?');
// $XMLUrl = substr($fileUrl, 0, $pos) . '.xml' . substr($fileUrl, $pos);
$XMLUrl = $fileUrl . '.xml';

The two commented lines was the old version. Line 1 searches for the position of the first '?' inside the fileUrl and line 2 generates the XMLUrl by copying the fileUrl up to that position just before the '?', appends '.xml' and the appends the rest of the fileUrl. Replacing '?' with '.xml?' like Kalanyr is doing does exactly the same thing as long as there is only one '?' in the string which should be the case in URLs.

The third line is the new version which generates the XMLUrl by simply appending '.xml' to the FileUrl.

I have looked into this as well.

The servers seem to behave erratic right now (most likely because they are under stress from the sale) when it comes to delivering the XMLs. That's not the first time that has happened.

What I have observed is that some XMLs sometimes deliver a file only containing '.' or some random binary data. So I would not recommend doing an update right now.

Apparently the server sometimes responds with gzip compressed XML data
For lgogdownloader this was a simple fix to set CURLOPT_ACCEPT_ENCODING to enable decoding of a response when Content-Encoding header is received
From my own quick testing that seems to have fixed the issue for lgogdownloader

TY!!

I think there might bigger compatibility problems then this. I've had errors updating the 1mb installer of "Forgotten Fields" with my client and the response for the ".xml" augmented url gives me this in the headers:

HTTP/2 200
content-type: text/xml
content-length: 1
content-encoding: gzip

Yeah, content length of 1. That's quite the compression we're talking about here.

Hopefully, it's a transient error, else maybe getting the download checksum in offline installers will no longer be supported for some games.

Thanks. That gives me hope that this particular problem might go away.

Anyways, I guess if gog drops the ".xml" in the future, I can always fallback to my original plan of just downloading the the same binary at least twice and ensuring that the checksum match (combined with a size match on the expected content length, that should be enough).

I'll keep an eye on this and fix the URL/encoding detection, when things have stabilised. Thanks for the reports.

I did a bit of a hack on the original script from eddie3 which I've been using (I don't remember why I didn't switch to kalanyr's fork which looks more advanced).

And yeah, even after accepting the gzip content in response, which fixes a few more of the md5s for games (eg. technobabylon) there are still cases that are not fixed and seem to be an issue on GOG's side (responses seem empty). Such cases for me currently are Cyberpunk_2077 (most of its setup files, except for 9 or 10 of them), Serious Sam Siberian Mayhem (only for the .exe file from the new setup files), and No Man's Sky (the new setup files, but not the patch files).

If the only issue is how to verify the integrity of the downloaded files, I would presume it would make more sense to perform some kind of test on the compressed files, that they still decompress fine without errors (rather than downloading the same file several times and comparing their checksums; both files might still be broken if they were that already on GOG servers). The decompression test should guarantee they are ok. Sure it is a rather slow process I guess, but at least it should be foolproof.

Not sure how to automate that for the GOG offline installers, I presume it could be done with a tool like innoextract or whatever, and maybe it has a similar "test the integrity of the compressed installer" function as all compression utilities (7-zip, WinZip, WinRAR etc.) have.

For zip files, I do exactly that. I haven't explored the same with windows' innoextract (might be some overhead if I want to implement this in a self-contained multi-platform way, I'm not sure). Not sure about GOG's Linux "bash files" or MacOS' .pkg files (again, running well across platform in a self-contained manner).

I'll see how this situation progresses.

I'd think a mature http library/framework would handle things like gzip compression (assuming it is properly reported in the response headers) seemlessly.

I don't think I've had issues with gzip per say (just updated No Man's Sky and Techno Babylon without issues) as much as malformed xml content (ex: length 1 responses).

Doesn't gogrepo use requests?: https://2.python-requests.org/en/master/user/quickstart/#binary-response-content

It uses GET requests. And it's quite flexible. Again I think the issue is with bad responses -- but it could also be:
- that I get cached responses even though perhaps the issue is fixed. If that's the case it will resolve itself sometime in the future
- that I'm using 2.7 Python -- the script supports both Python 2.7 and 3.x but perhaps 3.x behaves better in such situations(?)
- my local fix was sloppy. I just looked at what the response would be at a web browser, used the dev tools to see response and request headers and tried to adjust the script to accommodate the gzip responses. In the remaining problematic cases that I've checked the browser do not shows anything either.

Oh no.. pls Gog don't mess up gogrepo +_+

I already had a hunch that the binary data wasn't quite as random as I first thought and could have been compressed data but I couldn't test that since the compression only happened rarely and I had no way of reproducing that in a reliable way. So thanks for confirming.

But that still doesn't cure the 1 byte or sometimes completely empty files. I do hope that issue automatically clears up as soon as the sale is over.

Perhaps there is caching on their end. On the client-side, browsers tend to cache things, but unless you go out of your way to do so, client libraries certainly won't between successive program executions.

I think you should definitely upgrade to 3.x at some point (2.x is end of life), but I think that problem would have been resolved for some time with requests so even if you are using an older version, I don't think this should be a problem.

Otherwise, what files are causing me pains seem to change.

I had issues with Strangeland and Simpleplanes yesterday, but not today.

Here are some files that are giving me consistent issues so far:

/downloads/the_longest_road_on_earth/en1patch0
/downloads/northgard_lyngbakr_clan_of_the_kraken/en1installer0
/downloads/forgotten_fields/en1installer0

I think I'll be able to dance around this particular issues should it persist, but otherwise, if it ever reaches the point where I can't backup my installers properly with a client, I won't see much of a point in buying things here anymore.

Sadly I agree, the ability to mass update offline installers is fundamental.
Doing it manually is unfeasible for large collections (even more due to the poor changelogs).
It's already absurd that we have to rely on third party scripts for that.