Dumb question...Don't know if I should ask here or on github, so doing both...

Is there a simple way to download EVERYTHING. Every language, every installer, every os everything?

I keep trying different cmd options and all I get is the bonus files. I know there's a bash script over on github that should sorta work only it didn't quite do it.

Thanks

I haven't tried it, but without having to enumerate all languages, it looks like it might work to call update without the -lang option and use an invalid language with -skiplang (e.g. -skiplang none, "none" just serving as something not in the built-in list of internal language identifiers).

It looks like the same trick doesn't work for OS as there's an additional check(?). But there are only 3 to worry about (not likely to change often) so using -lang windows mac linux should suffice.

I think you can omit the -lang and -os parameters with the download command, as it'll just default to all the variations stored in the manifest generated by update.

Might be worth a few extra lines of code to implement something like -lang all and -os all.

EDIT: I can't remember if those were deliberately not implemented to be kind to gog's servers.

Thanks. That's pretty much what I (and Kalanyr when he replied on github) thought.

I found a really stupid hack that for now works around the bad XML issue for me at least - if you don't auto-follow the URL redirects, you will finally end up with an URL that looks like this:

ht tps://gog-cdn-lumen.secure2.footprint.net/token=nva=XXXXXXXXXXXXXXXXXXXXXXXXXXXX/secure/offline/17167 51705/1716751705/55280881222617076/6349/patch_tunic_1.0-T1280-B23_%2854397%29_to_1.0-T1298-B24_%2854414%29.exe .xml

If you then replace the last slash with a double slash so it becomes

ht tps://gog-cdn-lumen.secure2.footprint.net/token=nva=XXXXXXXXXXXXXXXXXXXXXXXXXXXX/secure/offline/17167 51705/1716751705/55280881222617076/6349//patch_tunic_1.0-T1280-B23_%2854397%29_to_1.0-T1298-B24_%285441 4%29.exe.xml

it invalidates whatever bad caching is going on in the CDN, and gives you a proper download

I'll try that but that is difficult to reproduce because you never know which URLs are going to be problematic right now. I just tried several URLs that were problematic in the past and they worked just fine right now.

So I deleted my whole database and started getting everything from scratch to find which files have non-working XMLs right now and try that method on them.

I'll report back with my findings later.

Two thumbs up, works like a charm!

I had 89 faulty XMLs on the first run, then made the code change to include the extra slash in the URL and retried the faulty XMLs. All of them downloaded just fine even without any tricks like a VPN to get a different CDN.

Hmmm. Interesting. I should theoretically be able to automate this. Though that might just result in bad caches being generated with sufficient use.

Wow, nice. I'll have to tryout this one then.

Sadly it doesn't work as perfect as it seemed. I deleted my database again and tried getting everything again from scratch with the double slash method to check if this method doesn't make any problems with other XMLs. And it does make problems.

It's still a big improvement though. There were only 6 unreadable XMLs which I had to get using the usual CDN server hopping shenanigans instead of 89.

Can you --- or someone else --- please post how to incorporate this into gogrepoc?

If you have time could you show how you did that, would love to give it a shot.

Right now, it takes me ~12 hours to generate a manifest and someone else reported ~34 hours (with the workaround method of computing the hash twice when you can' get the metadata).

I'll take any performance improvement that I can. I'll give it a try this weekend.

Yeah, I also thought that might work. So I tried that right now.

But it is still a crutch since there is no guarantee you will get the XML after X retries and you have to give up at some point. I settled for 5 retries (after the first failure, so a total of 6 tries) and according to the log some files took up to 5 tries.

I did get everything without resorting to CDN server hopping by adding one more slash on every retry though.
Twelve hours???!!!!?!?!

And I thought 2:15 was slow...

How big is your library?

Around 2500 games (I played about 20% of them to various degrees so far, though admittedly, I'm unsure if I'll play them all, its a toss at this point that have led me to become increasingly selective about my purchases here). The bulk of that time was spend computing md5 checksums for files with bad metadata.

I think it is also contingent on your location, because somehow, I don't see the user that had to wait ~34 hours having ~7500 games in his collection (although I do cut Mac installers and installers that are not English, French, Spanish or Japanese so it would take even longer if I included those as well, but even so, it doesn't explain the difference in time)