It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
avatar
Tarhiel: Well, for some mysterious reason, it did prevent me from downloading the games themselves.

But I managed modify my own file for creating .bat files for individual games to first do the update of manifest info of said game (with -ids) and THEN start to download it - and now it works.
avatar
Kalanyr: It's this: chunk request has unexpected Content-Range. expected '0-617406559/617406560' received '0-617406559/632833656'. skipping.
That was the error that stopped the download.

If you see errors of that form, it's best to do a quick chest for updates to the game and if there isn't to manually force a refresh with -ids.

That happens when GOG does a stealth update (they change a file, without marking the game as updated or changing the file name), it's a result of the data in the manifest being for a different file (so it's expecting a different file size). It's actually unrelated to the XML error.

(There are very rare occassions when the update won't help though, because GOG actually has the wrong metadata for the file. )
Good to know, thanks for the explanation.
avatar
mrkgnao: Update #1: I continued testing the games that had XML failures on a daily basis. A week or so later, of the 16 games that had failures for me, 9 "corrected" themselves, 7 not yet.
Update #2: I continued testing the games that had XML failures on a daily basis. After more than two weeks, of the 16 games that had failures for me, 12 "corrected" themselves, 4 not yet.
Post edited April 11, 2022 by mrkgnao
Hi. I got some messages that files are orphaned. But they shouldn't have been as they are genuine game files that are listed on GOG web page for download and not some old games files that were later updated:
19:39:51 | orphaning file 'gabriel_knight_2_the_beast_within\setup_gabriel_knight_2_-_the_beast_within_1.1_(20239)-1.bin'
19:39:52 | orphaning file 'il_2_sturmovik_1946\setup_il-2_sturmovik_1946_4.10m_(14392).exe'
19:39:52 | orphaning file 'il_2_sturmovik_1946\setup_il-2_sturmovik_1946_4.13.4m_(14395)-2.bin'
I do not think this is a gogrepo.py problem, because only some of the game files (for example 2nd file of 3 files; always the same files on multiple tries) are moved to orphaned. But I have nowhere else to report the problem so I am writing it here.
Best regards, D.
avatar
faiyazahmed: Hi Everyone,

I am a novice at this, but through research, I was able to successfully create a manifest file

Is there a more user-friendly way to read it? For example, is there a way to import it into Microsoft Excel?

I didn't happen to see any commands that gave this type of summary information within the cmd window. I am using gogrepoc, on Windows 10.

I am trying to find out what total size of my library is. It would be great to be able to get a list of windows installers, and how large each one is.

Thank you in advance!

EDIT: I was able to get some basic information by using the -dryrun flag with the download command, as follows:

python gogrepoc.py download -dryrun -skipgalaxy -skipextras -os windows -lang en

I ran the command with and without the -skipextras flag, and got two different results (which makes sense).
avatar
Kalanyr: The manifest isn't really particularly meant to be human readable, it just kinda is. I do have a script somewhere that ingests the manifest and spits out yaml, I can dig it up if that would be helpful for you. It's still not precisely human readable but the different formatting might be more natural depending on what you're trying to process.
Thanks Kalanyr for all of your great work! No need to dig up the script. The "dryrun" mode of the "download" command was sufficient to give me the main thing I was looking for - the total size of my library.

Fortunately, my library was small enough to fit on the spare hard drive I had, as long as I limited the content to English, Windows, non-galaxy installers.

I was successfully able to perform the download. I ran into some sort of issue about a quarter of the way through where the download process stopped (wasn't making progress). However, I ran the command again, and it was able to continue without having to start all over, and ran for many hours uninterrupted until completion. I had set my Windows power settings to NOT put my computer to sleep beforehand.

Thanks once again!
avatar
DSmidgy: Hi. I got some messages that files are orphaned. But they shouldn't have been as they are genuine game files that are listed on GOG web page for download and not some old games files that were later updated:
19:39:51 | orphaning file 'gabriel_knight_2_the_beast_within\setup_gabriel_knight_2_-_the_beast_within_1.1_(20239)-1.bin'
19:39:52 | orphaning file 'il_2_sturmovik_1946\setup_il-2_sturmovik_1946_4.10m_(14392).exe'
19:39:52 | orphaning file 'il_2_sturmovik_1946\setup_il-2_sturmovik_1946_4.13.4m_(14395)-2.bin'
I do not think this is a gogrepo.py problem, because only some of the game files (for example 2nd file of 3 files; always the same files on multiple tries) are moved to orphaned. But I have nowhere else to report the problem so I am writing it here.
Best regards, D.
I would recommend doing a manual update of those games using -ids and then letting it download the files again. Getting orphaned by clean means the files either aren't in the manifest (possibly the name changed) or have the wrong file size (which means the file has probably been corrupted).
Hi Kalanyr,

Thank you for your great work, really appreciated!
I have a feature request to create a switch / switches that will allow only single architecture downloads, e.g.: -singlearch_prefer_x86 and -singlearch_prefer_x64
- or -
-singlearch with -preferx64 (default) and -preferx86.

I think these are self explanatory, but just in case:
I'd like gogrepo to avoid downloading both x64 and x86 installers - I prefer x64 only (Win7 fan here), but just to make this tool versatile it would be nice to also have x86 preference option (for XP lovers, I guess).

BTW is there a way to donate?

Cheers!
Post edited April 15, 2022 by KoЯni
avatar
KoЯni: I have a feature request to create a switch / switches that will allow only single architecture downloads
I'm not sure if Gog can differentiate them in the xml.
Btw I avoid duplicate 32bit versions with this:
-skipfiles setup_*_32_* setup_*_(32bit)*
Post edited April 15, 2022 by phaolo
avatar
KoЯni: I have a feature request to create a switch / switches that will allow only single architecture downloads
avatar
phaolo: I'm not sure if Gog can differentiate them in the xml.
Btw I avoid duplicate 32bit versions with this:
-skipfiles setup_*_32_* setup_*_(32bit)*
Yeah, you're correct. GOG doesn't flag these, you have to use heuristics like these (I wouldn't necessarily recommend that first one as something you use passively (it may catch some build identifiers) but the second one should be fine 99.9% of the time).
avatar
mrkgnao: Update #2: I continued testing the games that had XML failures on a daily basis. After more than two weeks, of the 16 games that had failures for me, 12 "corrected" themselves, 4 not yet.
Update #3: I continued testing the games that had XML failures on a daily basis. After more than three weeks, of the 16 games that had failures for me, 13 "corrected" themselves, 3 not yet.
avatar
B1tF1ghter: At least for me and a number of other people the Kalanyr's fork has less errors.
It's also actively worked on (contrary to the original that was forked from) so I encourage you to try it (WITH the manual fixes).
If you decide to do so I also personally encourge you to use the dev branch as it has the resume functionality (very useful for anyone with unstable network connection, also useful for squeezing that extra download in short free time).

https://github.com/Kalanyr/gogrepoc/tree/dev
Many thanks for the link. This thread is imposing enough without searching the whole thing to try and find the "current state" (I've no idea what happened to the OP).

Hmm... interesting. A definite Windows user here and not a Python guru, but... with 288 games on GOG this could be quite interesting indeed!

Thanks also to Kalanyr, of course, for forking and updating from the original gogrepoc
avatar
phaolo: Btw I avoid duplicate 32bit versions with this:
-skipfiles setup_*_32_* setup_*_(32bit)*
Thank you phaolo, very good idea indeed! I have manually confirmed that both these filters match what I needed from my 825 games sample.
That being said, I also realized that these files constitute only 48 GB out of my 2,58 TB collection, so I gave the idea up entirely - just in case anyone else is concerned about it.
If I understood the discussion right, there are issues with various installers to get valid md5 checksums from GOG servers, which are needed for e.g. verifying your GOG installers (with "gogrepoc.py verify")?

Has anyone yet tried to verify their GOG installer collection by using innoextract? By experimenting with it yesterday, all that is apparently needed is that all the .exe files in your GOG collection are tested with "innoextract -t 'the_name_of_the_installer_or_patch_file.exe'. If it doesn't give an error, then that installer/patch file (the .exe file and related .bin files) are fine.

That way one doesn't have to rely on GOG's md5 checksums which may or may not be valid or exist at all, at least for the verification.

I think I have both a bash script (for Linux) and a bat file (for Windows) for recursively doing such verification of .zip files in all subdirectories, I think those scripts should work the same for testing the installers (using innoextract)...

I tested it yesterday manually by both deleting one of the .bin files, and flipping one bit on either the .exe file or one of the .bin files, and in both cases innoextract seemed to give a clear error. I don't think it points out which .bin or .exe file exactly is faulty though, that you have to find out some other way, but at least you find out which installer is not working.
Post edited April 19, 2022 by timppu
avatar
timppu: Has anyone yet tried to verify their GOG installer collection by using innoextract? By experimenting with it yesterday, all
Yes, I tried and it works . But innoextract is slow as molasses and I can not recommend to use it for verification.
avatar
timppu: Has anyone yet tried to verify their GOG installer collection by using innoextract? By experimenting with it yesterday, all
avatar
Geralt_of_Rivia: Yes, I tried and it works . But innoextract is slow as molasses and I can not recommend to use it for verification.
It's open source so I can have a look at least, maybe I can reimplement it's checking. It's still only a solution for Windows though, but Linux / Mac wouldn't be worse off.
avatar
Kalanyr: It's open source so I can have a look at least, maybe I can reimplement it's checking. It's still only a solution for Windows though, but Linux / Mac wouldn't be worse off.
Yes, only Windows installers use innoextract, though the tool itself runs on Linux it seems (if you want to manage the backup of all your installers including windows ones from a Linux box): https://constexpr.org/innoextract/