Oops, had a silly moment.
Yes, you do an automated resume for downloading.

Be interesting if you had a parameter to skip that, and I could check what difference, if any, it made.

Python 2 vs 3 is weird.

Python 3 broke backwards compatability, so 2 continued on for years after 3 came out while modules were converted. That means there can be modules that work on Python 2.7 but don't work on eg 3.1 but do work on 3.8

It's better to use Python 3 since Python 2 has officially EOLed now.

The goal here was just to see if the issue is due to the way Python does Timers and if that's changed.

The normal way of disabling download resume wouldn't help you at all, since it'd just skip trying to resume interrupted files the next time they are downloaded, it wouldn't make any difference to the initial download, it would take a fair chunk of code revision to remove the timers, preallocation and use of provided MD5 chunk data to enable the live resume.

Yes, I had read somewhere about that Python 2 / 3 issue.
I remember doing a similar thing with one of my programs years ago, where I was still doing updates to a version 4, but was simultaneously working on a revamped and improved version 5. In the end, it did my head in and I scrapped version 5, and continue to this day to put up with the flaws in version 4 (though we are at something like version 7 now, and a different version 5 has come and gone). They weren't terrible after all, just that version 5 was a better way of doing things ... or would have been if I had finished it.

I'm still a bit lost with what you are saying, because I never used a version 2 variant of Python with either version of gogrepo.py. So even if your timers in the forked version are different to those (if any) in the original version of gogrepo.py, I don't see the relevance of using version 2 of Python. Please excuse me for being a bit confused about this.

I guess another way to say that last statement, is that the original version of gogrepo.py is not having a timer issue for me with Python 3.7.0. That's not to say the timer issue isn't at fault, it probably is.

It seems a lot of trouble to me, to do whatever I need to do to install a version 2 variant of Python, for what seems a slim chance without any anecdotal evidence it might improve the supposed timer situation.

And like you said, better off sticking with a version of Python that hasn't reached EOL.

Probably in a few months anyway, I will have been able to redo my work and play environment, and move onto a better PC for my Web usage, and so hopefully solve this download speed issue in doing so. Just a bridge too far right now.

The original doesn't use threading Timers because it doesn't monitor the download threads (which is why sometimes you'll find its only downloading 2 or 3 things instead of 4), I think the threading timer implementation on your CPU/OS is the most likely cause of the performance difference, so trying a different implementation might help eg Py 2 vs Py 3 or 32 bit vs 64 bit Python or if you can do so easily try it out on a Linux distro on your system etc .

Some of you might find some of the following an interesting read.

https://www.gog.com/forum/general/gogplus_download_checker/post145

It is about my latest comparison checking endeavors with manifest updates.

Ok, THANKS, I'm clear on it now.

Though it probably doesn't help, as I have two agendas (primary aims), downloading for myself, downloading for others, both of which use my GUI, and which won't run on Linux.
I'm also on a 32 bit OS, so the 64 bit option isn't available.
If version 2 of Python did overcome the download speed issue, then it would put me in a dilemma.

I guess for me personally, I will just have to stick to using the original gogrepo.py for now.

At some point I might do an experiment with Python 2, but it isn't a priority right now.

My GUIs make up for some of the shortcomings, and most users of my GUIs should be able to happily use them with your forked version of gogrepo.py.

Hi guys,

I'm using Kalanyr's gogrepoc.py to maintain my offline GOG collection.
Just a friendly warning from me: if, like me, you want to keep your collection up to date, do not use -skipfiles 'patch_*' to skip patches - GOG doesn't always include latest patches in standalone installers. Moreover, some patches also include extras. See Dragon Age Origins or Giana Sisters Rise Of The Owlverlord examples.

The solution I chose is to have one-time bandwidth and a bit of disk space wasted, and I keep all the files - share if you have found better option.

Take care!

Oh, interesting to know.
However, saving all patches is a bit annoying.. I wish Gog simply named those files properly like "bonus_*" -_-'
And for patches not included in standalones, I hope it mostly happens for games with development in progress O_o

Thanks for the warning. There was discussion about this earlier (not all main installers being in the same level as the latest patch) but good point that there are also cases where the patch includes also extra content not in the main game.

What would be the best solution for this? Listing all unnecessary patches (and other files; can use wildcards/regex as well) one by one in some filter file that gogrepoc uses... but then that file would become huge, and possibly cause performance problems for the script if it checks that list for each and every file it is about to download?

Of course it could be some kind of community effort to keep up a public filter file that people can download and use, including all files (patch files or otherwise) that can be deemed unnecessary? Then people can download that filter file and use it with gogrepoc, edit it if they want etc. It can be quite an effort to keep the file up to date, though, also removing entries that are not relevant anymore.

And the performance problem... maybe it could be an extra step right after "gogrepoc update" where gogrepoc goes through the manifest file, removing any files from it mentioned in the filter file? Then that filter file doesn't need to be checked on the fly with gogrepoc download/clean/verify anymore?

Oh well, for now I just keep downloading everything... I know there are also some extras files that I don't want, like fan-made Russian translations of games, or Mac/Linux versions included in the extras of the Windows version. Quite a lot of space would be saved by excluding those + the unnecessary patches.

In theory there could be an exclude & include list for skipfiles (with the 2° obviously overriding the 1°).

The first one would contain regex like "patch_*" and the other entries like "patch_dao_2115.exe" , "patch_giana_1234.exe", etc.

EDIT: modified some parts, because they were incorrect.

Yeah it can get quite complicated and hard to understand, to make it "perfect".

Now that I think about it (and maybe this was discussed already earlier), such filter script that edits the manifest file directly doesn't even need to be part of gogrepo, but could be a separate script altogether. You just run it right after "gogrepo update", to weed out any unnecessary files in the manifest file that you don't want to be included with download / clean / verify.

That's the issue - for example, the aforementioned DAO patch + extras file is called patch_dragon_age_origins_ultimate_2.1.1.5.exe, meaning you can't construct a regex that would match these and not other, usual patches :/

The most reasonable way to handle this is for the GOG to make a file names cleanup - but if they care about us offline file hoarders, they could implement full library backup functionality in Galaxy - it has been requested multiple times, to no avail. Also a white / blacklist of files would work, but I think it would be rather hard to maintain it. For now, I just happily waste my disk space :)

Ah ops nvm that, I didn't click the files while checking the examples, so I incorrectly only saw "en1patch2".

The name "patch_dragon_age_origins_ultimate_2.1.1.5.exe" would be still viable for my suggestion.

But yeah, it would be excluded with the current gogrepo + skipfiles version.

Is your current working directory the same place you've downloaded the games previously?