I have to say that the ability to skip downloading files based on keywords is getting higher in my wishlist, as I just noticed that e.g. The Witcher 3 extras has six non-English "goodies packs" amounting to well over 5 gigabytes, which I definitely don't need nor want.

So I'd really much like the ability to enter keywords to some kind of text or config file, and downloading of a file should be skipped if any of them are detected in the filename. That would also cover the teceem's and Lebostein's wish to skip patches, as they could use "patch_" as such a keyword.

My personal top two wishes at the moment:

1. The ability to divide the downloading of your GOG library to two or more different paths (I was meant to do something like this myself with a project fork, but I'm afraid there will be a massive update to gogrepo just when I'm done...).

2. The aforementioned ability to skip downloading files based on filename keywords. It should also clearly list all the files which it has skipped, just so that one can check that it didn't accidentally skip something you wanted (like the example that the keyword "russian" would also skip picking up some file which might be called e.g. "the_prussian_conquest".


The third one would have been that gogrepo skips hidden games in your library (e.g. demo versions which you don't want to keep anymore, as you already have the full version), but that was already added to the script, a fulfilled wish. :)

^ This
Plus the pre-allocation for me, so I can avoid extra hours of defrag.

I think it would be more easy to have an easy way to edit the manifest file.
There you can delete any unwanted file anyway.

This is also already possible when you edit the manifest file and split it into two (or more) separate files.
Then you can just download all files of each manifest file into a specific folder.

JSON editors:
https://github.com/rsuter/VisualJsonEditor
http://www.xml-buddy.com/json-editor.htm

That is one option I guess, but one would have to do that each and every time one recreates the manifest file. I redownload all the file details from GOG.com each and every time because I don't want to miss any silent updates (ie. I don't use the -skipknown or -updateonly options).

I.use a skipknown followed by update only for my standard checks and do a full once a month or so.

I also use I'd to force checks if I've bought DLC or know there's an unlisted update.

What is the advantage to use 4 download threads parallel?

In my eyes 1 thread should be faster because the bottleneck is writing on disk.... and parallel writing don't work. Or I am wrong?

Umm.. maybe if you download them to multiple hdds? Lol
I don't know, but isn't the available bandwidth diminished too? (yours and Gog's)

Can you elaborate? Aren't mechanical hard drive write speeds somewhere around 200 MB/s or so? Are you saying your internet download speed is faster than that, or what? If that was really the case, maybe you should invest for a SSD drive for more speed?

I am quite sure that at least for me the bottleneck is the internet download speed, not how fast gogrepo can write the data to the disk.

Whether to use one or more download thread (in gogrepo, or elsewhere), I've figured the potential advantages are:

a) Using several (e.g. four) download threads in parallel: occasionally it seems the download speed from GOG's side might be throttled to around 400kB/s. So if that happens and you have only one download thread, that's the speed you'd be getting.

If you have e.g. four download threads, then you would get at least 4 x 400kbytes/sec (probably much more because you'd have to be quite unlucky that all four downloads would be hit by the same throttling, but I guess it is possible).

b) Using only one download thread: if you need to abort the download and resume later, gogrepo needs to completely redownload any files which were downloaded only partially. Hence, with four download threads you might end up having to redownload four files all over again, while with one download thread there will be only one incomplete file that needs to be redownloaded.

Not sure if there are any other reasons (file fragmentation will be higher with several download threads maybe?)... I usually use the default four download threads.

Oh dear, I always only did the updateonly and skipknown, now I ran a full update and... 147GB to download.

Next week I get a new 6TB drive...

p.s. Your Scan Computers order will be delivered today between 10:45-11:45

Yeah, That's GOGs fault not GOG.downloaders. They fail to mark things updated when they should (DLC added, extras updated, patch files added, preorder released etc)

Can you explain why the fetching process is so slow? Is it the md5 generation on gogs servers?

PS: Complete Pillars of Eternity (round about 40 GB) got an update, without setting the update flag. Sometimes I think they make it on purpose...

Yeah, this is why I just run the whole gogrepo update from scratch each time... :D

On the flipside, I don't do it that often, more like once a month or even less. So I don't have any automatic script doing a full scan and update every night or anything like that.

That also has the benefit that I don't have to redownload the whole Titan's Quest that often, I think already now it has been updated a few times all over since I last downloaded it. :)

Also I specifically avoid running it now during big sales, as I am sure GOG and the CDN servers already get enough hammering from all over the world...

Wait, how many GBs do you download each month? O_o

Depends completely which games and how many games GOG has decided to update in the meantime. :) If they have decided to update the Witcher 3 installers again, that alone is assloads of downloading. (Now the GOTY version has been added to my account too, so I guess I'll be downloading that too the next time I run gogrepo. Oh yeah and Darksiders Warmastered Edition was recently added too...)

I'd say quite often gogrepo downloads something like 50GB for me, but it could be more or it could be less. Whatever, I put it to download over night on a dedicated small laptop.

Ugh.. btw, I still have to start from 0, so I fear I could reach 500Gb in one go, or more O_o'
I really need woolymethodman to update the script, though..