Yeah, I am aware of that, which is why I suggested calculating MD5 in the script for EXTRAS files matching the size - i.e. on your own, not relying on what GOG provides. Besides matching size, MD5 = none might be a good indicator as to when perform this hash calculation.

I do not have the file at that stage (this check is performed at update, there is also an MD5 check in verify when you do have the file but I'm having difficulty thinking of a context where calculating the MD5 in place of an authoritative one would be useful there )

Ooops, a bit of omission in my thinking - you can't calculate an MD5 if you don't have the file and you need to download it :) It would be useful to clean the directory of duplicates, but will also result in re-download (and deletion) every time... That's a bad solution, I agree.

But, as it is the only game (two IDs actually) that has this issue, I would think of hard-coded workaround, i.e. if extras are being downloaded for one of these two IDs, skip downloading these few files if non-English version(s) is selected. However, this solution is ugly enough I would understand if you decide to not implement it at all, I'd rather waste 65 GB of my disk space than have the script code messed up.

Thanks for looking into it, whatever you decide :)

EDIT: Maybe I will just use:
-skipfiles *witcher*(1)*

It would do the work, at least for my collection - luckily, it seems GOG is not using this in any file names, so even "*(1)*" mask should do.

EDIT2: Yep, that did the trick.

I have been wondering about multiple hard drives and downloading with a split manifest file to those multiple drives.

A few months ago I tested mergerfs [1] when I wanted to see if I could download (only using basic browser downloading so far) to one directory, and that download would automatically be split between multiple directories to simulate multiple drives.

With only minimal testing so far, it was a success.
Unsure if mergerfs is worth using along with gogrepo, or if just splitting the manifest file is easier.


[1] https://github.com/trapexit/mergerfs

I actually outsmarted myself here because I used a hash bucket solution for fast matching but that makes it difficult to put in an exception. So I'm still thinking about this.

-skipfiles *witcher*(1)* really solves this issue. I think the major problem now are those pesky timeouts during UPDATE, but I guess not much you can do about that crappy CDN GOG is using nowadays.

Thanks for your effort anyway, you might want to consider asking for donations :)

Umm.. for some reason gogrepoc decided to put in the Orphaned folder "guacamelee_super_turbo_championship", even if both the file name and MD5 are the same.
It shows in the library and it's not hidden either.
Strange

EDIT: Ah lol, the folder was just renamed to "guacamelee_super_turbo_championship_edition"

Just a suggestion.

Because of GOG's current server breakdown (supposedly fixed, but actually not even close), running a gogrepoc update would "ruin" one's gog-manifest.dat, as the thousands of files currently returning a 404 error will be considered missing (and be orphaned upon clean).

I recommend saving a backup of your pre-outage gog-manifest.dat under a different name. I called mine:
LAST_gog-manifest_BEFORE_OUTAGE.dat
to allow for an easy restore.

Or even better just wait a afew days to do an update run.
I was planning to do one yesterday, after the last big sale, but when i found out about this problem i postponed it for a week.

There is no real way to know when the problem is over without doing a run.

According to GOG's recent update, it is mostly resolved, when it is clearly not.

I am running a full update to count and report the number of failures (so far, more than 250 for a quarter of my library).

A simple way to spread your GOG collection across multiple disks is to use NTFS junctions in Windows and symbolic links in *nix based systems.

Starting in your <savedir>, you can move some games to a second drive and create a junction / symlink to them. gogrepoc will treat each junction / symlink as local to <savedir>; as does explorer.exe so when you open a junction it opens it like any other folder, rather than acting like a URL and opening the location the junction points to.

It's a manual process and can require maintenance (juggling things across disks), especially if you don't have alot of free space available.

Let's say I've moved The Witcher 3 from my main archive in D:\Games\GOG to a secondary archive on E:\Games\GOG. From a DOS prompt:

cd /D D:\Games\GOG
mklink /J the_witcher_3_wild_hunt_game_of_the_year_edition_game E:\Games\GOG\the_witcher_3_wild_hunt_game_of_the_year_edition_game

If you do this, please make sure the bridged disks use the same file system (especially if you're on Windows)

Ran it again 300 and counting for a library of 860 almost exclusively XML errors. Like an idiot I didn't know this was going on and when I did my update on Saturday like normal it pushed 700GB around 800 files to orphan... they are not bad, but the update bug made gogrepoc think they were so now im in a holding pattern to figure out how to fix this.

Just move them back to their original location and then run a full update again when it's fixed.

ETA - I suppose, I should probably make a rolling backup of the manifest, for update, to avoid situations like this.

Until you do, here is the solution I use on Windows.

Before invoking gogrepoc update from my batch files, I have the following line:
copy gog-manifest.dat gog-manifest-%date:~6,4%%date:~3,2%%date:~0,2%.bak

This effectively creates one backup per day (for days I use gogrepoc), which has proven a good compromise solution. Once every couple of months, I just delete older backups.

EDIT: My suggestion works as is only if your windows date format is DD/MM/YYYY. If not, you'll want to modify it (or ignore it). Sorry.