Given the recent outage, you might want to consider differentiating between a removed file (e.g. when an older installer version is replaced by a new one) and a previously-valid broken file link (e.g. 404 error, as we have seen over the weekend). I believe the latter should not be orphaned in a clean operation.

Only by the update flags that GOG provide, that gogcli.exe grabs when I update the list of game titles found in my library. My GUI keeps a record of all updated games in a list and their entries get a red color background for their row in the GUI, as seen in the following screenshot.

SCREENSHOT

You are aware that this is only a subset of updated files? There are hundreds of games that GOG, by design, never gives flags for, not to mention all the extras in all the games, which are also never flagged.

I run a gogrepoc flag update practically every day, so I never miss these update flags. Yet whenever I run a full update (usually once a month) there are always games that get updated only then, because they never got a flag --- at least 2-3 such games per month, sometimes as many as 10.

Yes I am aware of that irritating aspect of GOG.

Basically I decided to live with it, and any game I really care deeply about I can do a manual check, by replacing their manifest entry, and because my GUI also stores file names and MD5 values etc in a database, my download window will only show me what I don't already have.

I'm not fond of the idea of my PC spending much of a day every month, re-downloading all manifest entries, or having related automatic downloading. For me, it is all about control and having hands on the wheel. I am prepared to sacrifice something for that.

As it is I can never keep up, and already have big backlog of updates to get.

Sounds good.

To some extent that's always going to be heuristic and may result in a set of install files where eg the ,bins and/or exe are mismatched unless GOG starts providing info on official build versions or information on which files are sets. I'll definitely think about it though and see what I can come up with.

However that's not what I was talking about, I was talking about stuff that eg hasn't been released yet or Kickstarter reward stuff that never went up on GOG because it was delivered through another channel, if my suspicions are correct an info value of 0 indicates that the file isn't intended for user download (either because it's a Galaxy file (like the Song of Conquest DLC which comes as a bundled installer for offline or because it was never released)
Most of the unflagged updates these days are things where it's an internal GOG change (eg separating out language packs or changing the installers) but won't actually effect an installed game. Though there's still a few per month that should be flagged. It used to be much much worse.

Yes. I was just using your last post to trigger a notification. I should have deleted your text.

Ah - thanks for that. Good to know.

~76% of the way through and so far only 3 items actually "need" downlink and 2 of them are Imperator Rome , everything else is either good on downloads or is missing the md5 on downlink too or is broken on both (though so far the broken stuff seems to be some imperator rome files when the CDN missies downloads because they aren't on downlink either or are genuinely missing files (I have found the flag but it's silly and bad and not what I thought but *sigh* it works, so I'll have to test it and there's at least 4 links where GOG provides bad data in that field (so it's downloadoble on the website too) including the 3 split up Galaxy versions of the supporter pack.

So testing 2292 games
amnesia_a_machinge_for_pigs is missing MD5 for en2installer2
anomoly_defenders is missing MD5 for en2installer2
avadon_the_black_fortress is missing MD5 for en2installer1
baldurs_gate_iii is missing MD5s for
en2installer0 ( also the file is not available on downlink )

en2installer22 ( also the file is not available on downlink )
en2installer23 ( also the file is not available on downlink )
en2installer24 ( also the file is not available on downlink )
en2installer25 ( also the file is not available on downlink )
en2installer26 ( also the file is not available on downlink )
en2installer27 ( also the file is not available on downlink )
en2installer28 ( also the file is not available on downlink )
en2installer29 ( also the file is not available on downlink )
(This is basicallty eveyrthing for Mac and includes the installer, language packs and the special data file archives (which aren't on the list because they are exempt from having md5s )
Divininity Oriignal Sin 2 is missing MD5s for
e2installer0 - en2installer12 (and none of them are available on downlink) and divinity_original_sin_2_sir_lora/en2installer0 (so all the mac files)

imperator_rome is still random because some files are missing on downlink so you still get to play a game off "is this file available ?' on the downloads CDN but all files/md5s are theoretically available
The files missing on downlink are en1installer1, en2installer0, en3installer0 , /imperator_rome_deluxe_artbook_wallpapers/en2installer0 , imperator_rome_deluxe_artbook_wallpapers/en3installer0, imperator_rome_magna_graecia_content_pack/en2installer0 , imperator_rome_magna_graecia_content_pack/en3installer0 , imperator_rome_epirus_content_pack/en2installer0, imperator_rome_epirus_content_pack/en3installer0 , imperator_rome_the_punic_wars/en2installer0, imperator_rome_the_punic_wars/en3installer0, imperator_rome_heirs_of_alexander_content_pack/en2installer0, imperator_rome_heirs_of_alexander_content_pack/en3installer0, imperator_rome_hellenistic_world_flavor_pack/en2installer0, imperator_rome_hellenistic_world_flavor_pack/en3installer0
So an excellent array of files carefully chosen to make it uninstallable unless you get hits on downloads.

knights_of_the_chalice_2 is missing md5s for en2installer0 (and the file isn't available on downlink )
republique incorrectly has https://www.gog.com/downloads/republique/41353 marked with a 1 MB size but it's an unreleased manual (this one can be downloaded and fails on the website too)
rimworld is missing md5 data for en3patch1 (and the file is missing from downlink)

songs_of_conquest has the issue with the Galaxy only version of the supporter pack , unfortunately this is probably unflaggable because they do actually have a size if you're on the correct API, I just am not.

spacebase startopia is missing MD5s for en2installer0 - en2installer4 and all those files are missing from downlink.

succubus prison is missing md5 data for en1patch0 (and the file is missing from downloads) and en1installer0 is missing from downloads but both the file and md5 data are available on downlink

the bards tale directors cut is missing md5s for it's weird split archives but that's on me for not blacklisting those for md5s so that's fixed now.

the_witcher_2 is missing md5s for
en2installer0 (which is missing from downlink )
(Also either md5s got added while I was testing or it's got the imperator_rome issue but only for md5s because my first test had way more failures.

tropico_6 is missing md5s for all Mac installer files (including DLC) and all those files are missing from downlink (the output doesn't quite look like that because some of those files are extensionless data files which are exempt from md5 checks but that's what's happening)

wasteland_2_directors_cut unfortunately has https://www.gog.com/downloads/wasteland_2_directors_cut_digital_deluxe_edition_upgrade/34743 marked with a non-0MB size despite it being a Kickstarter reward that wasn't issued through GOG in the end.

x4 foundations has en1patch0 only available through downlink, however en1patch1 is not available at all and it's a bin file for enpatch0 so this is fatal. This is the only fatal error, I've encountered so far (fortunately it's an old patch and a fresh install doesn't need it)

So down to 11 games missing MD5s, and only 1 fatal error ( for a patch set). That's a huge improvement.

I guess you could count imperator_rome as a fatal error for a normal person too since I doubt they'd know to persist to get the files right.

Only 3 files actually really benefit from downlink (and I guess imperator rome might depending on your luck)

For my next trick I need to fix that game file as an extra issue (when MD5s go wrong) and work out how to handle when multiple games have the same slug then I'll upload this.

I find it great that you have a hobby that we all profit from.

At this point a big THANK YOU! for all your work.

Oh, if you have multiple games with the same slug (ie the folder name) or you have that issue with the witcher (where you get an indication that the same file exists twice with a different MD5) please reply to this post, with what you've got and send me a friend request if you're willing to help test.

Edit - Oh dear, seems that some extras do have MD5 data now , unfortunately the only way to tell is to test, and I think I'd need to recheck this for all the currently black listed extensions too just to verify. That's gonna be a long , slow and painful process. It's also gonna make updates much slower, since every extra will need to be check for md5s even if most of them fail (excluding any file extensions that I can verify never have MD5s )

That has always been the case. At least I have seen md5 on extras over 2 years ago already. But they have always been rare exceptions.

In the beginning I also tried to check for md5 on all files but later decided to drop checking extras because the rare exceptions simply do not justify the wasted time.

I have reported both of these to GOG in a January 2023 ticket and again in a May 2023 ticket. Both tickets, after months of useless back-and-forth, were eventually marked as "solved" (to ensure the 90+% resolve rate reported at the end of the year) and closed, with no change.

Ok strange thing, CDN is back and I rebuilt my manifest and ran a full verify but I have a bunch of failed MD5 which I know is correct because the downloads are corrupt.

I cant remember the command to make gogrepoc remove them and redownload them.

22:30:09 | --totals------------
22:30:09 | known items......... 5860
22:30:09 | pre-verified items.. 5835
22:30:09 | have items.......... 5860
22:30:09 | skipped items....... 0
22:30:09 | missing items....... 0
22:30:09 | md5 mismatches...... 25
22:30:09 | size mismatches..... 0
22:30:09 | zipfile failures.... 0
22:30:09 | --
22:30:09 | total time: 0:16:58.111225
22:30:09 | exiting...

I want to get rid of the 25 mis matches and redownload them, just cant trigger gogrepoc to do it.

Edit: is it just gogrepoc.py update -skiphidden ? I bet it is, maybe I'm to tired to do this tonight
Edit2: It is not, it did not work.