Also Galaxy ignores the hidden tag from the servers. I have hidden all my demos for example and Galaxy keeps showing them.

I have never used hiding on the Galaxy client so I'll have to look into this, It's possible it sets a tag that I can still access and respect.

It seems like you're experiencing an error during the GOGRepo update. However, you can still download the games using GOGRepo, even though the verification process may not be properly conducted based on the MD5 data. This error might occur for certain games like Kyn and The Walking Dead Season 1, but other games in your collection don't seem to have this issue. It could be helpful to notify GOG about these specific games (Broken Age, Kyn, The Walking Dead Season 1) so they can address the missing MD5 data. Unfortunately, I'm not aware of a way to check the existence of MD5 data for a game training from the download page.

edit:
There is a problem with the game: "Sam & Max Season 2: Beyond Time and Space"
Both the classic and the remaster version share the same ID "sam_max_beyond_time_and_space"

Original 2008 version
https://www.gogdb.org/product/1207659064
Remake
https://www.gogdb.org/product/1901266299

First it seemed that it would download both games, but in the end only the remaster was on my hard disk.

The disk space for the files of the original was reserverd, but it's only filled with null-values, the files (installer and extras) were not actually downloaded. They remain in the temporary download directory and are unusable.


edit: As I just saw, the same happened to "Schein". Both the original and the new version have the ID "schein". In the game list the original is called "Schein Base", the id is "schein" however, just like the one of the re-release.
Also in this case the installer, extras and the content of !images of the original version is missing.

https://www.gogdb.org/product/1207665003
https://www.gogdb.org/product/1430953337



This is clearly GOGs fault, but maybe you want to implement a workaround for it?

No, this looks very much like a gogrepo bug. Actually, two bugs:

1) It's using slugs instead of ids to identidy products. While they typically are unique and do not change they are neither guaranteed to be unique nor are they guaranteed to be unchanged.

2) It's not picking up on changed slugs.

Just click on the links you posted yourself and check out the slugs. You'll see that not only the ids are different but the slugs are different as well.

It looks to me like gogrepo never expected the slugs to change. Have you tried to delete the manifest and rebuild it from scratch? That should clear up your problem until gogrepo is fixed to expect changed slugs and correct them in the manifest.

Ideally gogrepo should use the ids to identify games and not the slugs.

Heh, who'd have thought ... it was a error in the DB afer all. After I manually updated schein_base and sam_max_beyond_time_and_space_2008_original_version, the entries were separated again.
I assumed, it was gogdb that did not handle them correctly.

I saw slugs changed in the past that were handled correctly, maybe it could not handle that one replaced the other.

But using IDs only would not do any good if anyone wants to update single games, the ID to type in is much easier to handle if it's readable and not numeric.
Of course GOG has some pretty f**ed up slugs, "star_wars_republic_commando_copy3" for Jedi Knight for example, but in most cases it's ok.

Currently gogrepo it goes from slug to numeric ids and gets the file list for these, but only downloads the files for the higher one.

edit: something's still fishy. Now both schein and schein_base lead to the new version. I guess I'll have to delete the 'schein' entry manually and refresh after that ... or I just let rebuild the whole archive over night.

Can definitely handle changing slugs, I suspect the issue here is that they reused the old name for the new version and gave the old version a new name. I didn't expect that because that's extremely silly, especially if it didn't get tagged as updated.

ETA - Just double checked and the only thing that seems to hinge on slugs is the name of the game folder for downloads. Which could definitely be an issue if 2 different games have the same slug. I guess I could rename all folders to slug + id to guarantee uniqueness.

ETA2 - Hmm, probably better for readability if I check for duplicate slugs and append the ID if there is.

Before you do anything, I just wanted to let you know that I have the two Schein versions, and when GOG did the switch, gogrepoc had no problem handling it, without any help from me.

I don't think that the slug+id renaming is a good idea. It would make directories unnecessarily illegible.

I'd recommend against that. Since the slugs can and obviously do change you can have duplicate slugs today and unique slugs tomorrow.

On top of that: It looks like the slugs are generated from the product name by removing characters other than letters, replacing spaces with underscores and lowercasing the result.

For example let's look at product id 1207659064:
Title = "Sam & Max: Beyond Time and Space (2008 Original Version)"
Slug = "sam_max_beyond_time_and_space_2008_original_version"

Now have a look at the changelog: The title wildly flips around between "Sam & Max: Beyond Time and Space (2008 Original Version)" and "Sam & Max Beyond Time and Space" (and sometimes even " product_title_1207659064").

When that happens your workaround would lead to a world of fun for everyone! Sadly without a holiday.

You wish ....
The most impressive one:
"Dark Forces : Jedi Knight": "star_wars_republic_commando_copy3"
Republic Commando???

Also:
"Jedi Knight: Mysteries of the Sith": "star_wars_jedi_knight_dark_forces_ii_copy3"
"SNK 40th Anniversary Collection": "snk_40th_anniversary_collection_copy3".
Then there's "act_of_war_high_treason_copy3" and "downfall_redux_copy3".


Funny enough they all end with "copy3", never with "copy2".

Sam & Max S1 classic: sam_max_save_the_world
Sam & Max S1 remake: sam_max_save_the_world_remastered
Sam & Max S2 classic: sam_max_beyond_time_and_space_2008_original_version
Sam & Max S2 remake: sam_max_beyond_time_and_space

Also many slugs contain extensions like "_game" or "_base"
In many cases there's a leading "the_", in other cases it follows at the end, and sometimes it's missing completely.

No, I think that slugs are generated manually, maybe by copy/pasting the titles into a converter, but it's definitly done manually. These are human mistakes.

A manual DB fixer might be useful that only updates all IDs and slugs and optionally in a quick&dirty step looks for double slug entries and only updates these.
When the user inputs a slug to update or download one single game, go the traditional way to determine the ID from the DB. But then check if the slug provided online is different has changed and proceed with the changed one.

I actually proposed that years ago and the general opinion was that it was better to use the GOG data as much as possible and not do manual fixes to know bad slugs.

There's still an issue with the IDs because GOG also doesn't make a strong guarantee those are inviolable either, the Spear of Destiny situation nuked the old versions and created a new unified version with a different ID.

Not really sure how to handle this situation, the folder name definitely needs to be unique but I doubt anyone wants them to be the raws IDs practically. If anyone has any suggestions I'm definitely open to ideas.

That does sound very ominous indeed.

I am tempted to just update the full db every night, but then I would miss which games have actually been updated and need to be redownloaded.

Maybe when the user does something with a single slug, double check, if the stored ID is ok? Do a online check for that ID, see if the slug stored there is the same as the user typed in.

This is exactly the reason why I practically always delete the old manifest file, and start it from a scratch, when I run gogrepoc update / download. Less "unexpected" problems like this, also if and when the manifest format changes.

Yeah I know I know, it would cause much more stress to GOG servers if everyone did that all the time, but I refresh my local GOG installer collection maybe like 2-3 times a year so I don't feel that bad about it. I am not one of those freaks who run update/download every night, or even weekly.

Updates to games already rebuild the entire entry, so if a match is generated the ID gets redone when the update happens. Actually if a new ID is assigned it gets treated like a new game since it won't match the existing entry in the DB, and if the existing entry is on the scan list (eg you're doing a full manifiest update) it'll delete the old entry because it no longer matches a valid ID.

So it doesn't resolve the actual issue. Which is that the slug is the same, which results in 2 games trying to use the same folder to store their data.