Making the download folder f"{slug}/{id}" would keep the upper-level folder readable while maintaining uniqueness in these edge cases.

Downside is that it requires traversing one more folder, might break someone's workflow if they expect a single directory before reaching the download files, and would require some sort of conversion helper function to move games from the old folder to the new folder.

edit: maybe the {id} subfolder could be created only for the edge cases? I'm not familiar enough with the script to know if you could reliably detect changes to the slug->id relation in the manifest.

not a fan of that. The subdir should be optional at best, I prefer flat directories for the games.
[slug]_[id] would suit me better.

Basically a opntion or variable if the separator was "_" or "/" should do the trick, assuming that python is able to create parent and subdirectory with one command.


I hope ther eis a better solution however than to use ids at all.

You may be right, it could be human error as well. Maybe the generated slugs are only the default that can be manually changed. Only GOG knows for sure.

How about adding an option for the user to set subdirectory using templates like I do it in lgogdownloader.
https://github.com/Sude-/lgogdownloader/blob/b554894/src/util.cpp#L423-L472
Perhaps have templates like %slug% and %product_id% so users could select what they want to do.

If user wants to keep the current behavior they can use
--subdir %slug%
which would be the default value for --subdir option

If they want product id subdir in the slug directory (like lupineshadow proposed) then they can use
--subdir %slug%/%product_id%

If they want the product id in directory name (like you proposed) they can use
--subdir %slug%_%product_id%

I recently added an option to transform the slug using regex rules to lgogdownloader.
This allows users to change GOG's stupid slugs ("Dark Forces : Jedi Knight" is "star_wars_republic_commando_copy3") but still use the original GOG value internally.
https://github.com/Sude-/lgogdownloader/blob/b554894/src/util.cpp#L966-L1001
https://github.com/Sude-/lgogdownloader/blob/b554894/man/lgogdownloader.supplemental.groff#L111-L157

Option is good but I obviously don't want to give the user the option to do the thing I know is broken.

Oh I would absolutely love custom paths & subfolders, so I wouldn't need to keep a separated directory for Gog games from other stores.
But it's probably too complex -_-

I also think it's overcomplicated using a parameter for this.
Maybe a set of variables at the beginning of the script which everyone interested can edit.

The current way does work in most cases, except for these few when slugs were swapped, it's definitly not the highest priority.


Btw, is there a command to remove a game from the data file using it's numeric id?

If it's for an entry that should no longer exist simply using update -ids
should remove it. I think you can remove them by hiding it on GOG and updating by id too. Don't think there's a direct removal for an item that should otherwise be visible.

Relying on GOG named slugs, due to the many weird ones, is probably not a great idea.

What would be better, is creating slugs based on your own conversion of the title. If gogrepo.py did this it would be better than using an added ID. This method might get around the reported issue.

Personally I prefer to use the title with my GUI, but my code removes or replaces all the illegal characters, so it does end up like a slug, but with no underscores and it has capitalization.

Have you ever thought about refactoring gogrepo to use a proper database instead of a flat manifest file? Even if you don't want to require users to install a full database application like MariaDB or MySQL, it should be possible to use SQLITE. I'd be surprised if phyton doesn't have built in SQLITE support or a plugin for that.

If I had to chose, I would just go for JSON, a relational database model is overkill in my opinion. Tables only have an advantage, if queries can be made ("list all games that have no changelog entry" for example).

Thanks for the hint with the ids btw. I was thinking of something ID specific, but this should do the trick as well in most cases.

JSON, XML, SQlite go with whatever is easier. Heck windows 3.1 *.ini files works too. LibreOffice Base is also an option (as a cross-platform solution). SQLite is already used for Firefox bookmarks I think.

Regardless of file encoding, you could also just go full Calibre (as in the ebook manager), assign your own identifiers and directory structure. I even think Calibre is written in python too, and it's open source (https://github.com/kovidgoyal/calibre gpl3) so you can probably "borrow" some code

Just

I've considered a DB but the user readability of the manifest seems pretty useful. The next big version I was working on uses YAML rather than the current JSON.

+1 for this

Those of us who might want to "get fancy" can always do so.

Dumb question time:

I keep getting "missing items.....1" What should I look for in the log file?

Edit: it *looks like* Imperator: Rome is the issue. At least the files are missing when I looked in the download dirs.

We'll see if it validates since I manually downloaded them. Kept getting 404 errors on manual download too.

Here's what clued me in. So I'm assuming this is what I should look for going forward Edit 2 (5 June 2023): Imperator Rome was indeed the missing set of downloads. When I was finally able to download the files manually, they all verified with noting missing.

Sometimes the informations on the download servers is not up to date. Try again to update the game information in a few hours.