Indeed. Where a lot persons nowadays enforce their "for me"to others(and blame else for doing that). Also major are narcissists thins others as "others", while minor use "we" to describe craziness of human beings (im not always follow the right way to say such things).

And made up nonsense arguments. Although for categorisation argument i maybe miss/misunderstood you. How The is not part of T folder? Or you mean you have more sub-folders/sub-liters in filesystem and/or table catalog?

I myself a person of Order. Mathematical meaning. Argumental, binar (yes or no, 1 or 0). I almost have no preferences in anything, just like never able to make such or chose something. I need some reliable to decide.
There is The as start of name. Fact. Argument. Value. And i see preference "for me" how visually make it "nice looking" by refuse to take The as even part of the name. And ask to change that for someone else. Well, you explain it that way (for Beatles). If im not mis something again (possible).

working now - thanks :)

I too, am a person very much in favor of order, and in fact all my stuff is very ordered, based on some rules.

The difference perhaps between us, is that I see a leading 'The' word as additional and not a defining part of the name.

It is also the case, that sorting on the second word, which is definitive, can avoid other issues ... perhaps the author later decides on a rename, removing the leading 'The', for example. And folk can often forget there is a leading 'The' as another example.

Then as I said, too many sub-folders in the 'T' folder, especially where each of those has a folder image (i.e. Folder.jpg) draws more system resources. I also like the content of the main alphanumeric folder to be more balanced in number. That meaning, being more on par with the number of sub-folders in other alphanumeric folders.

Basically, if you don't do as I do, then the 'T' folder has a huge number of games starting with 'T', when compared to all the other alphanumeric folders. And the more in a folder, the more to manage in one go. So I believe in natural dividers of content, and any leading 'The' not being definitive, is perfectly suitable for relocating to the end of the game name.

Some folk of course go further and do the same with a leading 'A' word or even a leading 'An' word. I don't because they are far less in number, as I have taken note of over the years.

In any case, I am not alone in this, and Libraries for instance do the same, and even go further than me, and you can kind of consider them 'official'.

On the other side of the coin, I hate author names sorted by the surname, so you will never see that in my records or lists. So for me for instance, 'James Patterson' is always that and never 'Patterson, James'.

But like I said, each to their own. I do what I prefer.

Another case of where I differ from many, is the use of a capital on every word in a title. For me that is what defines a title, every word being capitalized. I don't know who made up the rules about many words like 'a' and 'and' etc needing to be lowercase, but in my view I deem that a stupid rule. I know why they do it, but disagree totally. A title is a title and should not look like a sentence in my view.

On the topic of Drive Pooling:

On linux I've been using MergerFS (https://github.com/trapexit/mergerfs) - which just let's you combine drives and the files are copied depending upon how you set it up (percent full, etc). Yes, this means that a game will likely be split among several disks but realistically does that matter? You can mix and match different size drives. So a 4tb and 2tb can live happily together. And you can add drives as needed.

So like raid except stored by file rather than by data block. I think unraid (https://unraid.net/) works this way too.

The nice thing is MergerFS doesn't care if the drives are on a usb or on the motherboard.

For backup, I've got another bunch of drives that live in a usb enclosure. Again mergerFS and just copy from "drive 1" to "drive 2". Then on the backup I useSnapraid (https://www.snapraid.it/) for parity

Windows has a "drive pooling" feature which is much more like raid (as I read it). I think they call it a "storage pool". Though based on what I remember I'd just as soon get a cheap used desktop computer somewhere and set it up as a NAS

I'm back now, so if the potential of stuff happening while I was away have held you back from updating you're good to go.

Hello, i'm trying to import my collection from one folder on my external drive to another using the gogrepoc.py import command, it will copy one bin file and then i get the error of

Traceback (most recent call last):
File "E:\gogrepoc\gogrepoc.py", line 3980, in <module>
main(process_argv(sys.argv))
File "E:\gogrepoc\gogrepoc.py", line 3759, in main
cmd_import(args.src_dir, args.dest_dir,args.os,args.lang,args.skipextras,args.skipids,args.ids,args.skipgalaxy,args.skipstandalone,args .skipshared,False)
File "E:\gogrepoc\gogrepoc.py", line 2251, in cmd_import
entry = items(folder_name,file_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'dict' object is not callable

then if i run the command again it copies the next bin file and gives the same error, what could cause that? sorry i'm a beginner at this. Would it be better to just forget about import and download my whole collection initially? And if i understood it correctly when you want to update again after some time, you use the gogrepoc.py update command to get the manifest and then the gogrepoc.py download command will download everything on the manifest but skip identical files that are already in your folder?
Thanks

This might be a bug, gimme a little bit to look into it. You are correct about the 3rd question.

Yes, this is a bug, I changed the way import worked to handle the new update checking by update if needed but I didn't change one line correctly. I'm not in a position to upload / text a fix right now (I'm in the middle of testing something else) so it'll take a couple of days for me to fix officially.

However in the mean time on line 2251 , changing

entry = items(folder_name,file_name)

to

entry = items[(folder_name,file_name)]

should fix the issue.

ETA - If you don't feel comfortable making that change yourself here is an *untested* version of gogrepoc with that change made; https://www.dropbox.com/scl/fi/3vjpklrehs89994399epu/gogrepoc.py?rlkey=mtixrng5up32gim9izo18hlbt&amp;dl=0

What I'm testing at the moment is making sure the strict extra update works properly, and yeah it was a good decision to have this turned off by default, if I start storing some config info in the manifest I might default it to on for new manifests.

But for me with an old game collection it orphaned 632.64 GB of extras (ie pretty much all of them ) and will take a long time to download versions that I know match the current last updated time stamp.

ETA - Looks like I found some kind of error with my MD5 xml handling too, it seems like I've gotten a blank item name from somewhere which causes a script crash.
ETA2 - Trapped in a hell of my own making, because the last resume manifest update happens at the same time the XML generation does , I'm stuck reupdating the last almost 50 games while I debug this. Not so clever now to handle that all at once hey smart guy ?

Thanks a lot it's working now! And thanks for working on this script it's awesome

So found my error but it's not really a me problem, the Offworld Trading Company DLC installers for Mac are super borked ( the entries exist in the XML but the keys are invalid ) , that means that all I can do is treat those DLC as deleted, so you'll need to manually archive those from being orphaned (they also won't be reported as having a problem on the verify check because as far as GOGrepoc is concerned they aren't supposed to exist ).

I'll file a ticket with GOG but the response times are usually not great for this kind of thing.

ETA - I've tweaked my copy now so that hopefully it will generate a warning in situations like this, if you're getting the MD5 XML files (it's the easiest point to detect this since the invalid key puts the DLC entries in a bizarre state (language + OS but no name) that gets detected as part of the XML maintenance )

It appears I can either 'update, download, and verify' either one game title at a time, or all game titles in the manifest.

I do not have that much drive space currently to download all.

I do not like the idea of having to manually split the manifest for downloading batches of games at a time, nor for multiple drives.

What works for my needs, with games split between multiple drives, is this.
In my gogrepo folder, each game title either stays on the main drive, or is symlinked to other drives.
So when I do an 'update, download, verify' batch, each game gets put on the drive that I symlinked.

I wrote a script where I can download custom batches of game titles.
https://gitlab.com/version2013/gog.com/-/blob/main/update_download_verify

My thread also mentions this script and of course gogrepo.

Thank you 'Kalanyr' for your continued coding excellence with gogrepo.

Your script seems kind of like overkill for what is already possible with -ids and xargs. For example

grep "'title':" gog-manifest.dat | sed "s/\s*'title':\s*'\([a-z0-9_]*\)'.*$/\1/g" | sort | grep -E "^[a-e].*" | xargs python gogrepoc.py download -ids

will download all your games from A to E.

An interesting (Linux) approach. Maybe some Powershell or Windows command prompt guru is able to create a similar for Windows, or then use the Linux subsystem in Windows...

A couple of questions:

1. Does that same basically work for clean and verify as well? I guess it does if it just picks the titles one by one based on the first character?

2. Since not all titles start with an alphabet, I guess there should be an alternative command to e.g. download all titles that do NOT start with a-e? That way making sure you get to download all titles, and not only those starting with alphabets (e.g. if you download a-e and f-z, then you will miss many titles that don't start with a-z).

So first you'd give that command to download titles starting with a-e, and after that the rest that do not?

EDIT: For that, should this part be modified somehow: grep -E "^[a-e].*
Ie. negate that, download those which do not start with a-e? I recall grep had an option for such negation but I don't recall without checking the manual pages, and I am now on Windows...


Anyway, thanks for coming up with that commands (sed and awk etc. make my head spin, as well as if I have to work with regex), I have to experiment with that as then one could keep using one manifest file without having to divide and remerge it manually to achieve the same, something I did in the past...

EDIT: Well, maybe there are some drawbacks after all with that approach?

1. Since it downloads each game separately one by one, you can't test with e.g. --dryrun how much space the games you intent to download will take, just to make sure you have enough space?

2. If it downloads one game at a time, then I guess with games that consist of only 1-3 parts, you don't get to download in 4 threads all the time?

Not biggies but still some restrictions to take into account...