Goodo.

As I was waking today, it suddenly occurred to me that my GUI could have a right-click option to remove a manifest entry.

It could do it in two ways.

(1) Remove from manifest, but leave the list entry in my GUI, which could then be quickly used to do an update for that game.

(2) Remove from manifest and GUI list.

Even with the first option, the GUI list entry would only remain until the next update, after which the list gets rebuilt.

I guess you're using dev, because the stable version doesn't have "skipfiles" nor "full" yet.
Btw, why aren't you using an asterisk even for extensions? Like "patch_*.*"

I perfectly understand why you and some others do that, and in principle I agree ... to a point.

My point or points being.

(1) More than five hours for me to do a full update, which as I get more games, is only going to increase. Sure, one day I may be doing my updating via a super fast PC, and so take much less time, however currently I am not.

(2) I always keep all of my files and never replace or delete unless they are corrupted.

(3) When a game gets an update, I want to know why, as some are just not worth wasting time etc getting ... especially if really large files. Generally if available, I will always opt for a patch file instead ... but then again, only if necessary ... though I am less concerned if the patch file is somewhat small, and so will probably grab anyway.

(4) Unless a game is having an issue for me, that makes an update worthwhile, I tend to only care when said issue happens, at which point i go looking for a patch or update. I know that's not ideal, but that's how many of us used to approach games. The only exception I guess, is where an update features something new ... more levels or abilities or features etc etc.

(5) As I have said before, when I get a new DLC etc, I always check my online GOG library and download straight away, and backup.

Yes it's dev, but works for me.
I think I avoided wildcards as not to accidentally omit something that I don't want to omit. Tbh I'm not sure :) Could've been "patch_*" afaik. But "*patch*" for sure is not a good idea as some installers have patch in the name. Probably that's why I used first the extension and then removed the preceding wildcard.
Anyway it's a script that calls the whole command line, and it seldom gets edits.

By the way does it (clearly) list in the log file which files were skipped? Do I have to search the whole log file for the skipped files, or does it list them separately? It would give a peace of mind if one could easily check which files were skipped. (I guess in Linux I can just grep the log file to see the skipped files, automatically making a separate log file for the skipped files...).

I must start using that option, in order to skip all the Russian/Mac/Linux versions which are added to extras (even if you use the -lang en -os windows flags, as those don't affect extras). And maybe those patches too, I presume there are no cases where the main files are older and actually need the patches?

To expand on what I said earlier.

This is how I would see that working.
(1) The best viewer for your games, is your online GOG library pages.
(2) With the right use of the browser links, you can get exactly what you want.
(3) Backing up your full library is not something most do all in one go. Most would be doing it in stages, if they do it at all.

So how would that work in practice?
(1) You have a simple GUI ... not much more than a couple of inputs and buttons and a few checkboxes.
(2) Two inputs or them popping up after clicking a button, could be used to create your cookie file.
(3) Another input, could be used for pasting a game title or browser link. It could also support drag & drop.
(4) An input and button for destination folder.
(5) A download button would then either download via pasted browser link, or if you used a game title, download based on checkbox selections - game files, extras, OS, language, etc.
(6) This uses aspects of gogrepo.py to do all the heavy lifting and checksums etc.
(7) This could be further developed into a batch queue arrangement.
(8) Most importantly things are checked on-the-fly and no manifest is needed. In short, this version of gogrepo.py queries GOG for all the necessary detail to then start downloading ... basically an update, download and verify all rolled into one.

NOTE - Yes I am well aware this idea doesn't include dealing with silent or unknown updates, but it probably caters for most people. And gogrepo.py does already do that, for those who want it.

As far as I can tell, yes, as long as you don't use the -new or -updateonly flags. At least, that's how I've written my batch files, and that's the performance I've gotten.

My "full" update is just the update command with these flags (your mileage may vary) update -os windows -lang en -installers standalone, and it checks every game in my library.

Trine, at least, would be one of those cases.

Trine Enchanted Edition - 2.11 (gog-5)
Patch 2.12 - 2.11 (gog-5) to 2.12 (gog-6)

Yes, you can always grep for them.
That's a bummer, likely we should give gog a heads-up to finally update the installer also properly. Ok, for Trine the patch is small, but in total I think I got rid of 150GB of patches.

And I've occasionally seen other examples. So, get gog to fix this one and guaranteed another will rear it's head another time. So, I'd say that there's absolutely NO guarantee that patches will never be required to install the latest version.
That's basically what my script generates and I often see it identify a lot more updated items to download than those with just an update flag. (I'm using the last stable Kalanyr release).

I did, but since the zip files aren't actually verified against the manifest it probably wouldn't have made a difference anyway.

My understanding is that extras in zip files are verified using the "self test" that you can perform on a zip file using something like 7-zip.

Yeah I think you're right. Oh well, mysterious.

And all the more reason why I think it could be drive related, which is why I suggest testing on another drive with a fresh copy of the zip files.

If it was some zip based python issue, you think that would occur for others ... unless there is something fundamentally different about his system ... perhaps even a clash with another program ... but why then only those zips?

There is certainly a mystery to it.

P.S. Does Python use the inbuilt zip ability of the OS or do something all its own? If the former, then maybe it is a system file issue that occurs with certain criteria.

Today i decided to do some experiments with gogrepo.py.

(1) The first one, was to see what was the least info I could get returned, just to get the list of my games. Having in mind the idea that it could be a faster FULL UPDATE, which i could then utilize to do a full update for each game in a batch manner, a bit at a time.

Alas, while I got no file information returned, the Changelog data was being returned, and some Changelogs are huge, so that was a spectacular fail. It would be nice if there was a parameter to skip the Changelog detail.

(2) Secondly, I decided to test out manually stopping a FULL UPDATE by closing the console window after about half a dozen games had been processed.

That was both good and bad. The good, was that I got a list of all my game titles (1,010) in the Resume manifest file (gog-resume-manifest.dat), and it was pretty quick. The bad, is that there was no extra data in it for those half a dozen games that were processed, and no manifest file.

(3) Thirdly, I ran gogrepo.py again, with the same commands, but with the additional -resume resume.

As I suspected, it loaded the saved resume manifest and skipped having to get the game titles again, but alas there was no extra data for those half a dozen games, and so it processed them again.

So I am left wondering what the point of resume is?

(4) Fourth, I did the same tests again, and used the soft close option of CTRL-C instead of the console window X.

That made no difference what-so-ever.

--------------------------------------------------------------------------

Unless I am missing something, something that is in the DEV version of gogrepo.py, then Resume doesn't do much, and doesn't appear to be helpful if you want to FULL UPDATE your manifest in stages, using regular methods.

--------------------------------------------------------------------------

That said, I did get one real positive idea out of it all, that should assist me in doing my own version of a FULL UPDATE in stages.

Basically I would monitor the output of a FULL UPDATE, and then kill the process after the Resume manifest had been saved with all the game titles.

I could then use those titles, to build a batch option that does the FULL UPDATE in stages ... perhaps doing 10, 20, 30 games at a time, using the -id parameter with multiple game titles. It might require me doing yet another window for my GUI or I may be able to get away with modifying the existing UPDATE window.

All this appears to lead to other possibilities, especially now that I have written and implemented the code for removing a game from the manifest. In short, it means I could do a batch delete option for a bunch of games, and then a FULL UPDATE for them, to pick up any new stuff, that may not have been given the update tag.