Yup, agreed on both counts.

Another question, is there a way to checkpoint a forceverify? Or clear all previously verified status so the normal verify can start over? In my case, I moved my repository to a new cloud location, and I'd like to reverify it cloud-side, but my session time is limited, and so I need to keep restarting.

EDIT: Followup question. What does the script do during a download operation between printing out the directory and printing out all the "pass" lines? In the aforementioned cloud setup, the first time through after starting the VM session, there's a large delay processing 1500 passes before downloading for instance 4 updates. If I run it again, it's as fast as running locally. I'm curious what I can do to speed it up. I tried for example stat'ing each file (ls -la */*), but that didn't improve performance. Thoughts?

Use tmux to keep processes running in a tmux session even if you are disconnected:

Install tmux, then run:

tmux

Now you are inside the tmux session, run your gogrepoc commands as usual.

If you get disconnected, then reconnect and:

tmux attach

to resume the tmux session.

If you want to detach the tmux session for whatever reason while the command is running, do:
CTRL-B CTRL-D
then
tmux attach
to get back.

No, I have thought about logging the date of last verify though, so that eg rolling checks to guard against bit rot were possible.

I'd have to check the code , I didn't think it did anything that would have per session effects that major that wouldn't be equally effected by ls.

I have checked the code now and I suspect what's going on is that it's caching all the !info.txt files, since it checks them to see if anything has changed and only writes to them if it has. The other thing it does is ask the OS for the size of each existing file in the directory that matches an entry in the manifest for that game but your LS should be caching that.

Preallocation hangs for my sometimes too. Apparently only when the file size is large like for cyberpunk*.21.bin.
Happened only after reinstalling WSL2 with new Debian inside. Starting gogrepoc.py in Debian (inside WSL2), writing to ext4 on a NAS. Will also comment out the lines later to make it work.

I'll add timeouts and make the Posix optional , I need to see if I can semi-reliably detect the target filesystem and current OS too.

I really don't like the timeouts because this is terrible behaviour from system calls and the state of the files and system handles is without guarantees in this scenario, should probably terminate and recommend disabling pre-alloc at that point probably v

Thanks. FWIIW commenting out works for me too. Downloading at the moment.

Thanks. Though I'm quite familiar with tmux, it's not allowed in this environment. There's no way to bypass the limitations, or even break out of the environment into your own terminal.

Hi,

I am completely no to this, so I apologize if this was asked before: I have many items in my library (free games etc.) that I would like to omit from the library backup. What is the best way to do this, so that these specific games are not downloaded?

Thank you in advance!

The easiest to do, in my opinion, is to hide the games you do not want to backup in your GOG library, then use the -skiphidden flag in your gogrepoc update command.

Good suggestion. It might be important to know that the Galaxy flags and hidden status are not synchronized with the homepage, so one has to hide the game through the homepage.

That's great advice, thank you!

I have discovered html5lib is no longer being actively maintained, going to look into using the built in parser ( GOGrepo doesn't do much complex parsing at this level but the GOG html layout can be idiosyncratic and we do need to get some complex stuff from it so maybe it will work) but figured I'd ask here:

Does anyone know what (if any). the current standard actively maintained python advanced html parser module is ?

Some quick poking suggests there isn't really anything.

Just for curiosity, but.. why, did it stop working?
Or does it miss important features?

Last commit dates from March: https://github.com/html5lib/html5lib-python

Its not bad. Its not a hot new project that lots of people are committing to daily, but I don't see a reason to believe urgent needed fixes wouldn't get done if such a need surfaced.

If you need something that is not getting done in the project, you could always fork it if you are feeling ambitious. Its not a tiny project, but it is not humongous either.

Otherwise, for html parsing in Python, for my use-case I had years ago, I used the standard library: https://docs.python.org/3/library/html.parser.html .

You need to grok your head around its inheritance model, but once you do, it is kind of elegant.