Looks like I have to add a trash flag for cleaning up bg/cover images too now that I orphan those by default. GOG updates the bg images significantly more often than I thought, so they'll spam your orphan folder pretty quickly without a clean up option.

I do apologise for how long this is taking, m whatever they did to change the last-modified if the extras seems to have hit at least 60% of my extras, it finished downloading today but it's still verifying.

ETA - Turns out only 797 of my 5340 extras didn't get a new Last-Modified, so ~85% of my extras were. Ouch.

Hey no worries, we're already happy that you're still working on this useful script after all this time.

Very much this!

Testing looks good. I'll add that trash subparameter for managing orphaned images and then upload what i've got to Dev. That'll probably be sometime tomorrow.

Then I'll look into the archive set stuff.

ETA - Found the time to do that, is now available for testing on Dev , I'll get the archive test stuff implemented and up as soon as I can.

Does anyone own Bard's Tale 4 Director's Cut and a computer with <32 GB of RAM, and a ~90+ GB of free disk space ? It's fine, maybe even preferable if a significant amount of the RAM is used by the OS / other apps. Preferably running Windows for now.

I may need to do some testing on whether the best way to deal with a multi-archive set is to concatenate it in memory or write it to a temp file.

Just thought I'd try connecting my NTFS formatted USB drive to a Raspberry Pi (with a powered hub) and run gogrepoc on that.

The preallocation steps take a looong time for the multi-gigabyte files. Is that to be expected?
Not a major problem. It just surprised me. And everything seems to otherwise be working fine.

Happens to me with all Linux PCs (also RPi4) when connecting a NTFS USB drive to it for gogrepoc.

The preallocation still succeeds in your case, just takes a long time? Last time I tried it would time out with an error so I always have to disable manually the preallocation stuff in the script. I haven't tried the latest version of the script though, in case there is any change regarding that.

That's why I've asked to make it optional to preallocate. If it causes problems, one could easily disable it and be done with it.

It depends on the file system, if it's done optimally it should be mostly file size independent but a lot of file systems seem to reserve space by actually writing empty files (I assume there's historical / security reasons for this ? You could theoretically preserve deliberately deleted data if you just directly allocated clusters without overwriting them and magnetic disk file systems didn't usually automatically clear sectors on delete for performance reasons ) so the time taken to preallocate is directly proportional to the file size on those.

So I have the multi part zip archive (provisionally*) sorted out now but while doing research for this I learn that I can also test tar.gz files easily so I'll implement that and that I can possibly do rar files ( zip / tar.gz and rar are the 3 archive types that GOG uses at the moment ) however python has no native handling of rar and this is very tricky to deal with due to licensing issue.

There are a couple of different ways I can handle rars and I wondered if anyone had any feedback the first is patool , which hooks an appropriate tool on your system ( however on Linux you probably need to install an additional package because most distro repository versions of 7z don't come with rar support so you either need to install 7z-rar or winrar ) or to use a python lib that binds to the rar dynamic link library that RARLabs provides. I'm inclined to go with the first since it mostly abstracts the complexity from the user (they don't need to muck around with providing the DLL and binding ) but I'm willing to here. In either case the library will be optional and I catch an Error indicating that the module isn't configured correctly for support of rar I'll just warn and skip.

* I have successfully assembled and tested the archive in memory and it works but the assembled file is 52 GB and I think assuming everyone has what is almost certainly in practice 96 -128 GB of RAM is probably pretty questionable so I dunno how well this will work when swap files inevitably have to come in to play.

Linux Mint 21

[~]$ python3 --version
Python 3.10.12

not sure what LM22 uses

I'll be playing with some other distros over the next couple weeks. I'll post here what versions they use

Thank you for this but some one already did the testing for me for Linux for the issue in question.

::thumbs up::

Sorry for the late reply.
But yes, I think it succeeded. No errors were reported at least.



Thanks for the explanation, Kalanyr.

Feature request:

GOG seems to be adding more and more meaningless entries to some games. For example:

18:47:22 | no known filename for "blood_west (Blood West)"
18:47:33 | no known filename for "hacknet (Labyrinths DLC)"
18:47:33 | no known filename for "iratus_lord_of_the_dead (additional skins for all minions)"
18:47:33 | no known filename for "jagged_alliance_2_wildfire (Jagged Alliance 2: Wildfire)"
18:47:33 | no known filename for "omerta_city_of_gangsters (Omerta - City of Gangsters - The Bulgarian Colossus DLC)"
18:47:33 | no known filename for "omerta_city_of_gangsters (The Arms Industry DLC)"
18:47:33 | no known filename for "omerta_city_of_gangsters (Damsel In Distress DLC)"
18:47:33 | no known filename for "omerta_city_of_gangsters (The Con Artist DLC)"
18:47:33 | no known filename for "shadow_tactics_blades_of_the_shogun (Shadow Tactics: Blades of the Shogun)"
18:47:46 | no known filename for "stray_gods_the_roleplaying_musical (Stray Gods: The Roleplaying Musical)"
18:47:46 | no known filename for "warhammer_skulls_2022_digital_goodie_pack_goodies (Warhammer Chaos Gate)"

Perhaps gogrepoc can somehow learn to ignore these kind of entries.