It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
avatar
_Auster_: (at least once it started downloading 9 GB worth of files when my external hard drive had only 2~3 GB of free space)
Aw, that's unfortunate. Did you cancel it straight away, or let it reach 0 free space? If latter, did it stop at first error, or continue trying to download on 0 space?

avatar
_Auster_: About saving in specific directories, the argument savedir mentioned in the GitHub page should let you download the files to specific folders.
Yeah, I saw that, but that would only specify a single directory. basically I'm trying to achieve something along the lines of:
Specify several directories
Download some games to Directory 1, then once capacity is reached (or at least a game number, if there's no capacity checking), automatically switch to Directory 2 and go on like that.

I have a feeling I'll be downloading hundreds games one by one regardless due to my situation.
Post edited January 09, 2022 by SargonAelther
avatar
SargonAelther: Aw, that's unfortunate. Did you cancel it straight away, or let it reach 0 free space? If latter, did it stop at first error, or continue trying to download on 0 space?
Neither cancel or let it completely fill the storage. I deleted some redundant files that I forgot to delete after I used them.

avatar
SargonAelther: Yeah, I saw that, but that would only specify a single directory. basically I'm trying to achieve something along the lines of:
Specify several directories
Download some games to Directory 1, then once capacity is reached (or at least a game number, if there's no capacity checking), automatically switch to Directory 2 and go on like that.

I have a feeling I'll be downloading hundreds games one by one regardless due to my situation.
Not sure what the system is, but on Linux, I'd run:
cd /path/to/gogrepo.py ; python3 gogrepo.py update -os windows linux mac -lang en bl ru gk sb ar br jp ko fr cn cz hu pt tr sk nl ro es pl it de da sv fi no ; python3 gogrepo.py download -id game_example_1 /first/download/path/ ; python3 gogrepo.py download -id game_example_2 /second/download/path/ ; python3 gogrepo.py download -id game_example_n /N/download/path/
The command tends to get pretty big like this, but I have the process mostly streamlined thanks to the native macros function my text editor has.
And the symbol ; in Linux commands separates multiple commands and makes so that after a command is done, the terminal will run the next command automatically.
avatar
SargonAelther: Does the script check the free space on the drive?
avatar
_Auster_: From my experience, no, it doesn't. (at least once it started downloading 9 GB worth of files when my external hard drive had only 2~3 GB of free space)

Edit: forgot a small detail and adding the explanation below.

About saving in specific directories, the argument savedir mentioned in the GitHub page should let you download the files to specific folders.
It doesn't check for free space at the moment, I could implement something but it wouldn't always be correct (some files will be updated rather than duplicated, some files might be deleted dependent on how you organise things, some files might fail to download and end up taking up "work" space in !downloading).

You can put a download limit on the amount to be downloaded though, and while that's not exact either it should work fairly well for this purpose. I'd make sure to leave a buffer of ~500 MB on top , so that there's room for the error in estimation + the logging. Add the argument

-downloadlimit X

to the download command where X is the limit in MB.
avatar
_Auster_: Not sure what the system is, but on Linux, I'd run:

cd /path/to/gogrepo.py ; python3 gogrepo.py update -os windows linux mac -lang en bl ru gk sb ar br jp ko fr cn cz hu pt tr sk nl ro es pl it de da sv fi no ; python3 gogrepo.py download -id game_example_1 /first/download/path/ ; python3 gogrepo.py download -id game_example_2 /second/download/path/ ; python3 gogrepo.py download -id game_example_n /N/download/path/
avatar
_Auster_: The command tends to get pretty big like this, but I have the process mostly streamlined thanks to the native macros function my text editor has.
And the symbol ; in Linux commands separates multiple commands and makes so that after a command is done, the terminal will run the next command automatically.
If you're only using a small handful of locations eg one disc for windows and another for linux for example, you can use multiple download commands each with a different specified download directory and the appropriate -lang/os/extra/game filters, and that saves having to manage individual paths. I've done this in the past (one disk for Windows / Extra, one for Linux / Mac )
Post edited January 09, 2022 by Kalanyr
avatar
_Auster_: Not sure what the system is, but on Linux, I'd run:

cd /path/to/gogrepo.py ; python3 gogrepo.py update -os windows linux mac -lang en bl ru gk sb ar br jp ko fr cn cz hu pt tr sk nl ro es pl it de da sv fi no ; python3 gogrepo.py download -id game_example_1 /first/download/path/ ; python3 gogrepo.py download -id game_example_2 /second/download/path/ ; python3 gogrepo.py download -id game_example_n /N/download/path/
avatar
_Auster_: The command tends to get pretty big like this, but I have the process mostly streamlined thanks to the native macros function my text editor has.
And the symbol ; in Linux commands separates multiple commands and makes so that after a command is done, the terminal will run the next command automatically.
Thanks. I am on Windows, so I may have to think about it and potentially come up with some additional commands.

You'd thing GOG would offer an official downloader with a full list of games and their extras, selectable by a tickbox, but alas.



avatar
Kalanyr: You can put a download limit on the amount to be downloaded though, and while that's not exact either it should work fairly well for this purpose. I'd make sure to leave a buffer of ~500 MB on top , so that there's room for the error in estimation + the logging. Add the argument

-downloadlimit X

to the download command where X is the limit in MB.
Oh, that's very interesting! Is that an official argument of gogrepo? I didn't see it on the GitHub page. I'll try it out. If it works well, it would go along way towards solving my problem.

I have a question though. How does gogrepo store data? Does it reserve the space for each file at the start of the download, or does it make files grow in size? The reason I ask is that, if there's no reservation, there's a chance the downloadlimit argument could cause corrupt/incomplete files, no?
avatar
SargonAelther: Thanks. I am on Windows, so I may have to think about it and potentially come up with some additional commands.

You'd thing GOG would offer an official downloader with a full list of games and their extras, selectable by a tickbox, but alas.
About Windows, the system probably has a similar argument as Linux's ; , and what changes about each command is that the beginning changes, where, if I'm not mistaken (but please correct me if I am), python3 gogrepo.py rest-of-the-command becomes just py gogrepo.py rest-of-the-command, and pip install html5lib html2text becomes py -m pip install html5lib html2text.

About GOG offering an official downloader, they did but apparently they axed it for the sake of Galaxy 1.0. And [edit] even if you try to install it, it doesn't work anymore. I just tried it on Win7 and it tries to connect infinitely.
Post edited January 09, 2022 by _Auster_
avatar
SargonAelther: Does the script check the free space on the drive?
No, but you can easily find out how much it is going to download, by running the "gogrepoc.py download" with the -dryrun option. It doesn't actually download anything then, but tells you how much it would download.

avatar
SargonAelther: Does the script support automatically storing games on multiple specified drives / directories?
No, but you can do it manually. I am downloading (and updating) my games to two separate USB hard drives, by manually splitting the updated manifest file to two separate manifest files, and then running gogrepo download/clean/verify twice towards two different paths.

When splitting the manifest file, make sure the syntax is correct in the split files, ie. the [ and ] at the start and the end of each manifest file.

When you run gogrepo update, you need to have the original, complete manifest file though or create a new complete manifest file, which you then need to split again before running download...

Yeah it is a bit of hassle but at least it can be done.
Post edited January 09, 2022 by timppu
I give up. I dont get the script to run....
avatar
Lebostein: I give up. I dont get the script to run....
Feel free to try some of the other alternatives:

https://github.com/Sude-/lgogdownloader
https://github.com/Magnitus-/gogcli

Haven't used it myself, but if you use Linux and don't need a plugable storage engine (ie, your filesystem is good enough for your needs), lgogdownloader may be the most mature tool of them all (they integrated with the GOG Galaxy apis which the other tools don't).

Otherwise, if you want to do your backups in either a filesystem or an s3 store (and potentially others in the future, though for now, its s3 or your file system), easily keep multiple backups updated and have a binary that just runs, feel free to use mine. The only pain is setting up the cookie initially (there are instructions in the readme), but in terms of dependencies, it is the lightest tool of the bunch.
avatar
Lebostein: I give up. I dont get the script to run....
I haven't had the chance to fold the patch in, been busy with other stuff, I'll try and do it today or tomorrow and post a notification here when I have.
avatar
Kalanyr: Try installing this: https://pypi.org/project/pyobjc/
avatar
Lebostein: Sorry, don't work:

18:02:12 | fatal...
Traceback (most recent call last):
File "gogrepoc.py", line 2777, in <module>
wakelock.take_wakelock()
File "gogrepoc.py", line 2685, in take_wakelock
self._PMerrcode, self._PMassertID = self._IOPMAssertionCreateWithName(a,self._kIOPMAssertionLevelOn,"gogrepoc")
File "gogrepoc.py", line 2643, in _IOPMAssertionCreateWithName
p_assert_name = self.raw_ptr(self._CFSTR(assert_name))
File "gogrepoc.py", line 2636, in _CFSTR
return CoreFoundation.CFStringCreateWithCString(None, py_string, CoreFoundation.kCFStringEncodingASCII)
ValueError: Expecting byte string of length 1, got a 'str'
This issue should be fixed now, thanks to https://github.com/stevensmedia .

(Side - Also fixed a pretty niche bug and made Wakelock on Windows more aggressive by default (this should help Surface owners)

I'd appreciate if if Mac users could test it on Python 3 / Python 2 and let me know your OS version, python version used to execute gogrepo and how things went.
avatar
Kalanyr: You can put a download limit on the amount to be downloaded though, and while that's not exact either it should work fairly well for this purpose. I'd make sure to leave a buffer of ~500 MB on top , so that there's room for the error in estimation + the logging. Add the argument

-downloadlimit X

to the download command where X is the limit in MB.
avatar
SargonAelther: Oh, that's very interesting! Is that an official argument of gogrepo? I didn't see it on the GitHub page. I'll try it out. If it works well, it would go along way towards solving my problem.
It's an official argument of gogrepoc , the documentation is unfortunately a bit out of date, I got distracted by other stuff half way through a big rewrite and haven't been keeping the documentation up to date, I really need to finish the rewrite and then update the docs.


avatar
SargonAelther: I have a question though. How does gogrepo store data? Does it reserve the space for each file at the start of the download, or does it make files grow in size? The reason I ask is that, if there's no reservation, there's a chance the downloadlimit argument could cause corrupt/incomplete files, no?
Gogrepoc currently reserves space on Windows system immediately before the file starts downloading, but grows files during download on Linux/Mac. This is because it was implemented originally to avoid fragmentation which is more of an issue on Windows Filesystems (and also because Linux/Mac don't have a good way to *really* reserve a block of contiguous space on a filesystem that works across the variety of fileystems that could be being written too)
Now it WORKS!!! Thanks!

Edit: It is possible to skip patch-installers?
Post edited February 13, 2022 by Lebostein
doh, double post
Post edited February 14, 2022 by neumi5694
If this is something unknown yet, I have something "urgent" for you.
Until yesterday I was using a older version of gogrepo, which worked just fine.
Since this morning however, I get this error when checking for new or updated games (some have been updated):
"xml parsing error occurred trying to get md5 data for setup_craft_the_world_1.9.006_2_(53668).exe"
This happens for all files and all games.

If you want something less dramatic, here's a wish: a file where we can store file name patterns to exclude would be nice, this way we could avoid the oversized DLCs for games like Succubus or Stellaris . Those have extras to download which are also available as DLCs for the game, but in the case of Succubus the extras have a file size of 280MBs together, while the DLC version is 2 x 30GB. It seems that all game files are integrated into the DLC installer, even if the different file is only a pdf.
A file for saving the patterns would be nice, so we don't have to type the filter in every time.
Post edited February 14, 2022 by neumi5694
I'm using the latest version (3rd Feb 2022) with Python 2.7.14 and I'm having the same issues:

12:44:03 | xml parsing error occurred trying to get md5 data for <removed>/setup_ys_i_2022-01-03_%2852678%29.exe
12:44:03 | chunk request has unexpected Content-Range. expected '0-698089735/698089736' received '0-698089735/708100728'. skipping.
12:44:03 | not moving uncompleted download '/datenpool/share/GOG/!downloading/ys_i/setup_ys_i_2020.08.10_(40401).exe', success: False remaining bytes: 698089736 / 698089736

12:44:08 | xml parsing error occurred trying to get md5 data for <removed>/patch_tooth_and_tail_1.7.0.1_%2839497%29_to_1.7.1.0_%2842691%29.exe
12:44:08 | chunk request has unexpected Content-Range. expected '0-4489263/4489264' received '0-4489263/6767320'. skipping.
12:44:08 | not moving uncompleted download '/datenpool/share/GOG/!downloading/tooth_and_tail/patch_tooth_and_tail_1.7.0_(39284)_to_1.7.0.1_(39497).exe', success: False remaining bytes: 4489264 / 4489264
Post edited February 14, 2022 by Tylak1