It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
avatar
phaolo: Mm.. I don't see such option.
I really hope that the manifest will skip the hidden games next time.
For now, I deleted their entry manually from the file.
https://www.gog.com/forum/general/gogrepopy_python_script_for_regularly_backing_up_your_purchased_gog_collection_for_full_offline_e/post506

So it seems it is not optional, but gogrepo update always skips hidden games.

But if such game is already in your manifest file, then you either have to delete it once from the manifest file manually, or delete the manifest file and get a whole new one with the update command. At least according to that message, not sure if it was made smarter later.

avatar
phaolo: Also, I wish I could independently skip extras too. For example:
- rott_2013_level_editor.zip --> 8Gb! Too big for my interest in it.
- wing_commander_4_dos_german_version.zip --> 3Gb, not my language.
(etc)
You mean per game? I'd think a blacklist (with e.g. regex support) would be better, after all for some game you might want some of the extra files, but not one of them, or whatever. Also some people said they don't want to download patches, so they could also blacklist any files which include the "patch_" string in the filename.

Anyway, one workaround for now that someone suggested would be to modify the script a bit so that it doesn't perform a size check for the download (instructions for the change were in some earlier message in this thread), and replace the unwanted files with dummy files whose size is 0 kB but the filename is the same. Then they wouldn't be downloaded because the download commands sees the empty dummy files there and thinks the files exist already.
Post edited October 02, 2017 by timppu
avatar
timppu: So it seems it is not optional, but gogrepo update always skips hidden games.
But if such game is already in your manifest file, then you either have to delete it once from the manifest file manually, or delete the manifest file and get a whole new one
Ah, that's good.

avatar
timppu: You mean per game? I'd think a blacklist (with e.g. regex support) would be better, after all for some game you might want some of the extra files, but not one of them, or whatever.
Also some people said they don't want to download patches, so they could also blacklist any files which include the "patch_" string in the filename.
Yeah, I meant some kind of customizable blacklist file.
For main installers would be good too (e.g: x86 version when x64 exists).

--

Btw, the "dryrun" option shows the GBs that will be downloaded, but not WHICH games are going to be downloaded.
Not super convenient.
Post edited October 02, 2017 by phaolo
avatar
kbnrylaec: gogrepo.py is quite unstable for slow connection.

While I can surf on the web without problem,
gogrepo.py often regrab the same piece of data again and again and again,
make it forever to download a really big single file.

So, I mixed wget and gogrepo.py.
Grab big files with wget, and small files with gogrepo.py.

My GOG offline backup have 2.39 TiB currently.
How exactly did you have gog repo do wget instead? I would like to do that as well
avatar
Revan67: How exactly did you have gog repo do wget instead? I would like to do that as well
wget -c --load-cookies [cookie file for gog] [URL of the installer]
You may do similar things with curl or other tools.

[URL of the installer] are listed in gog-manifest.dat, and in https://www.gog.com/account
Post edited October 04, 2017 by kbnrylaec
Has anybody seen this before?

10:14:51 | (265 / 626) fetching game details for kyn...
10:14:56 | request failed: HTTP Error 404: Not Found. will not retry.
10:14:56 | no md5 data found for setup_kyn_2.1.0.4.exe
10:14:59 | request failed: HTTP Error 404: Not Found. will not retry.
10:14:59 | no md5 data found for setup_kyn_2.1.0.4-1.bin

I assume it just means that it can't find the md5 data, it still shows up in the manifest, have not tried to download yet
avatar
Revan67: Has anybody seen this before?

10:14:51 | (265 / 626) fetching game details for kyn...
10:14:56 | request failed: HTTP Error 404: Not Found. will not retry.
10:14:56 | no md5 data found for setup_kyn_2.1.0.4.exe
10:14:59 | request failed: HTTP Error 404: Not Found. will not retry.
10:14:59 | no md5 data found for setup_kyn_2.1.0.4-1.bin

I assume it just means that it can't find the md5 data, it still shows up in the manifest, have not tried to download yet
A few games have no md5 data and always throw that error on a clean manifest update. It will download and SHOULD be fine, unless you run into some of those pesky download issues from slow connections others mentioned.
avatar
Revan67: Has anybody seen this before?

10:14:51 | (265 / 626) fetching game details for kyn...
10:14:56 | request failed: HTTP Error 404: Not Found. will not retry.
10:14:56 | no md5 data found for setup_kyn_2.1.0.4.exe
10:14:59 | request failed: HTTP Error 404: Not Found. will not retry.
10:14:59 | no md5 data found for setup_kyn_2.1.0.4-1.bin

I assume it just means that it can't find the md5 data, it still shows up in the manifest, have not tried to download yet
Yeah, on my collection (1398 games now) there are two games where the md5 data can't be obtained from gog.com: "Kyn" and "A Golden Wake". There are also a couple others like X3 The Terran War Pack, but there the md5 checksum is missing only from some patch file, not the main installers, so I don't care about those.

The games will download fine but when you run "gogrepo verify", those games can't be verified against a md5 checksum, but it is merely checked that their size is correct (so that they were downloaded completely). So to be absolutely sure those installers are fine, you should check the installers manually (e.g. by installing the games). Thankfully there are only few of them.

I was thinking earlier that in case more games started missing md5 checksums, would one option for "gogrepo verify" be that it could run "innoextract -t" for the installer (innoextract tests that the installer uncompresses fine), if there is no md5 checksum? Of course then the user would have to add innoextract.exe to the path himself, but at least that way all installers would be verified completely, even those missing the md5 checksum.
Post edited October 04, 2017 by timppu
Crazy idea.. to store games as I wish, I could maybe create a folder full of symbolic links with relative paths.
I'd need to create a script and a relations table, however.
I wonder if this could work.

-Gogrepo simlinks
--gog_game_name_0001
--gog_game_name_0002
--gog_game_name_NNNN
-Games
--my game name 0001- year- developer (gog)
-----main setup
-----mods
-----savegames
-----etc
--my game name 0002- year- developer (store\retail\free\etc)
--my game name XXXX- year- developer (store\retail\free\etc)

With this method:
- previous games will be still automatically updated in the correct folders.
- new games will be wrongly added to the simlink dir, but I'll catch their folders by looking at the creation date.
- renamed dirs by Gog.. well, it depends if I'll modify them or if gogrepo will do it.
- extras would require way too much work and I'd prefer them extracted and organized anyway (aka manual).

----

Or.. maybe I could just give up with auto updates, follow the update thread and download each new\modified game with gogrepo singularly.
Post edited October 06, 2017 by phaolo
With the latest Woolymethodman gogrepo.py the update command doesn't advance past the first game.

Servers might be filled but im not sure... anyone else know if that other fork is still being worked on or is viable?

any updates on the Python 3 version? haven't looked in a long time and this is the first thats gogrepo has given me this much trouble.
avatar
Starkrun: With the latest Woolymethodman gogrepo.py the update command doesn't advance past the first game.
Servers might be filled but im not sure... anyone else know if that other fork is still being worked on or is viable?
Umm.. 1 month ago I tried the original to download all my games and it worked normally.
Are you sure you didn't make some mistake?

I never tried the fork, instead.
I completed a full manifest update and download about three days ago and it worked fine.
avatar
phaolo: Umm.. 1 month ago I tried the original to download all my games and it worked normally.
Are you sure you didn't make some mistake?
avatar
ikrananka: I completed a full manifest update and download about three days ago and it worked fine.
Deleted my current manifest and redid it, something went wonky in the old one.
Post edited October 14, 2017 by Starkrun
avatar
Starkrun: With the latest Woolymethodman gogrepo.py the update command doesn't advance past the first game.
I just started full update (deleted the old manifest file first), and it seems to work fine (fetched file details for 5 games, then I cancelled the script).

However, there was some anomaly with login cookies. When I first ran the update using my existing login cookie, it gave this error:

C:\gogrepo\gogrepo-master>python gogrepo.py update -os windows -lang en
11:18:51 | loading local manifest...
11:18:51 | fetching game product data (page 1)...
11:18:51 | failed to load product data (are you still logged in?)
So it appeared as if my existing gogrepo login cookie, which I've been using for many many months, is suddenly invalid? Also I am unsure if it somehow affected it that at the same time I was still logged into my GOG account also with Firefox? Probably this doesn't affect it...

Anyway, I deleted (renamed) my old gogrepo login cookie, and ran gogrepo login again. It logged in fine, and after that gogrepo update seemed to work fine. One more odditity was that when I logged into GOG with Firefox afterwards, GOG.com login triggered the CAPTCHA for me. I have no idea what are all the things that trigger that and why it was triggered now. Oh well, fortunately it wasn't triggered when I logged in with gogrepo...

avatar
Starkrun: any updates on the Python 3 version? haven't looked in a long time and this is the first thats gogrepo has given me this much trouble.
What do you mean? I think gogrepo has supported python 3 for a long time, and nowadays it is even recommended you use python 3 with it (not python 2).

I am using only python 3 with gogrepo. There may still be a remark on the gogrepo github page that python 3 is in the to-do list, but I think that is an error. It has supported python 3 for a long time already.
avatar
Starkrun: Deleted my current manifest and redid it, something went wonky in the old one.
Ah ok. I do that every time, delete the old manifest file and reload all file details all over again, just so that I receive also all the silent updates for GOG games.
Post edited October 14, 2017 by timppu
avatar
timppu: So it appeared as if my existing gogrepo login cookie, which I've been using for many many months, is suddenly invalid? Also I am unsure if it somehow affected it that at the same time I was still logged into my GOG account also with Firefox? Probably this doesn't affect it...
gogrepo.py do record expire date on its cookie file.
avatar
timppu: So it appeared as if my existing gogrepo login cookie, which I've been using for many many months, is suddenly invalid? Also I am unsure if it somehow affected it that at the same time I was still logged into my GOG account also with Firefox? Probably this doesn't affect it...
avatar
kbnrylaec: gogrepo.py do record expire date on its cookie file.
It's 1 year for me. It should be possible to manipulate the "expires" date using a simple text editor, but I have no idea if the server would accept this...