It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
I've updated the dev branch with some minor fixes, once of which should help a bit with the weird things happening on GOG.com at the moment due to the sale.
avatar
Kalanyr: I've updated the dev branch with some minor fixes, once of which should help a bit with the weird things happening on GOG.com at the moment due to the sale.
Dev branch? I'm guessing that's the one that reads "gogrepoc.py Various Fixes 16 hours ago" to us end users.

And more errors.

Commands
python gogrepo.py update -os windows -lang en -skipknown
python gogrepo.py update -os windows -lang en -updateonly
python gogrepo.py download -dryrun -skipextras > Downloads.txt

Log
18:10:51 | loading local manifest...
18:10:52 | loading local resume manifest...
18:10:52 | incomplete update detected, resuming...
18:10:52 | saving resume manifest...
18:10:52 | saved resume manifest
18:10:52 | (1 / 1) fetching game details for transistor...
18:11:20 | -> gog flagged this game as updated
18:11:20 | -> changelog was updated
18:11:20 | saving manifest...
18:11:20 | saved manifest
18:11:20 | saving resume manifest...
18:11:20 | saved resume manifest
18:11:20 | saving manifest...
18:11:21 | saved manifest
18:11:21 | saving resume manifest...
18:11:21 | saved resume manifest
18:11:21 | resume completed
18:11:21 | returning to specified download request...
18:11:21 | fatal...
Traceback (most recent call last):
File "gogrepo.py", line 2693, in <module>
main(process_argv(sys.argv))
File "gogrepo.py", line 2437, in main
cmd_update(args.os, args.lang, args.skipknown, args.updateonly, not args.full, args.ids, args.skipids,args.skiphidden,args.installers,args.resumemode,args.strictverify)
File "gogrepo.py", line 1330, in cmd_update
cmd_update(save_os_list, save_lang_list, skipknown, updateonly, ids, skipids,skipHidden,save_installers,resumemode,save_strict)
TypeError: cmd_update() missing 1 required positional argument: 'strict'
18:11:21 | loading local manifest...
18:11:22 | loading local resume manifest...
18:11:22 | fetching game product data (page 1)...
18:11:22 | fetching game product data (page 2 / 5)...
18:11:23 | fetching game product data (page 3 / 5)...
18:11:23 | fetching game product data (page 4 / 5)...
18:11:23 | fetching game product data (page 5 / 5)...
18:11:24 | no new game or updates found.
18:11:24 | --
18:11:24 | total time: 0:00:02.277604
18:11:24 | exiting...
18:11:24 | loading local manifest...
18:11:24 | scanning manifest for renames...
18:11:24 | Cleaning up .\!downloading
18:11:24 | fatal...
Traceback (most recent call last):
File "gogrepo.py", line 2693, in <module>
main(process_argv(sys.argv))
File "gogrepo.py", line 2458, in main
cmd_download(args.savedir, args.skipextras, args.skipids, args.dryrun, args.ids,args.os,args.lang,args.skipgalaxy,args.skipstandalone,args.skipshared, args.skipfiles)
File "gogrepo.py", line 1468, in cmd_download
for cur_dir in sorted(os.listdir(downloadingdir)):
FileNotFoundError: [WinError 3] The system cannot find the path specified: '.\\!downloading'
Post edited September 06, 2018 by SKARDAVNELNATE
avatar
Kalanyr: I've updated the dev branch with some minor fixes, once of which should help a bit with the weird things happening on GOG.com at the moment due to the sale.
avatar
SKARDAVNELNATE: Dev branch? I'm guessing that's the one that reads "gogrepoc.py Various Fixes 16 hours ago" to us end users.

And more errors.

Commands
python gogrepo.py update -os windows -lang en -skipknown
python gogrepo.py update -os windows -lang en -updateonly
python gogrepo.py download -dryrun -skipextras > Downloads.txt

Log

18:10:51 | loading local manifest...
18:10:52 | loading local resume manifest...
18:10:52 | incomplete update detected, resuming...
18:10:52 | saving resume manifest...
18:10:52 | saved resume manifest
18:10:52 | (1 / 1) fetching game details for transistor...
18:11:20 | -> gog flagged this game as updated
18:11:20 | -> changelog was updated
18:11:20 | saving manifest...
18:11:20 | saved manifest
18:11:20 | saving resume manifest...
18:11:20 | saved resume manifest
18:11:20 | saving manifest...
18:11:21 | saved manifest
18:11:21 | saving resume manifest...
18:11:21 | saved resume manifest
18:11:21 | resume completed
18:11:21 | returning to specified download request...
18:11:21 | fatal...
Traceback (most recent call last):
File "gogrepo.py", line 2693, in <module>
main(process_argv(sys.argv))
File "gogrepo.py", line 2437, in main
cmd_update(args.os, args.lang, args.skipknown, args.updateonly, not args.full, args.ids, args.skipids,args.skiphidden,args.installers,args.resumemode,args.strictverify)
File "gogrepo.py", line 1330, in cmd_update
cmd_update(save_os_list, save_lang_list, skipknown, updateonly, ids, skipids,skipHidden,save_installers,resumemode,save_strict)
TypeError: cmd_update() missing 1 required positional argument: 'strict'
18:11:21 | loading local manifest...
18:11:22 | loading local resume manifest...
18:11:22 | fetching game product data (page 1)...
18:11:22 | fetching game product data (page 2 / 5)...
18:11:23 | fetching game product data (page 3 / 5)...
18:11:23 | fetching game product data (page 4 / 5)...
18:11:23 | fetching game product data (page 5 / 5)...
18:11:24 | no new game or updates found.
18:11:24 | --
18:11:24 | total time: 0:00:02.277604
18:11:24 | exiting...
18:11:24 | loading local manifest...
18:11:24 | scanning manifest for renames...
18:11:24 | Cleaning up .\!downloading
18:11:24 | fatal...
Traceback (most recent call last):
File "gogrepo.py", line 2693, in <module>
main(process_argv(sys.argv))
File "gogrepo.py", line 2458, in main
cmd_download(args.savedir, args.skipextras, args.skipids, args.dryrun, args.ids,args.os,args.lang,args.skipgalaxy,args.skipstandalone,args.skipshared, args.skipfiles)
File "gogrepo.py", line 1468, in cmd_download
for cur_dir in sorted(os.listdir(downloadingdir)):
FileNotFoundError: [WinError 3] The system cannot find the path specified: '.\\!downloading'
avatar
SKARDAVNELNATE:
The dev branch uses gogrepoc.py not gogrepo.

I'll double check those errors tho.

As a general rule of thumb I would suggest not updating in incomplete states ie outstanding resumes and downloads.

I'll add a bit more padding
I don't suppose you have a copy of the resume manifest that lead to that error ? I think I know what happened but I'm not 100% sure.
I had changed the name of gogrepoc.py to gogrepo.py so I wouldn't have to change it in the batch commands I'm using.

I think I tracked down my particular problem. I have 2 locations.
D:\GOGrepo - this is the main archive where gogrepo.py and my batch files are located.
D:\GOGrepo\zzDownloads - this is for new downloads to be imported afterward.

And the newer version created some special folders.
!downloading
!orphaned

These folders were created in zzDownloads but I cleared out that folder after the last import job. GOGrepo apparently expects them to still be there. Once I recreated them the download and import commands started working again.

However this one still wasn't.
python gogrepo.py download -dryrun -skipextras > Downloads.txt

It seems GOGrepo also expects them in D:\GOGrepo. Once I copied the folders there I was able to get a normal Downloads.txt report.
Post edited September 06, 2018 by SKARDAVNELNATE
Pushed a new update that should hopefully resolve your cases (no !downloading directory created yet & me forgetting to test resume after changing the way update worked) . Let me know if it resolves your issues and works now.
avatar
SKARDAVNELNATE: I had changed the name of gogrepoc.py to gogrepo.py so I wouldn't have to change it in the batch commands I'm using.

I think I tracked down my particular problem. I have 2 locations.
D:\GOGrepo - this is the main archive where gogrepo.py and my batch files are located.
D:\GOGrepo\zzDownloads - this is for new downloads to be imported afterward.

And the newer version created some special folders.
!downloading
!orphaned

These folders were created in zzDownloads but I cleared out that folder after the last import job. GOGrepo apparently expects them to still be there. Once I recreated them the download and import commands started working again.

However this one still wasn't.
python gogrepo.py download -dryrun -skipextras > Downloads.txt

It seems GOGrepo also expects them in D:\GOGrepo. Once I copied the folders there I was able to get a normal Downloads.txt report.
Yeah, that's a bit of an edge case. I assumed people might split collections but not that they'd manually filter downloads.

If you could set up a copy of your filter setup and let me know when / where things break I can add appropriate padding.

You shouldn't need Downloads.txt either. One of the changes I made long ago was automatic rotating log files.
Post edited September 06, 2018 by Kalanyr
Tested by deleting the special folders in both locations.

Build manifest and Downloads.txt - created !orphaned folder in D:\GOGrepo
Download transistor - worked with one hiccup, !downloading and !orphaned folder in D:\GOGrepo\zzDownload
Import transistor - worked.

The hiccup.
19:45:49 | 0.0MB 12.7MB/s 1x transistor/patch_transistor_0.10113_(18323)_to_1.50440.8123_(23298).exe
19:45:49 | 2.41GB remaining
19:46:29 | request failed: HTTPSConnectionPool(host='cdn-hw.gog.com', port=443): Max retries exceeded with url: /secure/offline/1429612159/1429612159/51522173732619478/6945/setup_transistor_1.50440.8123_%2823298%29-1.bin?t tl=1536277837&hw_l=108&hw_token=0ec9bd137f25c1bde601c33f76ed80a5&_token=eyJhbGciOiJBMTI4S1ciLCJlbm MiOiJBMTI4Q0 JDLUhTMjU2In0.XcprV6lE6r7FjI-XuzCJDrIakcqYm_oMQ0XpzIMnl7eqifKEIl4ixQ.ekVqRySP4BWNzs7b-h945g.K-Ph2HowOEc8k_SK3L iqYMIx-KMGj8OI75tJhPsU2ySVNlMcjHfIWhhM8GrQZGY2qDEVjqSGOzdc9Ew1hYwfmg3OUV-U8VBHSkJeR0Pr9t0aLFtY8nvvIxYTxL3VUod- 2pRIs_5EF5dwgyeaXINyZxClzOp9DPAbHu2efiHqeBJY5v8VRFHOwRgqmM8WjWV19Lk6ovx3p9SbDnyGumnHeYDHa7uNkz4uC0NFSFdPzjKm0w qHfT1AL6pWKGBUw7vhmNgNdmBFjs_kKL2H7zxi8qLs84vNID5LgSLPJJkefCz_67PNex_5q_vKoA7m3B1Wu89_sGYdTR5NNXYQnB3gQZOaYiF4 hSYkNCvw8Gh4Ct683mCDwCB-OfpCTk4Z-MlRss9ynA8YakZYQv5am0gRTnKkY7HDouqdI6yjB-T4TZ-YcoAVKA_tPCqn7LoJyrCkNd98K2UymK Ne7EWjn0K5mLO-xqOgNXNOU6o94Np35axMiG3bnRZaB6ANY-OsKv-McZDYk6piMsMd-sOi4QnrZOnSJhkbv55UWEsVbgT-fu9b9oz_rZSkV-Do Atyk6ArbVhiXFEm8tGBjkleVsVLcVA.FQa64DaNlqdgNdckmbwJGQ&source=hw (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x0000000003C0A1D0>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond')) (3 retries left) -- will retry in 5s...
19:46:57 | 2464.0MB 10.5MB/s 1x transistor/setup_transistor_1.50440.8123_(23298)-1.bin
19:46:57 | 2.41GB remaining
avatar
Kalanyr: Yeah, that's a bit of an edge case. I assumed people might split collections but not that they'd manually filter downloads.

If you could set up a copy of your filter setup and let me know when / where things break I can add appropriate padding.

You shouldn't need Downloads.txt either. One of the changes I made long ago was automatic rotating log files.
I might take my games library a bit more seriously than most people. Here is my approach to GOG.

Build Manifest
python gogrepo.py update -os windows -lang en -installers standalone -skipknown
python gogrepo.py update -os windows -lang en -installers standalone -updateonly
python gogrepo.py download -dryrun -skipextras > Downloads.txt

I just added the -installers standalone. Occasionally I delete gog-manifest.dat and start a new one since it seems some updates don't get flagged.

Then I check Downloads.txt and note games that have anything marked for download.

zzDownload
python gogrepo.py download -skipextras -id _____ D:\GOGrepo\zzDownload

-skipextras is removed for a recently purchased game.

Afterward I copy the zzDownload folder to a different computer. There I run each installer, note their performance, and add them to my sorted archive. Back on the GOGrepo computer I delete the old game folder from the main archive and run import.

python gogrepo.py import "D:\GOGrepo\zzDownload" "D:\GOGrepo"
python gogrepo.py download -dryrun -skipextras > Downloads.txt

If there is still something marked for download it tells me that something was wrong with either the file or the information pulled from GOG during Build Manifest.
Post edited September 06, 2018 by SKARDAVNELNATE
Running a new manifest build and encountering a lot of errors.
python gogrepo.py update -os windows -lang en -installers standalone -skipknown

18:58:49 | ( 12 / 410) fetching game details for aliens_versus_predator_classic_2000...
18:58:51 | request failed: 404 Client Error: Not Found for url: _____. will not retry.
18:58:51 | failed to fetch https://www.gog.com/downloads/aliens_versus_predator_classic_2000/en1galaxy_installer0
18:58:53 | saving manifest...
18:58:53 | saved manifest
18:58:53 | saving resume manifest...
18:58:53 | saved resume manifest

18:59:30 | ( 25 / 410) fetching game details for ashes_of_the_singularity...
18:59:36 | request failed: 404 Client Error: Not Found for url: _____. will not retry.
18:59:36 | failed to fetch https://www.gog.com/downloads/ashes_of_the_singularity/en1galaxy_installer0
18:59:37 | request failed: 404 Client Error: Not Found for url: _____. will not retry.
18:59:37 | failed to fetch https://www.gog.com/downloads/ashes_of_the_singularity/en1galaxy_installer1
18:59:38 | request failed: 404 Client Error: Not Found for url: _____. will not retry.
18:59:39 | failed to fetch https://www.gog.com/downloads/ashes_of_the_singularity/66543
18:59:49 | saving manifest...
18:59:49 | saved manifest
18:59:49 | saving resume manifest...
18:59:49 | saved resume manifest
18:59:49 | ( 26 / 410) fetching game details for ashes_of_the_singularity_escalation...
18:59:57 | request failed: 404 Client Error: Not Found for url: _____. will not retry.
18:59:57 | failed to fetch https://www.gog.com/downloads/ashes_of_the_singularity_escalation/en1galaxy_installer0
18:59:58 | request failed: 404 Client Error: Not Found for url: _____. will not retry.
18:59:58 | failed to fetch https://www.gog.com/downloads/ashes_of_the_singularity_escalation/en1galaxy_installer1
18:59:58 | request failed: 404 Client Error: Not Found for url: _____. will not retry.
18:59:58 | failed to fetch https://www.gog.com/downloads/ashes_of_the_singularity_escalation/en1galaxy_installer2
18:59:59 | request failed: 404 Client Error: Not Found for url: _____. will not retry.
19:00:00 | failed to fetch https://www.gog.com/downloads/ashes_of_the_singularity_escalation/en1galaxy_installer4
19:00:12 | saving manifest...
19:00:12 | saved manifest
19:00:12 | saving resume manifest...
19:00:12 | saved resume manifest

19:00:25 | ( 29 / 410) fetching game details for baldurs_gate_2_enhanced_edition...
19:00:28 | request failed: 404 Client Error: Not Found for url: _____. will not retry.
19:00:28 | failed to fetch https://www.gog.com/downloads/baldurs_gate_2_enhanced_edition/en1galaxy_installer1
19:00:31 | saving manifest...
19:00:31 | saved manifest
19:00:31 | saving resume manifest...
19:00:31 | saved resume manifest
Roughly 1/5th of the log is like that.

I removed the long URLs for readability but there is something odd going on with them.
This is the installer for aliens_versus_predator_classic_2000 taken from the source URL.
setup_avp_classic_20160330_%282149%29.exe

This is what GOGrepo was looking for.
setup_avp_classic_20160330_%282149%29_%28g%29.exe
Post edited September 10, 2018 by SKARDAVNELNATE
Original gogrepo.py was updated recently (if nobody posted that before).

Also - what s the difference in between "gogrepo.py" and "gogrepoawake.py"? (if Im not mistaken, this repo is the most feature-rich fork, as for now). 1st is main branch, while 2nd is just fixed original?
gogrepoawake.py tries to prevent your PC from going to sleep - which is very useful if you have a large collection taking hours/days to download/verify ;)
avatar
Gekko_Dekko: Original gogrepo.py was updated recently (if nobody posted that before).

Also - what s the difference in between "gogrepo.py" and "gogrepoawake.py"? (if Im not mistaken, this repo is the most feature-rich fork, as for now). 1st is main branch, while 2nd is just fixed original?
Interesting. Both of these have already been fixed on my branch I'll fix.

Eddie / Wooly:
If you're reading this t, I'm happy to return this thread to you and/or work out a way to remerge these projects.
Post edited September 13, 2018 by Kalanyr
Is there a way to skip all the patches (patch_* files)? I'm only interested in backing up the full installers.
avatar
jabba1971: Is there a way to skip all the patches (patch_* files)? I'm only interested in backing up the full installers.
I presume this version at least should have that option:

https://www.gog.com/forum/general/gogrepopy_python_script_for_regularly_backing_up_your_purchased_gog_collection_for_full_offline_e/post1589

Not sure if this was already merged to kalanyr's version. EDIT: Apparently it was, to the dev branch?
Post edited September 12, 2018 by timppu