It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
Community
General discussiongogrepo.py - Python script for regularly backing up your purchased GOG collection for full offline enjoyment(3896 posts)(3896 posts)
(3896 posts)
Pages:
1
50
64
65
66
67
68
150
200
250
260
This is my favourite topic
Posted November 16, 2017
After replacing the parentheses and other symbols in my manifest, it still wants to download over 450GB of games with the %28, %29 characters.

So I'm baffled. I've downloaded the fork that supposedly fixes this and am running a new manifest right now, but that will take a few hours to complete and I'm at work, so I won't see this until I get home.
Posted November 17, 2017
Is anyone here using my fork ? I've added the unquote fix and can push that out if people are using it, but the verification change is half done. It won't actually *break* anything (it's just lacking several bits to make it functional) but I feel kind of uncomfortable rolling out code with half done code path ways in it.
Posted November 18, 2017
avatar
Kalanyr: Is anyone here using my fork ?
Yes, I am using your fork.
Posted November 18, 2017
avatar
Kalanyr: Is anyone here using my fork ?
avatar
solar_dome: Yes, I am using your fork.
Okay, then. I'm just about done with a "quick" manifest update / download test (which has worked fine so far) , then I'll fix the couple of tiny issues pointed out to me earlier in the thread and push it out. I'll post here / or edit this post when it's done. Apologies for the wait.
Posted November 18, 2017
avatar
Kalanyr: Okay, then. I'm just about done with a "quick" manifest update / download test (which has worked fine so far) , then I'll fix the couple of tiny issues pointed out to me earlier in the thread and push it out. I'll post here / or edit this post when it's done. Apologies for the wait.
Are the commands the same on your fork? I may try it out later if they are since I'm lazy and setup batch files to run my gogrepo.
Posted November 18, 2017
avatar
Kalanyr: Okay, then. I'm just about done with a "quick" manifest update / download test (which has worked fine so far) , then I'll fix the couple of tiny issues pointed out to me earlier in the thread and push it out. I'll post here / or edit this post when it's done. Apologies for the wait.
avatar
paladin181: Are the commands the same on your fork? I may try it out later if they are since I'm lazy and setup batch files to run my gogrepo.
I think I've preserved one for one backwards compatibility* (assuming you use Windows / English / Standalone installers. You may have to make some minor adjustments otherwise).
I've added some extra functionality that might reduce some of the need for your batch files depending what they do though (it's generally more fine grained control of exclusion / inclusion). Have a look at the help.

*The commands may do some extra stuff like logging, but they should carry out their original purpose and side effects should be relatively minor.

I also think this update will fix that outstanding issue where Python3 no longer worked for my fork after the last update (sorry about that Python3 peeps). I'm just testing that at the moment.

Edit - Yes, it does fix the Python3 issue. Not however that Python2 manifest files probably are not compatible with python3 and may need to be regenerated (chances are excellent that a Python2 manifest contains a size that was a Long in python2 (encoded as size: xxxxxxL) , python3 doesn't have longs (since the long and int types have been unified). Unfortunately Python3 can't recognize a Python2 Long stored in manifest and treat it as an Int. Python2 on the other hand transparently turns any Long sized Int into a Long automatically). So basically don't use Python2 / Python3 interchangeably on the same manifest if you want to avoid grief.
Post edited November 18, 2017 by Kalanyr
Posted November 18, 2017
Okay my fork has been updated here:

https://github.com/Kalanyr/gogrepo/tree/working2

Apologies for the delay again. Had some issues with my GOG HDD. And then spend some time trying to work out if I could fix the Python2/3 issue with encoded Longs programatically but I couldn't find anything useful on this issue.
Post edited November 18, 2017 by Kalanyr
Posted November 18, 2017
I wish that you devs could work on the same project, so we wouldn't have N gogrepo versions, each with separate improvements O_o
Posted November 18, 2017
avatar
phaolo: I wish that you devs could work on the same project, so we wouldn't have N gogrepo versions, each with separate improvements O_o
Just to be clear , anyone is welcome to add the stuff from my fork to their version, especially if someone wants to try and put together a unified version.

I'm also open to pull requests to add functionality to my fork.
Posted November 18, 2017
avatar
phaolo: I wish that you devs could work on the same project, so we wouldn't have N gogrepo versions, each with separate improvements O_o
For me I have basically no idea what I'm doing so pull requests and the like seem presumptuous. I very nearly didn't bother posting my version cause I assumed it would be of no interest.

As another thing think of each version as a recipe and github a recipe site the original recipe writer isn't on that site any more and they are the natural goto to get your custom change into a definitive version. Each of us are working on different thing and perhaps folks won't want my horrific mess added to theirs or they don't think its a worthwhile feature. Technically we are all working on the same recipe we just aren't putting them all on the same page. All you need to do is copy and paste the ones you want to make your own.

Basically Kalanyr is your best choice for a simple grab this version as they have the longest track record of working on stuff. That said merging the different versions is probably fairly easy in many cases. Grab a merge tool say meld or the like then check there are no conflicts and copy the bits across to make your definitive version.
Posted November 19, 2017
Guys I'm getting a list of garbage in the names of my gog games. things like setup_little_nightmares_1.0.29.1_%2816240%29.exe what's the %2816240%29 all about?


Oh I just been reading about the bad file names. I recently had to do a system restore and though it might've had something to do with that.
Post edited November 19, 2017 by Magmarock
Posted November 19, 2017
Hi I'm using Kalanyr's working2 gogrepo and getting the following errors when using verify, import and backup

PS D:\GOGInstall> D:\python27\python.exe gogrepo.py verify D:\GOGInstall | tee postVerify.txt
08:36:17 | loading local manifest...
08:36:17 | verifying all known files in the manifest
08:36:17 | verifying ad_2044\setup_ad2044_1.0_(15637).exe...
08:36:23 | fatal...
Traceback (most recent call last):
File "gogrepo.py", line 1727, in <module>
main(process_argv(sys.argv))
File "gogrepo.py", line 1700, in main
cmd_verify(args.gamedir, args.skipextras,args.skipids,check_md5, check_filesize, check_zips, args.delete, args.clean,args.ids, args.os, args.lang,args.skipgalaxy,args.skipstandalone,args.skipshared)
File "gogrepo.py", line 1536, in cmd_verify
handle_game_updates(items[item_idx], item)
NameError: global name 'item' is not defined

PS D:\GOGInstall> D:\python27\python.exe gogrepo.py import F:\GOGInstall D:\GOGInstall
07:54:07 | loading local manifest...
07:54:07 | collecting md5 data out of the manifest
07:54:07 | fatal...
Traceback (most recent call last):
File "gogrepo.py", line 1727, in <module>
main(process_argv(sys.argv))
File "gogrepo.py", line 1679, in main
cmd_import(args.src_dir, args.dest_dir,args.os,args.lang,args.skipextras,args.skipids,args.ids,args.skipgalaxy,args.skipstandalone,args .skipshared)
File "gogrepo.py", line 984, in cmd_import
_ = item.galaxyDownloads
NameError: global name 'item' is not defined

07:51:51 | fatal...
Traceback (most recent call last):
File "gogrepo.py", line 1727, in <module>
main(process_argv(sys.argv))
File "gogrepo.py", line 1716, in main
cmd_backup(args.src_dir, args.dest_dir,args.skipextras,args.os,args.lang,args.ids,args.skipids,args.skipgalaxy,args.skipstandalone,args .skipshared)
AttributeError: 'Namespace' object has no attribute 'ids'

Everything else appears tto be working fine. I have tried this under python2 and 3 with the same results. The errors with backup and import were present on the previous build.
Posted November 19, 2017
avatar
Pitney_b: Hi I'm using Kalanyr's working2 gogrepo and getting the following errors when using verify, import and backup

PS D:\GOGInstall> D:\python27\python.exe gogrepo.py verify D:\GOGInstall | tee postVerify.txt
08:36:17 | loading local manifest...
08:36:17 | verifying all known files in the manifest
08:36:17 | verifying ad_2044\setup_ad2044_1.0_(15637).exe...
08:36:23 | fatal...
Traceback (most recent call last):
File "gogrepo.py", line 1727, in <module>
main(process_argv(sys.argv))
File "gogrepo.py", line 1700, in main
cmd_verify(args.gamedir, args.skipextras,args.skipids,check_md5, check_filesize, check_zips, args.delete, args.clean,args.ids, args.os, args.lang,args.skipgalaxy,args.skipstandalone,args.skipshared)
File "gogrepo.py", line 1536, in cmd_verify
handle_game_updates(items[item_idx], item)
NameError: global name 'item' is not defined

PS D:\GOGInstall> D:\python27\python.exe gogrepo.py import F:\GOGInstall D:\GOGInstall
07:54:07 | loading local manifest...
07:54:07 | collecting md5 data out of the manifest
07:54:07 | fatal...
Traceback (most recent call last):
File "gogrepo.py", line 1727, in <module>
main(process_argv(sys.argv))
File "gogrepo.py", line 1679, in main
cmd_import(args.src_dir, args.dest_dir,args.os,args.lang,args.skipextras,args.skipids,args.ids,args.skipgalaxy,args.skipstandalone,args .skipshared)
File "gogrepo.py", line 984, in cmd_import
_ = item.galaxyDownloads
NameError: global name 'item' is not defined

07:51:51 | fatal...
Traceback (most recent call last):
File "gogrepo.py", line 1727, in <module>
main(process_argv(sys.argv))
File "gogrepo.py", line 1716, in main
cmd_backup(args.src_dir, args.dest_dir,args.skipextras,args.os,args.lang,args.ids,args.skipids,args.skipgalaxy,args.skipstandalone,args .skipshared)
AttributeError: 'Namespace' object has no attribute 'ids'

Everything else appears tto be working fine. I have tried this under python2 and 3 with the same results. The errors with backup and import were present on the previous build.
Thanks for the report, I appreciate them.

Going to sleep for a bit but I'll look into it when I get back.

Eta- The verify thing is me missing the need to adjust to a slightly different variable name for a new context, as is import. Those are easy to fix. I've already fixed them locally.

Backup looks like I decided to add a new filtering condition and then completely forgot to add it to the parser, I'm afraid that one will have to wait until I'm awake enough to give proper attention to the code path and make sure everything does work properly.
Post edited November 19, 2017 by Kalanyr
Posted November 19, 2017
thanks for the quick response. i'll revert to my old script for the moment so there is no rush.
Posted November 19, 2017
avatar
paladin181: Are the commands the same on your fork? I may try it out later if they are since I'm lazy and setup batch files to run my gogrepo.
avatar
Kalanyr: I think I've preserved one for one backwards compatibility* (assuming you use Windows / English / Standalone installers. You may have to make some minor adjustments otherwise).
I've added some extra functionality that might reduce some of the need for your batch files depending what they do though (it's generally more fine grained control of exclusion / inclusion). Have a look at the help.

*The commands may do some extra stuff like logging, but they should carry out their original purpose and side effects should be relatively minor.

I also think this update will fix that outstanding issue where Python3 no longer worked for my fork after the last update (sorry about that Python3 peeps). I'm just testing that at the moment.

Edit - Yes, it does fix the Python3 issue. Not however that Python2 manifest files probably are not compatible with python3 and may need to be regenerated (chances are excellent that a Python2 manifest contains a size that was a Long in python2 (encoded as size: xxxxxxL) , python3 doesn't have longs (since the long and int types have been unified). Unfortunately Python3 can't recognize a Python2 Long stored in manifest and treat it as an Int. Python2 on the other hand transparently turns any Long sized Int into a Long automatically). So basically don't use Python2 / Python3 interchangeably on the same manifest if you want to avoid grief.
I create my own logs, are your logs stored in the parent folder of the gogrepo script?
Pages:
1
50
64
65
66
67
68
150
200
250
260
This is my favourite topic
Community
General discussiongogrepo.py - Python script for regularly backing up your purchased GOG collection for full offline enjoyment(3896 posts)(3896 posts)
(3896 posts)