It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
Community
General discussiongogrepo.py - Python script for regularly backing up your purchased GOG collection for full offline enjoyment(4237 posts)(4237 posts)
(4237 posts)
Pages:
1
50
100
105
106
107
108
109
200
250
283
This is my favourite topic
Posted July 26, 2018
avatar
timppu: [...]I'd maybe like it more to be in some form of blacklist file or something, where you can list the filenames that you want to be skipped, be it wildcards, regular expressions or even exact filenames. The blacklist entries could be also located in the separate config file that I recall kalanyr is going to implement, in its own section?
I wanted it to match the existing command line stuff for now, but I imagine having the default set from the config (once there is a config) would make sense.

avatar
timppu: [...]maybe the files should be included in the manifest file (during update), but they would be skipped when running download or verify.
That's what I did, actually (except I need to add to verify).

avatar
timppu: ALSO, I would very much like that it would keep a log somewhere which files exactly it is skipping, just so that you can detect if you are accidentally blacklisting files that you didn't mean to blacklist, when using either regular expressions and/or wildcards.
I did log what was skipped and why (see above).

avatar
Kalanyr: I hadn't even decided on whether to use fnmatch or regexp or something else.
I went with fnmatch since it's more human readable, but it should be possible to support both (fnmatch has code to translate to regexps)

I'll look into (also) supporting regexs, and adding it to verify.

UPDATE: Posted a pull request with verify added (didn't do the regexs because merging with the fnmatch stuff made it harder to log WHAT matched).
Post edited July 27, 2018 by jczorkmid
Posted July 27, 2018
avatar
timppu: [...]I'd maybe like it more to be in some form of blacklist file or something, where you can list the filenames that you want to be skipped, be it wildcards, regular expressions or even exact filenames. The blacklist entries could be also located in the separate config file that I recall kalanyr is going to implement, in its own section?
avatar
jczorkmid: I wanted it to match the existing command line stuff for now, but I imagine having the default set from the config (once there is a config) would make sense.

avatar
timppu: [...]maybe the files should be included in the manifest file (during update), but they would be skipped when running download or verify.
avatar
jczorkmid: That's what I did, actually (except I need to add to verify).

avatar
timppu: ALSO, I would very much like that it would keep a log somewhere which files exactly it is skipping, just so that you can detect if you are accidentally blacklisting files that you didn't mean to blacklist, when using either regular expressions and/or wildcards.
avatar
jczorkmid: I did log what was skipped and why (see above).

avatar
Kalanyr: I hadn't even decided on whether to use fnmatch or regexp or something else.
avatar
jczorkmid: I went with fnmatch since it's more human readable, but it should be possible to support both (fnmatch has code to translate to regexps)

I'll look into (also) supporting regexs, and adding it to verify.

UPDATE: Posted a pull request with verify added (didn't do the regexs because merging with the fnmatch stuff made it harder to log WHAT matched).
Thanks. I've merged it in (to the dev branch),
Posted July 28, 2018
Anyone else seeing a ton of gateway timeouts today?

09:21:03 | request failed: HTTP Error 504: Gateway Timeout (3 retries left) -- will retry in 5s...
09:21:09 | request failed: HTTP Error 504: Gateway Timeout (2 retries left) -- will retry in 5s...

trying to do a full update and my second pass updates are going to be a serious PITA to get all the stuff :/
Post edited July 28, 2018 by Starkrun
Posted July 28, 2018
forgot if I've asked already - can hashes be saved as files, to manually check file integrity later? For example - my current archive feature a plaintext file with md5sum output inside (to quickly do 'md5sum -c myfilewithhashes.md5', when necessary)
Posted July 28, 2018
avatar
Starkrun: Anyone else seeing a ton of gateway timeouts today?

09:21:03 | request failed: HTTP Error 504: Gateway Timeout (3 retries left) -- will retry in 5s...
09:21:09 | request failed: HTTP Error 504: Gateway Timeout (2 retries left) -- will retry in 5s...

trying to do a full update and my second pass updates are going to be a serious PITA to get all the stuff :/
I just did a big update last night (143GB) and didn't get any timeouts. Maybe I was just lucky !!!
Posted July 28, 2018
avatar
Gekko_Dekko: forgot if I've asked already - can hashes be saved as files, to manually check file integrity later? For example - my current archive feature a plaintext file with md5sum output inside (to quickly do 'md5sum -c myfilewithhashes.md5', when necessary)
The md5 hashes (where available) for whole files are saved in the manifest. The verify option can be limited to a set of games using -ids/-skipids and uses those md5s as well as a file size check and a zip integrity check. Given the skipfiles jczorkmid added it should be pretty easy to get it to scan only the desired file. I might extend it at some stage to scan only files matching the specified pattern which would make it trivial.

I could export them to files name filename.md5 but that's probably unnecessary clutter for most people so it would have to be optional.
Post edited July 28, 2018 by Kalanyr
Posted July 29, 2018
I'm just chiming in to say that I would prefer the settings to be in a separate file.
Posted July 30, 2018
Hey there, guys. I'm new to the gogrepo script. Any way to resolve the error in the attached screenshot?
Attachments:
screenshot_from_.png (96 Kb)
Posted July 30, 2018
avatar
RawSteelUT: Hey there, guys. I'm new to the gogrepo script. Any way to resolve the error in the attached screenshot?
Wooly doesn't seem to be maintaining the original any more. Try my fork located here: https://github.com/Kalanyr/gogrepo
and let me know if you stil have this problem.
Posted July 30, 2018
avatar
Gekko_Dekko: forgot if I've asked already - can hashes be saved as files, to manually check file integrity later? For example - my current archive feature a plaintext file with md5sum output inside (to quickly do 'md5sum -c myfilewithhashes.md5', when necessary)
avatar
Kalanyr: The md5 hashes (where available) for whole files are saved in the manifest. The verify option can be limited to a set of games using -ids/-skipids and uses those md5s as well as a file size check and a zip integrity check. Given the skipfiles jczorkmid added it should be pretty easy to get it to scan only the desired file. I might extend it at some stage to scan only files matching the specified pattern which would make it trivial.

I could export them to files name filename.md5 but that's probably unnecessary clutter for most people so it would have to be optional.
its fine, thats exactly what I wanted to know.

How many forks are alive at the moment and what s the differences, btw?
Posted July 31, 2018
avatar
Gekko_Dekko: How many forks are alive at the moment
Kalanyr's fork is the main one now.
Here is a list of all the available ones:
https://github.com/eddie3/gogrepo/network/members

I can't find the historical graph anymore, however..
Posted August 01, 2018
avatar
phaolo: I can't find the historical graph anymore, however..
This, you mean?
Posted August 01, 2018
avatar
phaolo: I can't find the historical graph anymore, however..
avatar
Lashiec: This, you mean?
Ah lol.. I didn't remember it was in "Network".
Posted August 01, 2018
The graph is probably more useful than the forks list if you're looking for active forks doing their own work, versus ones created for submitting PRs back to master (or another fork).
Posted August 02, 2018
Ugh. That feeling when you get back from a month away and find yourself with 343 GB to download.
Pages:
1
50
100
105
106
107
108
109
200
250
283
This is my favourite topic
Community
General discussiongogrepo.py - Python script for regularly backing up your purchased GOG collection for full offline enjoyment(4237 posts)(4237 posts)
(4237 posts)