It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
Community
General discussiongogrepo.py - Python script for regularly backing up your purchased GOG collection for full offline enjoyment(4247 posts)(4247 posts)
(4247 posts)
Pages:
1
50
100
136
137
138
139
140
200
250
284
This is my favourite topic
Posted May 21, 2019
avatar
timppu: You use the "clean" command to remove old obsolete files from your local GOG files, files that don't exist anymore on the GOG servers (ie. are not in your updated manifest file anymore).

Actually it doesn't remove the obsolete files outright, but moves them into a separate !oprhaned-directory, which you can later check and delete all the files in it, if you want.
Oh I see, thank you for the info.

The README.md files only mentioned a "clean" command in the TODO list, so I just ignored it.
I see that the program's -h parameter lists it correctly, instead.

Btw lol, the clean command reported 581Gb of orphaned data (just the games).. I created a backup recently, so I wasted a bunch of time and space for nothing -_-'

P.s: I'd like to keep the non-Scumvm installers too, is there a list somewhere?
EDIT: Magog can do it, but I don't know which versions are still the DOS ones..
Are custom subfolders safe in the "!oprhaned" dir?
Post edited May 24, 2019 by phaolo
Posted May 24, 2019
I just issued a tiny pull request back to kalanyr's repo [not yet accepted of course], though I guess I should have based it off the dev branch (I didn't know it was there). I added a -errorsonly feature to verify, so it only displays errors, and not skips/goods). I can actually meaningfully use verify now without having to parse a huge log after!

What's in the dev branch that's not in master?
Post edited May 24, 2019 by mqstout
Posted May 25, 2019
avatar
mqstout: I just issued a tiny pull request back to kalanyr's repo [not yet accepted of course], though I guess I should have based it off the dev branch (I didn't know it was there). I added a -errorsonly feature to verify, so it only displays errors, and not skips/goods). I can actually meaningfully use verify now without having to parse a huge log after!

What's in the dev branch that's not in master?
You can get a comparison here:

https://tinyurl.com/yyd7ob89

Sorry, GOG does not like the Github comparison links at all. Had to find a URL shortener.
Post edited May 25, 2019 by Kalanyr
Posted May 27, 2019
For some reason, I had to bootstrap pip today again after a Synology update and reinstall html5lib. Just a warning if someone runs the script on their NAS drive:
disi@disi-syno:/volume1/ISO/GOG$ wget https://bootstrap.pypa.io/get-pip.py
disi@disi-syno:/volume1/ISO/GOG$ sudo python get-pip.py
disi@disi-syno:/volume1/ISO/GOG$ sudo pip install html5lib
Post edited May 27, 2019 by disi
Posted May 29, 2019
not sure if its right place to ask (but have no other choice - due to good old bug, I cant create forum threads again), but does anybody know if gogdb provide correct hash sums? There was at least 3 times when I wasnt sure about integrity of my locally saved data, but had no access to gogrepopy, nor internet connection on anything but smartphone (and since my operator inject ads and other bs into traffic, I would be REALLY glad to avoid logging into my gog account (to download related xmls by myself) this way)
Posted May 31, 2019
also, back to gogrepo.
As far as I know - when something goes wrong with connection, it retries 3 times, then silently abort. Maybe print some unique text if download was aborted due to interruption? So I ll be able to grep this catchphrase from tee's output file and force the very same script to restart from where it has failed (instead of spamming gog with complete list of scripts few times per each day)
Posted June 06, 2019
I haven't run gogrepo for a couple of months, bought some games in the meantime and I know oodles of DOSBox games were also updated in the meantime (cloud save integration awhile back)...

...so yeah, yesterday I started downloading, about 340GB to download. Yikes! I guess it takes several days over my measly 10Gbps cable modem line. Oh well, I just let it run...

This just reminds me again that it would be nice to see the estimated remaining time when you are downloading a lot, not really knowing (without calculating) whether it will take a day or a week. Last summer I was inspecting how to add such functionality to gogrepo myself but got sidetracked. It _shouldn't_ be too hard because gogrepo already calculates the total amount of data to download all the time, I just couldn't easily figure out how to calculate the combined download speed.

After that it should be pretty straightforward, the net is full of good algorithms how to calculate the remaining download time (taking into account the fluctuating download speed, of course).
Post edited June 06, 2019 by timppu
Posted June 16, 2019
this thread feels kinda abandoned. I wonder why

I need to get a list of all IDs, I currently have. At the moment I use such pile of code to perform so:
python ./gogrepo.py download -os windows linux -lang en -skipgalaxy -skipstandalone -skipshared -skipextras
However, there is a problem - it create folders with '!info.txt' inside for each ID. Which kinda messes up my local library, coz I lose track of what I already have locally and what I didnt download yet. Thus - is there a better way to check for IDs?
Posted June 16, 2019
avatar
Gekko_Dekko: Thus - is there a better way to check for IDs?
With 'IDs', do you mean the lower case titles that are used as directory names?

If so, how about this (provided you have grep, sed and sort installed): grep "'title'" gog-manifest.dat | sed -e "s/\s*'title':\s*'\([^']*\).*/\1/" | sort
Posted June 16, 2019
avatar
Gekko_Dekko: Thus - is there a better way to check for IDs?
avatar
brouer: With 'IDs', do you mean the lower case titles that are used as directory names?

If so, how about this (provided you have grep, sed and sort installed): grep "'title'" gog-manifest.dat | sed -e "s/\s*'title':\s*'\([^']*\).*/\1/" | sort
THANKS SO MUCH. I never understood sed's magic to make something like that by my own, so this will come extremely handy!
Posted June 17, 2019
can anybody suggest me a good way to check if there are updates I've downloaded but not installed? I thought about tracking it via checking the contains of orphaned folder. But maybe there is a better solution?
Post edited June 17, 2019 by Gekko_Dekko
Posted June 17, 2019
avatar
brouer: find...
avatar
Gekko_Dekko:
I use this as my find: It only searches TITLE.

grep -E "^.*u'title'.*$1*" ./gog-manifest.dat

user@machine ~/gog/gogrepo4 $ ./gog-find brother
u'title': u'my_brother_rabbit'},
u'title': u'pilot_brothers'},
Post edited June 17, 2019 by mqstout
Posted June 25, 2019
Gosh.. the last stable release by Kalanyr is 1 year old, and dev is almost there.
Woolymethodman is back updating his original version, instead, but there hasn't been any kind of merge between the 2 gogrepos since 2017.
Has anyone tried it recently? What are the main differences?
Post edited June 25, 2019 by phaolo
Posted June 26, 2019
avatar
phaolo: Gosh.. the last stable release by Kalanyr is 1 year old, and dev is almost there.
Woolymethodman is back updating his original version, instead, but there hasn't been any kind of merge between the 2 gogrepos since 2017.
Has anyone tried it recently? What are the main differences?
Kalanyr was in the middle of rewriting a lot of gogrepo gamma which is why there have been no further updates other than to ensure it's still working with the site. Not sure if he's still working on it or when it will be released.

I think Kalanyr's version has a few extra options/QoL improvements that the original doesn't, but other than that the two are more or less the same.
Post edited June 26, 2019 by erephine
Posted June 26, 2019
avatar
phaolo: Gosh.. the last stable release by Kalanyr is 1 year old, and dev is almost there.
Woolymethodman is back updating his original version, instead, but there hasn't been any kind of merge between the 2 gogrepos since 2017.
Has anyone tried it recently? What are the main differences?
avatar
erephine: Kalanyr was in the middle of rewriting a lot of gogrepo gamma which is why there have been no further updates other than to ensure it's still working with the site. Not sure if he's still working on it or when it will be released.

I think Kalanyr's version has a few extra options/QoL improvements that the original doesn't, but other than that the two are more or less the same.
Gamma? Gogrepo dev hasn't been updated since about 1 year too.
Kalanyr said he was going to return, but it seems he has been too busy.
The script luckly still works, but the stable version misses some useful parameters like -skipfiles.
I'm not sure if the original even has all the improvements Kalanyr added and maybe the new changes are just minor bugfixes.
Post edited June 26, 2019 by phaolo
Pages:
1
50
100
136
137
138
139
140
200
250
284
This is my favourite topic
Community
General discussiongogrepo.py - Python script for regularly backing up your purchased GOG collection for full offline enjoyment(4247 posts)(4247 posts)
(4247 posts)