It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
For some reason it's extremely slow to download files, or rather to go through files and find out what to download. I've had the console window open for about 30 minutes now, and it's gone through under 100 files, most of them tiny (various game add-ons). Most of the time nothing appears to be happening, no progress bar or anything, then it goes through a few files (without downloading anything, because there are no updates), then back to doing nothing.

What's up with this, and are others experiencing similar issues?

edit: Now it started to download an actual file, and it's preposterously slow. 400 kb/s, and I have a 500 Mbit line...
Post edited June 15, 2019 by Pangaea666
Sometimes it happens. Try cancelling and restarting it. It could be GOG's CDN glitching out.
avatar
shmerl: Sometimes it happens. Try cancelling and restarting it. It could be GOG's CDN glitching out.
Must be glitching out a lot then. Running the script again now, and it's the same as before. 99% of the time it's preposterously slow, nothing seems to happen. Then it will go through a few files. Maybe with very low download speed, though sometimes I get the "proper" speed.

Is it because I simply run it with --download for easiness?

Wanted to update my offline library again, but it sure takes time, despite not needing to actually download much.

Edit: Something big started to download now, and I barely get 1 MB/s, which is quite frankly pathetic given my line. It often looks like each thread is throttled at max 1 MB/s tbh, though it is often less than that too. Sometimes it will spike up to 2.5-5MB/s, then bam down to 600 kb/s again.

Games that should take minutes to download will take two hours instead.
Post edited July 25, 2019 by Pangaea666
avatar
shmerl: Sometimes it happens. Try cancelling and restarting it. It could be GOG's CDN glitching out.
avatar
Pangaea666: Must be glitching out a lot then. Running the script again now, and it's the same as before. 99% of the time it's preposterously slow, nothing seems to happen. Then it will go through a few files. Maybe with very low download speed, though sometimes I get the "proper" speed.

Is it because I simply run it with --download for easiness?

Wanted to update my offline library again, but it sure takes time, despite not needing to actually download much.

Edit: Something big started to download now, and I barely get 1 MB/s, which is quite frankly pathetic given my line. It often looks like each thread is throttled at max 1 MB/s tbh, though it is often less than that too. Sometimes it will spike up to 2.5-5MB/s, then bam down to 600 kb/s again.

Games that should take minutes to download will take two hours instead.
Have you tried using just a single thread I see a consistent max with just one.
avatar
Sude: LGOGDownloader 1.6
- Added update downloading to update check (untested, it will warn the user and ask what to do before continuing)

http://sites.google.com/site/gogdownloader/lgogdownloader-1.6.tar.gz

lgogdownloader --update-check
Same as before. Shows how many games have been updated.

The new functionality is untested

lgogdownloader --update-check --list
Lists updated files.

lgogdownloader --update-check --download
Downloads updated files.

It needs repeating that this functionality is UNTESTED.
It may or may not work. The code uses boost::filesystem::remove() in 2 places so if something goes really really wrong it could DELETE the old version without downloading the new version.

Also running update check with "--list" or "--download" will remove the updated flag from your account page.
This is caused by how GOG handles the update flag in the account page and can't be avoided.

If you test the new functionality report what happened to this thread.
Does --update-check also download all new, undownloaded files? LIke when I buy something new, can I just buy it, then run this command to grab everything?
avatar
evoblade2000: Does --update-check also download all new, undownloaded files? LIke when I buy something new, can I just buy it, then run this command to grab everything?
The --updated flag (which I assume replaces the no-longer-available --update-check flag if you add --list) relies on gog's own horribly inaccurate update notifications. I doubt these include new games or games with new DLC. It hasn't even given me an update notification for Pathfinder: Kingmaker in ages, making for at least 6 surprise updates so far.

I actually think it would be a good idea to add a feature that tracks changes to the game details cache better. Perhaps also a last download timestamp in a more controlled location to track what's new/changed better. I'm not really motivated to do it myself, though. For now, I use rsync to tell me what's new or changed after --download I keep two separate copies of the results of downloading everything, and use a dry run rsync from one to the other after I do downloads to grab the name of new and changed files and soft link them into my processing directory. This sort of thing is not useful for most people. Maybe at least wrap lgogdownloader --update-cache in a script that diffs the before and after gamedetails would be a temporary workaround, but if I went through that much trouble, I'd just make lgogdownloader spit out a diff after --update-cache in the first place.

rsync -avH /mnt/usb3{f,b}/gog/ --dry-run | grep -v '/changelog.*\.html$' | while read -r x; do test -f /mnt/usb3f/gog/"$x" && ln -s /mnt/usb3f/gog/"$x" ~/games/gog/dl/; done

Speaking of which, I guess I (or someone) should fix the changelog downloading to only update the file if it actually changed, to make it a more useful feature. That's why I just filter out changelogs in the above line.

Edit: just to be clear, I haven't paid attention to lgogdownloader changes in a while, so maybe it already supports what I want, exactly the way I want it. What I do works for me, though, and I have little incentive to change. The only thing that really bothers me is the fact that it now takes way too long to update the cache, but I'm not even motivated enough to look into that.
Post edited July 30, 2019 by darktjm
avatar
darktjm: ...
That was my issue with lgogdownloader when I switched from gogrepo, the lack of feedback on updates etc. For update spotting I have a script that --update-cache then does a --status with output to a file then I commit the file into git and have it show the last changes with git show.
Post edited August 01, 2019 by Cusith
avatar
Cusith: For update spotting I have a script that --update-cache then does a --status with output to a file then I commit the file into git and have it show the last changes with git show.
Good idea, and it doesn't even need two full library copies, just one. I think git is overkill, but I guess there might be benefits. I normally only run --status to ensure that all my files have verified checksums, which they don't if I abort a download. In any case, I'll stick to my old habits.

The only thing missing now is a solution that requires zero local copies of your entire library. I guess placing gamedetails.json under source code control like you do with the --status output would be sufficient Perhaps filter it first to spit out the download path and size for only the files that would actually be downloaded. Needs work. I'm still not motivated enough to do it myself right now, though.
avatar
Cusith: Have you tried using just a single thread I see a consistent max with just one.
Changed the settings and am trying that now, but it's still horrifically slow :-(

Would it help to start over by cleaning out lgogdownloader? I'm just worried the token/crapcha thing will get bugged again.
avatar
Pangaea666: Would it help to start over by cleaning out lgogdownloader? I'm just worried the token/crapcha thing will get bugged again.
For a test, is it also slow if you download from the browser?
avatar
Pangaea666: Would it help to start over by cleaning out lgogdownloader? I'm just worried the token/crapcha thing will get bugged again.
avatar
shmerl: For a test, is it also slow if you download from the browser?
The actual download seemed better with one thread, actually. But the bigger problem is how slow it is to go through files that are already downloaded, to get to files that are actually needed. Don't have any many games as many do, but with 150+ titles it's still a lot of files. And most of them sit in the terminal window with seemingly no progress/anything happening. Then it goes through a few, before another 10-20 second wait. Even for tiny files.

If nobody else have these problems, maybe it's something on my end? Regular downloads are fast, however.

Not entirely unknown I see, also to a certain other person in here ;-)
https://github.com/Sude-/lgogdownloader/issues/117

No solution in there, though, it's still an open ticket. Intriguing to see the person mentioning 400 kB/s, because often it has dropped to that on my end as well, especially when I used 4 threads.
Post edited August 05, 2019 by Pangaea666
neat, with --xml-directory and automatic-xml-creation file validation is going like in seconds :D now need to find out how speed up updating cache :D
Just finding this and it is awesome! Not a Linux guy but the instructions were straight forward and I compiled my first Linux program but ....

when running --list-details it always errors at the same place. Any ideas?

HTTP response code said error 100
HTTP ERROR: 404 (https://www.gog.com/account/gameDetails/1444400283.json)
Error from reader: * Line 1, Column 1
Syntax error: value, object or array expected.
* Line 1, Column 1
Syntax error: value, object or array expected.