It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
I have a problem, and I don't quite understand what happens.
lgogdownloader rushes through the getting game names and getting game info part in the beginning and then downloads only a fraction of the games (about 20 percent) ignoring any new games I bought. They can be downloaded when I specify the name of the game but won't get downloaded when saying game=all

I tried deleting my config file and going with the default options (thinking it might have been a prob there)
I tried deleting the .cache/lgogdownloader directory
I tried toggling the options use-cache and no-remote-xml and automatic-xml-creation
Interestingly when I use ./lgogdownloader --update-cache it takes the apropriate amount of time when going through the game names but afterwards when using --download --game=all it rushes through again ignoring most of the games :/

The version I use is 3.4.8683fe0

EDIT:
heh, I tried to removing the "--game=all" part now and astonishingly it starts going through all games and not only a few now.
seems like "all" is not a keyword anymore but now a regex for any game having "all" in their name or something?!
Post edited December 06, 2018 by mchack
avatar
mchack: lgogdownloader rushes through the getting game names and getting game info part in the beginning and then downloads only a fraction of the games (about 20 percent) ignoring any new games I bought
Specifying the platform can affect it.
avatar
mchack: lgogdownloader rushes through the getting game names and getting game info part in the beginning and then downloads only a fraction of the games (about 20 percent) ignoring any new games I bought
avatar
shmerl: Specifying the platform can affect it.
thanks but the problem was solved with removing the --game=all parameter. It has been a while since I last downloaded my games but I'm pretty sure back then this worked as a keyword for all of the games. Now it actually reduces the games downloaded or checked to only those games that have the letters "all" in their names ;)

the --help tag says as much but I'm pretty sure it worked differently in the past
I'm still getting HTTP 404 errors for the XML file downloads. And the errors are getting more. For a while it was good enough to run lgogdownloader multiple times which eventually ended in a run without errors. Meanwhile even that does not work anymore. I already had set the "retries" parameter to 10 and have increased it to 100 now, but I still get these errors.

Does the "retries" parameter even have an effect in case of 404 errors? Is there any way to make lgogdownloader retry the downloads for a longer time?
Post edited December 20, 2018 by eiii
df628b2 Add option to include hidden products
Adds option --include-hidden-products that allows user to list and download products that have been set hidden in account page.


avatar
eiii: Does lgogdownloader have any support for downloading movies and their extras?
There isn't support for listing or mass downloading movies but --download-file can be used to download movies by their id
For example:
lgogdownloader --download-file tpb_afk_the_pirate_bay_away_from_keyboard/en1video1

avatar
diziet_sma: Something just occurred to me, I use the automatic xml creation option for extras so that I can check the files are not corrupted. But how would I know if the online file, say, some_games_artwork.zip, hasn't been updated? If gog.com don't put a version number on it then there's nothing to compare to tell. Anything with a server side checksum is different of course.

Some of my extras files are years old, I'd be very surprised if none of them had been updated in that time.
This is a limitation caused by GOG not providing checksum data for extras.
Automatic xml creation was a workaround just to have --repair support for extras but it isn't very useful for anything other than making sure your local files haven't been corrupted.
I've actually been thinking about removing automatic xml creation for extras because it can and probably will cause problems in the future.

avatar
mchack: heh, I tried to removing the "--game=all" part now and astonishingly it starts going through all games and not only a few now.
seems like "all" is not a keyword anymore but now a regex for any game having "all" in their name or something?!
Aliases ("all" and "free") for --game option were removed in version 3.4 because they had become useless.
"free" alias could no longer be used as originally intended and "all" was just unnecessary because leaving regex empty has the same effect.

avatar
eiii: I'm still getting HTTP 404 errors for the XML file downloads. And the errors are getting more. For a while it was good enough to run lgogdownloader multiple times which eventually ended in a run without errors. Meanwhile even that does not work anymore. I already had set the "retries" parameter to 10 and have increased it to 100 now, but I still get these errors.

Does the "retries" parameter even have an effect in case of 404 errors? Is there any way to make lgogdownloader retry the downloads for a longer time?
It uses Downloader::getResponse to get the XML data so it should retry until max retry count is reached.
The condition for retry is only that return value for curl_easy_perform() is not CURLE_OK, response is empty and max retry count has not been reached.

do {...} while ((result != CURLE_OK) && response.empty() && (this->retries++ < Globals::globalConfig.iRetries));
It outputs the http response code after max retry count has been reached so errors don't get spammed to console all the time
https://github.com/Sude-/lgogdownloader/blob/df628b2a5922830d72128a1434019e7a46c52e93/src/downloader.cpp#L1534-L1575

The only option to make lgogdownloader retry for longer time is increasing retry count.
Well technically there's --wait but it only delays the requests. It could help if the issue is caused by temporary server load issue which passes quickly but the drawback is that everything that uses Downloader::getResponse function gets the same delay to every request.
avatar
Sude: There isn't support for listing or mass downloading movies but --download-file can be used to download movies by their id
For example:
lgogdownloader --download-file tpb_afk_the_pirate_bay_away_from_keyboard/en1video1
I only own the free movies so that's good enough for me. Thanks!

avatar
Sude: Well technically there's --wait but it only delays the requests. It could help if the issue is caused by temporary server load issue which passes quickly but the drawback is that everything that uses Downloader::getResponse function gets the same delay to every request.
I'll try if that delay helps.

Edit: I've tried it. But the result is somewhat unexpected or my understanding is wrong. I've set retries to 1000 and wait to 1000. These settings delay the cache update by a few minutes as expected. But they do not delay the status check (lgogdownloader --status). I only got 4 HTTP errors during the status check, but shouldn't every HTTP error delay the status check by 1000 seconds with these settings (1000 retries x 1000 ms)? Shouldn't it run at least an hour with these 4 errors?
Post edited December 21, 2018 by eiii
avatar
eiii: Edit: I've tried it. But the result is somewhat unexpected or my understanding is wrong. I've set retries to 1000 and wait to 1000. These settings delay the cache update by a few minutes as expected. But they do not delay the status check (lgogdownloader --status). I only got 4 HTTP errors during the status check, but shouldn't every HTTP error delay the status check by 1000 seconds with these settings (1000 retries x 1000 ms)? Shouldn't it run at least an hour with these 4 errors?
Apparently Downloader::getRemoteFileHash still uses the old API for getting xml data.
I'll have to update it to use Galaxy API later.

But even if it was using Galaxy API it wouldn't have tried to retry because apparently I never added retry and wait support to getResponse functions used by either API implementation.
9bd416e adds retry and wait support to API::getResponse and galaxyAPI::getResponse
avatar
Sude: 9bd416e adds retry and wait support to API::getResponse and galaxyAPI::getResponse
Thank you for the update! lgogdownloader behaves as expected now. But GOG's download errors are quite persistent, I get the same 4 download errors as yesterday plus a bunch of new ones today.

avatar
Sude: Apparently Downloader::getRemoteFileHash still uses the old API for getting xml data.
Knowing that the CRC file downloads use the old API explains why I "only" get download errors for these files. Apparently GOG does not care much about the non-Galaxy API anymore.
I'm again hitting a weird problem of bandwidth throttling when downloading with lgogdownloader. In the browser I get full bandwidth (gigabit connection). With lgogdownloader i get 1/20th of it. Spoofing the user agent doesn't help either. Did anyone encounter such problem? Could it be some gnutls bug?
Post edited December 23, 2018 by shmerl
avatar
shmerl: I'm again hitting a weird problem of bandwidth throttling when downloading with lgogdownloader. In the browser I get full bandwidth (gigabit connection). With lgogdownloader i get 1/20th of it. Spoofing the user agent doesn't help either. Did anyone encounter such problem? Could it be some gnutls bug?
Sorry I am not actively using lgogdownloader right now (yet), but was it so that lgogdownloader uses the Galaxy API to download game installers?

I recall reading that if you download your offline installers with Galaxy, they come from a separate set of servers than when you use your web browser. So... could it simply be that for you (depending where you are located), the "web servers" just happen to offer better bandwidth at the moment, than the servers which Galaxy use?
avatar
timppu: but was it so that lgogdownloader uses the Galaxy API to download game installers?
I thought that only happens when you use --galaxy related keys? Or does it use Galaxy now for all downloads?
Post edited December 24, 2018 by shmerl
avatar
shmerl: I thought that only happens when you use --galaxy related keys? Or dies t use Galaxy now for all downloads?
Ok I have no idea, didn't know it gives an option. (I've used lgogdownloader once a couple of months ago...)
Then again, can you try it, in order to see whether you get better download speeds that way?

Yeah I know I am just shooting in the dark and not really helping analyze why it is slow for you...
No matter what I do I can't seem to get this thing to work.

Apart from the below I've tried to copy cookies.txt from both firefox and chromium after succesfully signing via the browser and I've disabled two-step authentication.


$gogdownloader --list
Email: *@gmail.com
Password:
HTTP: Login successful
Galaxy: Login failed
Email: ^C
$ lgogdownloader --login-api
Email: *@gmail.com
Password:
API: Login successful
Email: *@gmail.com
Password:
Couldn't resolve host name
HTTP: Login successful
Galaxy: Login failed
avatar
JeevesWooster: No matter what I do I can't seem to get this thing to work.

Apart from the below I've tried to copy cookies.txt from both firefox and chromium after succesfully signing via the browser and I've disabled two-step authentication.
Exporting cookies from browser doesn't work.
Website login and Galaxy API login are tied together and if one fails then the other fails automatically which is what you are experiencing.

I've been playing with QtWebEngine to help people that are getting recaptcha on login form.
I finally have something that's usable.

Anyone experiencing login issues because of recaptcha should test if this patch helps.
Let me know if you encounter any issues with it.

Download patch and apply to current git version
wget https://sites.google.com/site/gogdownloader/qt_gui_login.diff
git apply qt_gui_login.diff

Make sure you enable the new Qt GUI with CMake option -DUSE_QT_GUI=ON
mkdir build
cd build
cmake -DUSE_QT_GUI=ON ..
make

Requires qt5 webengine development files (qtwebengine5-dev)

If lgogdownloader encounters recaptcha on login form it will open a window with embedded chromium browser.
The window should automatically close when it has gathered the necessary details and the downloader should continue login process with valid cookies for website and authorization code for Galaxy API.
Post edited February 26, 2019 by Sude
From now on lgogdownloader should be able to handle recaptcha on login form if compiled with -DUSE_QT_GUI=ON
Enabling the new Qt GUI adds dependency on qt5 webengine (debian: qtwebengine5-dev, arch: qt5-webengine)

1e8ebbf Add GUI login
Use QtWebEngine for getting cookies and authorization code if login form contains recaptcha and downloader is compiled with -DUSE_QT_GUI=ON

a8d9d46 Make progress info show estimated total remaining size and time