Posted February 22, 2022
Tarhiel
Unn Von Taffer
Tarhiel Sorry, data for given user is currently unavailable. Please, try again later. View profile View wishlist Start conversation Invite to friends Invite to friends Accept invitation Accept invitation Pending invitation... Unblock chat Registered: Feb 2010
From Germany
Magnitus
Born Idealist
Magnitus Sorry, data for given user is currently unavailable. Please, try again later. View profile View wishlist Start conversation Invite to friends Invite to friends Accept invitation Accept invitation Pending invitation... Unblock chat Registered: Mar 2011
From Canada
Posted February 22, 2022
mrkgnao: Given that MaGog collected hundreds of pieces of information on every single one of the thousands of games in GOG's catalogue every six hours for more than six years with rarely a noticeable effect, I consider the stress created by a full update utterly negligible.
MaGoG was a single entity. Heck, even my 10000-30000 requests, if taken by themselves are a drop in the ocean. Now, empower hundreds, maybe thousands, of users to do the same thing (and they're not all doing it nice and sequential like Magog was over the course of a day, its more random than that with a bunch of live users) and its another story.
This is really a case of "What if everyone was doing as I do?".
I'm pretty sure that between all 3 unofficial clients, we're talking about hundreds of users at this point. If those clients were more mainstream (ie, really well polished extremely easy to setup and use with a gui), we'd be talking thousands.
Post edited February 22, 2022 by Magnitus
phaolo
Durik - Half-Orc
phaolo Sorry, data for given user is currently unavailable. Please, try again later. View profile View wishlist Start conversation Invite to friends Invite to friends Accept invitation Accept invitation Pending invitation... Unblock chat Registered: Dec 2013
From Italy
Posted February 22, 2022
I doubt blotunga meant just updating the manifest but not downloading the files.. what would that accomplish? O_o
Expecially for new big games that get frequent changes (if you don't have them, I'm speaking in general).
mrkgnao: I do a regular update (i.e. flagged games) whenever I notice a GOG flag, which could easily be several times a day. I do a full update about once or twice a month.
Wouldn't that just download way more files in those months (4-6 for me), instead of just doing 1 full update? Expecially for new big games that get frequent changes (if you don't have them, I'm speaking in general).
mrkgnao
https://www.amazon.com/dp/B00HCZVCTO
mrkgnao Sorry, data for given user is currently unavailable. Please, try again later. View profile View wishlist Start conversation Invite to friends Invite to friends Accept invitation Accept invitation Pending invitation... Unblock chat Registered: Apr 2009
From United States
Posted February 22, 2022
phaolo: I doubt blotunga meant just updating the manifest but not downloading the files.. what would that accomplish? O_o
The purpose of that is to find all those installer updates that GOG did not flag (and there are a lot of those). You do a full update, then download those unflagged new installers, not your entire library. It's not that different from a regular update, except that it takes much longer and finds unflagged games. But it's essential to do that regularly, otherwise your library is not up to date. mrkgnao: I do a regular update (i.e. flagged games) whenever I notice a GOG flag, which could easily be several times a day. I do a full update about once or twice a month.
phaolo: Wouldn't that just download way more files in those months (4-6 for me), instead of just doing 1 full update? Expecially for new big games that get frequent changes (if you don't have them, I'm speaking in general).
Post edited February 22, 2022 by mrkgnao
timppu
Favorite race: Formula__One
timppu Sorry, data for given user is currently unavailable. Please, try again later. View profile View wishlist Start conversation Invite to friends Invite to friends Accept invitation Accept invitation Pending invitation... Unblock chat Registered: Jun 2011
From Finland
B1tF1ghter
Execute order $rep_string
B1tF1ghter Sorry, data for given user is currently unavailable. Please, try again later. View profile View wishlist Start conversation Invite to friends Invite to friends Accept invitation Accept invitation Pending invitation... Unblock chat Registered: Jun 2015
From Other
Posted February 22, 2022
low rated
I haven't followed this thread in a while.
I have some questions.
Please excuse my rudeness but I don't have time to read the few dozen pages since the last time I checked the thread so I will just stright out ask.
1. What is the current most actively worked on / updated fork ?
2. Does it use GOG API for the *downloads* already ?
3. Does it support download pause / resume (within same chunk / file) ?
4. Would it survive IP change mid download ?
5. Does it allow getting checksums / downloads of non-latest builds ?
I would appreciate the response.
Feel free to make it in-depth.
I have some questions.
Please excuse my rudeness but I don't have time to read the few dozen pages since the last time I checked the thread so I will just stright out ask.
1. What is the current most actively worked on / updated fork ?
2. Does it use GOG API for the *downloads* already ?
3. Does it support download pause / resume (within same chunk / file) ?
4. Would it survive IP change mid download ?
5. Does it allow getting checksums / downloads of non-latest builds ?
I would appreciate the response.
Feel free to make it in-depth.
blotunga
GrumpyOldGamers.CyringOutMiserably
blotunga Sorry, data for given user is currently unavailable. Please, try again later. View profile View wishlist Start conversation Invite to friends Invite to friends Accept invitation Accept invitation Pending invitation... Unblock chat Registered: Apr 2012
From Other
Posted February 22, 2022
Basically this. Not every update is flagged, if you accidentally click in your library to an updated/new game, gogrepo update will miss it etc. I prefer to play it safe.
Nice idea with the filters, I only identified a handful (too lazy), but could you paste somewhere your full list, I think I might add some more :)
I'm using these at the moment: "patch_*.exe" "patch_*.bin" "patch_*sh" "*_(32bit)_*.exe" "*_(32bit)_*.bin"
Nice idea with the filters, I only identified a handful (too lazy), but could you paste somewhere your full list, I think I might add some more :)
I'm using these at the moment: "patch_*.exe" "patch_*.bin" "patch_*sh" "*_(32bit)_*.exe" "*_(32bit)_*.bin"
Post edited February 22, 2022 by blotunga
Magnitus
Born Idealist
Magnitus Sorry, data for given user is currently unavailable. Please, try again later. View profile View wishlist Start conversation Invite to friends Invite to friends Accept invitation Accept invitation Pending invitation... Unblock chat Registered: Mar 2011
From Canada
Posted February 22, 2022
timppu: Then again, if you wait longer before the next update, you might many times skip downloading versions that appeared in between. So on the average you will be downloading less from GOG servers, sometimes much less, like when I skipped a couple of Cyberpunk 2077 updates before I re-downloaded the game, I avoided downloading the whole game again a couple of times.
Fair enough.phaolo
Durik - Half-Orc
phaolo Sorry, data for given user is currently unavailable. Please, try again later. View profile View wishlist Start conversation Invite to friends Invite to friends Accept invitation Accept invitation Pending invitation... Unblock chat Registered: Dec 2013
From Italy
Posted February 22, 2022
blotunga: Basically this. Not every update is flagged, if you accidentally click in your library to an updated/new game, gogrepo update will miss it etc. I prefer to play it safe.
That's why I don't even bother with the non-full update ¯\_(O_o)_/¯ For installers I use:
(I've manually downloaded the few required found by the community)
-skipfiles patch_* setup_*_32_* setup_*_(32bit)*
For extras the list is long tho (I wish it allowed regex):
-skipfiles *.dmg *.sh *.gz *_german.zip *_german_2* *_german_v* *_de.zip *_french.zip *_fr.zip *_spanish.zip *_es.zip *_italian.zip *_ita.zip *_it.zip *_russian.zip *_russian_2* *_russian_v* *_ru.zip *_polish.zip *_pl.zip *_zh.zip *_handbuch.zip *_manuel.zip *_avatar*.zip *_trailers.zip *_sample.zip *_demo.zip *_wav.zip *_wma.zip blitzkrieg_v2*.zip driftmoon_vanilla* setup_outcast_* rott_2013_level* setup_strike_suit* bonus_pack_*_.zip *_redkit.zip all_goodies.zip
I skip some of these because they're old\alt versions, or they're mostly duplicates, or they're way too big (rott editor), etc.
Also, some of the similar entries are needed due to Gog's.. variable naming convention.
Post edited February 22, 2022 by phaolo
Andtha
New User
Andtha Sorry, data for given user is currently unavailable. Please, try again later. View profile View wishlist Start conversation Invite to friends Invite to friends Accept invitation Accept invitation Pending invitation... Unblock chat Registered: Feb 2013
From Austria
Posted February 22, 2022
I'm new to the party. I just downloaded gogrepo.py from https://github.com/eddie3/gogrepo and ran it on my Linux machine. 1) login was successful (to my surprise, I was afraid of the reCAPCHA thing). 2) downloading the game details only gets me "xml parsing error trying to get md5 data for ..."
I looked in this discussion for fixes, but I'm not sure what to change in the script. Even though it says it's not for Python 3 yet (README.md from the github repo), it actually runs with python 3.9.9 on my machine. Is the md5 error related somehow?
I cannot find a line with
response.url.replace('?', '.xml?')
as mentioned in this thread (somewhere), only
page.geturl().replace('?', '.xml?')
so I don't know how to fix this.
Help appreciated.
I looked in this discussion for fixes, but I'm not sure what to change in the script. Even though it says it's not for Python 3 yet (README.md from the github repo), it actually runs with python 3.9.9 on my machine. Is the md5 error related somehow?
I cannot find a line with
response.url.replace('?', '.xml?')
as mentioned in this thread (somewhere), only
page.geturl().replace('?', '.xml?')
so I don't know how to fix this.
Help appreciated.
Post edited February 22, 2022 by Andtha
timppu
Favorite race: Formula__One
timppu Sorry, data for given user is currently unavailable. Please, try again later. View profile View wishlist Start conversation Invite to friends Invite to friends Accept invitation Accept invitation Pending invitation... Unblock chat Registered: Jun 2011
From Finland
Posted February 22, 2022
Since no one has replied yet, I say what I can:
2. No idea.
3. You can exit the script (while it downloads) anytime you want with Ctrl-C, and then continue by running the same gogrepo download command again. It will resume the downloads from where you left (I think also partial downloads).
Just remember that if you wait so long (before resuming your downloads) that some of the files have been updated/replaced on the GOG servers, then you have to run also "gogrepo update" again, to get the details for the changed files.
4. Yes, if you "pause" and "resume" the download as mentioned in the previous point.
5. I don't think so.
B1tF1ghter: 1. What is the current most actively worked on / updated fork ?
2. Does it use GOG API for the *downloads* already ?
3. Does it support download pause / resume (within same chunk / file) ?
4. Would it survive IP change mid download ?
5. Does it allow getting checksums / downloads of non-latest builds ?
1. Kalanyr's fork is at least the one most talked about in this thread. No idea if there is some other "unknown" fork that is currently more actively updated. 2. Does it use GOG API for the *downloads* already ?
3. Does it support download pause / resume (within same chunk / file) ?
4. Would it survive IP change mid download ?
5. Does it allow getting checksums / downloads of non-latest builds ?
2. No idea.
3. You can exit the script (while it downloads) anytime you want with Ctrl-C, and then continue by running the same gogrepo download command again. It will resume the downloads from where you left (I think also partial downloads).
Just remember that if you wait so long (before resuming your downloads) that some of the files have been updated/replaced on the GOG servers, then you have to run also "gogrepo update" again, to get the details for the changed files.
4. Yes, if you "pause" and "resume" the download as mentioned in the previous point.
5. I don't think so.
PraetorianWolfie
ScummVM Developer
PraetorianWolfie Sorry, data for given user is currently unavailable. Please, try again later. View profile View wishlist Start conversation Invite to friends Invite to friends Accept invitation Accept invitation Pending invitation... Unblock chat Registered: Jul 2009
From Greece
Posted February 22, 2022
Andtha: I'm new to the party. I just downloaded gogrepo.py from https://github.com/eddie3/gogrepo and ran it on my Linux machine. 1) login was successful (to my surprise, I was afraid of the reCAPCHA thing). 2) downloading the game details only gets me "xml parsing error trying to get md5 data for ..."
I looked in this discussion for fixes, but I'm not sure what to change in the script. Even though it says it's not for Python 3 yet (README.md from the github repo), it actually runs with python 3.9.9 on my machine. Is the md5 error related somehow?
I cannot find a line with
response.url.replace('?', '.xml?')
as mentioned in this thread (somewhere), only
page.geturl().replace('?', '.xml?')
so I don't know how to fix this.
Help appreciated.
That particular fix is indeed similar for the line that you identified: I looked in this discussion for fixes, but I'm not sure what to change in the script. Even though it says it's not for Python 3 yet (README.md from the github repo), it actually runs with python 3.9.9 on my machine. Is the md5 error related somehow?
I cannot find a line with
response.url.replace('?', '.xml?')
as mentioned in this thread (somewhere), only
page.geturl().replace('?', '.xml?')
so I don't know how to fix this.
Help appreciated.
https://github.com/eddie3/gogrepo/blob/2cd3ac4a77d3e5104662fcfb4d96e7129a758bfe/gogrepo.py#L360
You change it to:
tmp_md5_url = page.geturl() + '.xml'
(Keep the indentation of the line identical).
It won't resolve all of the new issues (bad xml responses, gzipped responses) but it will resolve trying to download with a bad url.
Andtha
New User
Andtha Sorry, data for given user is currently unavailable. Please, try again later. View profile View wishlist Start conversation Invite to friends Invite to friends Accept invitation Accept invitation Pending invitation... Unblock chat Registered: Feb 2013
From Austria
B1tF1ghter
Execute order $rep_string
B1tF1ghter Sorry, data for given user is currently unavailable. Please, try again later. View profile View wishlist Start conversation Invite to friends Invite to friends Accept invitation Accept invitation Pending invitation... Unblock chat Registered: Jun 2015
From Other
Posted February 22, 2022
low rated
I was about to ask why I got downvoted to hell for asking legit questions.
I really don't get it.
My questions are totally legit.
I dare to say, even, crucial.
Thank you for your answer.
https://github.com/eddie3/gogrepo/network/members
The problem is 99% of them are under-documented, and they don't state what they changed, also some of them are incorrectly * merging pull requests out of different projects.
There's no changelog for 99% of these forks either.
* You shouldn't exactly be doing that. I mean. You "CAN". But you shouldn't. It creates confusion and can lead to unexpected issues.
I tested gogrepopy in july/august 2021 for the first time.
I used vanilla eddie3 build.
According to github it's not been commited to for 3 YEARS.
Suprisingly, there are open pull requests dated even in 2021 (that are... getting merged in random forks in random order...).
There's the Kalanyr's fork.
No "changed from eddie3 faq" that I could find.
Then there's this:
https://github.com/Magnitus-/gogrepoc
And then there's this clusterf**k:
https://github.com/eddie3/gogrepo/network
( GO FIGURE )
One would think this question alone would make my post worthy of a response instead of being utterly ignored but I guess I am expecting too much from this forum.
The fact that 99% (100% ? I don't have time to check) of these forks directly copy paste the FAQ from the master build doesn't help.
My questions 2, 3, and 4 came because I had an issue before and had the same-ish issue now when I tried to download things from GOG.
Since I am an IT versed in few related areas I debugged this a bit and quickly found out that GOG's server backend is configured in a totally backwards ancient way that resembles *decisions* from before 2000.
TLDR:
403 fest on *ANY* issue. A N Y issue.
As I said, I used the vanilla eddie3 half a year ago and at that point it was so broken for me that I was only able to use it for checksums.
Basically at any pocket loss or any random network hiccup the tool refused to resume downloads of the same FILES and the entire file that was mid-download had to be downloaded from the start.
That forced me to just gut the checksums with the tool and download the files with browser (which somehow happaned to be more reliable then).
In recent days I tried to download some other game, *with browser*, this time tho I am on a metered mobile broadband connection, which is unreliable, can have a random reconnect thus IP change.
Here's where the fun begins.
From my testing it seems that:
Your IP changes mid download - download DEAD - refuses to reconnect. Then sudden 403.
You try to download more than 1 file at a time (NO MATTER HOW SMALL) - first you get a sizeable delay - as a bonus you get added instability (on GOG's side).
You pause 1st download. Start 2nd. Resume 1st - you ISNTANTLY get a network error, and then you get 403 shoved into your face on your 1st download.
You try to download the same file twice at the same time (for whatever reason) - 403.
Also, downloads tend to suspiciously die at 99% even when my network connection, network equipment, and entire system is fine.
To say that all this pissed me off would be an understatement.
Now, GOG's " "support" " pages have this:
https://support.gog.com/hc/en-us/articles/212807525--Error-403-when-downloading-with-a-browser?product=gog
that basically says "use our 'TOTALLY OPTIONAL' client".
No thanks. It's attrocious.
But this says 2 things:
1.Gog doesn't care to fix their backend **
2.Galaxy API supports resume
Hence my 2,3 and 4 questions.
** That's the thing.
Any SANELY configured file server survives both IP change and network issues (even if the download requires login).
There's A LOT (and I mean LUDICROUS amount) of file servers in the field that are like that. Perhaps even most of them out there are like that.
Not GOG's tho.
What's even the point of using a token in the url if the server throws a tantrum on a random issue...
Sigh...
I really don't get it.
My questions are totally legit.
I dare to say, even, crucial.
Thank you for your answer.
timppu: 1. Kalanyr's fork is at least the one most talked about in this thread. No idea if there is some other "unknown" fork that is currently more actively updated.
There's like a metaphorical billion of forks. https://github.com/eddie3/gogrepo/network/members
The problem is 99% of them are under-documented, and they don't state what they changed, also some of them are incorrectly * merging pull requests out of different projects.
There's no changelog for 99% of these forks either.
* You shouldn't exactly be doing that. I mean. You "CAN". But you shouldn't. It creates confusion and can lead to unexpected issues.
I tested gogrepopy in july/august 2021 for the first time.
I used vanilla eddie3 build.
According to github it's not been commited to for 3 YEARS.
Suprisingly, there are open pull requests dated even in 2021 (that are... getting merged in random forks in random order...).
There's the Kalanyr's fork.
No "changed from eddie3 faq" that I could find.
Then there's this:
https://github.com/Magnitus-/gogrepoc
And then there's this clusterf**k:
https://github.com/eddie3/gogrepo/network
( GO FIGURE )
One would think this question alone would make my post worthy of a response instead of being utterly ignored but I guess I am expecting too much from this forum.
The fact that 99% (100% ? I don't have time to check) of these forks directly copy paste the FAQ from the master build doesn't help.
My questions 2, 3, and 4 came because I had an issue before and had the same-ish issue now when I tried to download things from GOG.
Since I am an IT versed in few related areas I debugged this a bit and quickly found out that GOG's server backend is configured in a totally backwards ancient way that resembles *decisions* from before 2000.
TLDR:
403 fest on *ANY* issue. A N Y issue.
As I said, I used the vanilla eddie3 half a year ago and at that point it was so broken for me that I was only able to use it for checksums.
Basically at any pocket loss or any random network hiccup the tool refused to resume downloads of the same FILES and the entire file that was mid-download had to be downloaded from the start.
That forced me to just gut the checksums with the tool and download the files with browser (which somehow happaned to be more reliable then).
In recent days I tried to download some other game, *with browser*, this time tho I am on a metered mobile broadband connection, which is unreliable, can have a random reconnect thus IP change.
Here's where the fun begins.
From my testing it seems that:
Your IP changes mid download - download DEAD - refuses to reconnect. Then sudden 403.
You try to download more than 1 file at a time (NO MATTER HOW SMALL) - first you get a sizeable delay - as a bonus you get added instability (on GOG's side).
You pause 1st download. Start 2nd. Resume 1st - you ISNTANTLY get a network error, and then you get 403 shoved into your face on your 1st download.
You try to download the same file twice at the same time (for whatever reason) - 403.
Also, downloads tend to suspiciously die at 99% even when my network connection, network equipment, and entire system is fine.
To say that all this pissed me off would be an understatement.
Now, GOG's " "support" " pages have this:
https://support.gog.com/hc/en-us/articles/212807525--Error-403-when-downloading-with-a-browser?product=gog
that basically says "use our 'TOTALLY OPTIONAL' client".
No thanks. It's attrocious.
But this says 2 things:
1.Gog doesn't care to fix their backend **
2.Galaxy API supports resume
Hence my 2,3 and 4 questions.
** That's the thing.
Any SANELY configured file server survives both IP change and network issues (even if the download requires login).
There's A LOT (and I mean LUDICROUS amount) of file servers in the field that are like that. Perhaps even most of them out there are like that.
Not GOG's tho.
What's even the point of using a token in the url if the server throws a tantrum on a random issue...
Sigh...
Orkhepaj
SuperStraight Win10 Groomer Smasher
Orkhepaj Sorry, data for given user is currently unavailable. Please, try again later. View profile View wishlist Start conversation Invite to friends Invite to friends Accept invitation Accept invitation Pending invitation... Unblock chat Registered: Apr 2012
From Hungary
Posted February 22, 2022
trigonometrie: Id welcome a common download api.
The valve steam launcher is a giant behemoth and every other launcher also brings its own copy of chromium embedded framework or electron. This is duplication that isn't necessary.
garbage chromium broaf The valve steam launcher is a giant behemoth and every other launcher also brings its own copy of chromium embedded framework or electron. This is duplication that isn't necessary.
they are so slow all the time... cant even get proper small pics and takes time to load them when you scroll..