It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
avatar
lupineshadow: Append a query string to the end of whatever the url is

ht tps://gog-cdn-lumen.secure2.footprint.net/token=nva=XXXXXXXXXXXXXXXXXXXXXXXXXXXX/secure/offline/this/that/theo therxxxxxxxxxxxxxxxxxxxx.exe.xml?timestamp=1652962228
avatar
Timboli: Thanks for the info, but unfortunately I don't have the knowledge to use it.
I don't know if it will work but it's worth a try and I suspect it will if it's a GET request.

Any URL can have an optional query string.

It is part of the HTTP protocol.

https://www.rfc-editor.org/rfc/rfc2616#section-3.2.2

A query string is a list of name value pairs separated by ampersands.

These can be accessed by the webserver if asked to by the script running the page.

But if you add redundant name value pairs it doesn't affect anything unless the script explicitly checks for weird query strings and decides it doesn't want to play ball.

Basically any webpage you can add ?favouriteonlinegamingshop=gog&leastfavouriteonlinegamingshop=epic
and it won't affect functionality at all.

If it has a query string already (the start denoted by ?) then change the ? to an & to add extra name-value pairs.

Let's try...

https://www.google.com/search?q=steam
https://www.google.com/search?q=steam&favouriteonlinegamingshop=gog&leastfavouriteonlinegamingshop=epic

As you can see the extra name-value pairs don't affect the page.

But to any proxy servers in the middle it should be a different URL as the proxy servers wouldn't know that the end server doesn't care about that query string (or name-value pairs)

A timestamp (in the previous example a unixtime timestamp) would make the query unique enough unless people's manifest updates run at the exact same second for the same game.
Post edited May 19, 2022 by lupineshadow
avatar
lupineshadow: I don't know if it will work but it's worth a try and I suspect it will if it's a GET request.

Any URL can have an optional query string.

It is part of the HTTP protocol.
Once again, thanks for the info, but I guess it depends on what gogcli.exe does with the partial URL that is passed to it.

I guess I can try with the following, but I imagine only Magnitus can really implement what you said.

gogcli.exe -c Cookie.txt gog-api url-path-info -p /downloads/dying_light_the_following_enhanced_edition/en1installer1?timestamp=1652962228

EDIT
Jeepers Creepers, it seems to have worked ... unless a correction has happened meanwhile.

EDIT 2
No correction yet for - Dying Light: The Following – Enhanced Edition

And now the detail for that download is returned even without the additional - ?timestamp=1652962228

But two of the 8 URLs I try, keep returning the following. So I cannot say it works. It seems quite random any success.

GetDownloadFileInfo(downloadPath=/downloads/dying_light_the_following_enhanced_edition/en1installer6) -> retrieval body error: unexpected EOF
Post edited May 19, 2022 by Timboli
avatar
Timboli: Once again, thanks for the info, but I guess it depends on what gogcli.exe does with the partial URL that is passed to it.

I guess I can try with the following, but I imagine only Magnitus can really implement what you said.

gogcli.exe -c Cookie.txt gog-api url-path-info -p /downloads/dying_light_the_following_enhanced_edition/en1installer1?timestamp=1652962228

EDIT
Jeepers Creepers, it seems to have worked ... unless a correction has happened meanwhile.

EDIT 2
No correction yet for - Dying Light: The Following – Enhanced Edition

And now the detail for that download is returned even without the additional - ?timestamp=1652962228

But two of the 8 URLs I try, keep returning the following. So I cannot say it works. It seems quite random any success.

GetDownloadFileInfo(downloadPath=/downloads/dying_light_the_following_enhanced_edition/en1installer6) -> retrieval body error: unexpected EOF
I was given the double-dash in the path and I've tried that without much result (neither adding an extra dash in the path by itself nor trying with the previous url after trying with an extra dash in the path).

I'll try adding a pseudo-random query string argument instead this weekend.

I've just realized that technically, you don't have access to the work-around method as it has been implemented at a layer above the gog-api functionality (at the manifest generation level, technically still in the sdk module though I'm in the process of refactoring that) and you don't use the manifest generation commands, just the gog-api commands. I guess it's true that the way it is structured now in my tool is leaky and maybe it should be embedded in the api call logic (technically, this is a quirk of the GOG api). Anyways, I think I'll wrap up resumable manifest generation and progress logs in manifest generation before I tackle that (one thing at a time).

However, if you want to discuss the particulars of how my tool deals (or should deal) with this problem, maybe this is a discussion best moved to the thread for gogcli as this is less useful for gogrepoc which this thread is about.

PS: I don't think adding a pseudo-random query string argument to the path you pass to the gogcli command will do the trick as this is not what is used for the request that will hit the cdn (as I recall, the first request hits the gog servers, not the cdn). There are redirections required to obstain that url.
Post edited May 19, 2022 by Magnitus
No worries. Thanks.

Even though I was using gogcli.exe as my example, I was presuming things were similar across the board, and relevant to gorepo.py as well.
fyi, I changed the double slash for a timestamp in the query string: https://github.com/Magnitus-/gogcli/commit/04ebede72fedae364cbfa3d9c6fe78505090cd16

My empirical observations on a sample of 5 files with bad metadata are that it didn't make a difference.
Post edited May 21, 2022 by Magnitus
I frankly don't get these CDN tricks.. you're supplying a wrong address to a file (double slashes or extra parameters) and the CDN is supposed to return the correct one? How.. O_o
avatar
phaolo: I frankly don't get these CDN tricks.. you're supplying a wrong address to a file (double slashes or extra parameters) and the CDN is supposed to return the correct one? How.. O_o
The extra slashes are a result of the underlying file system convention that (apart from a leading slash), extra slashes are just redundant ways to address the same thing (foo//bar is equivalent to foo/./bar, both just being alternate ways of saying foo/bar). I don't think it's *required* that URLs behave the same way, but it's uncommon to enforce extra checks at that level.

It's also a common convention to ignore unrecognised parameters in query strings (?unknown=value), mostly to aid with forward compatibility (don't reject a parameter that may have meaning in the future). Again, strict checking here might be allowed, but be more trouble than it's worth.

So, neither of these methods is foolproof, not guaranteed to work in the future for any given web server, but for now they provide a means to implement a workaround to real problems which isn't very likely to stop working any time soon.
avatar
phaolo: I frankly don't get these CDN tricks.. you're supplying a wrong address to a file (double slashes or extra parameters) and the CDN is supposed to return the correct one? How.. O_o
avatar
mvscot: The extra slashes are a result of the underlying file system convention that (apart from a leading slash), extra slashes are just redundant ways to address the same thing (foo//bar is equivalent to foo/./bar, both just being alternate ways of saying foo/bar). I don't think it's *required* that URLs behave the same way, but it's uncommon to enforce extra checks at that level.

It's also a common convention to ignore unrecognised parameters in query strings (?unknown=value), mostly to aid with forward compatibility (don't reject a parameter that may have meaning in the future). Again, strict checking here might be allowed, but be more trouble than it's worth.

So, neither of these methods is foolproof, not guaranteed to work in the future for any given web server, but for now they provide a means to implement a workaround to real problems which isn't very likely to stop working any time soon.
Ah I see, thank you for the explanation.
avatar
Magnitus: fyi, I changed the double slash for a timestamp in the query string: https://github.com/Magnitus-/gogcli/commit/04ebede72fedae364cbfa3d9c6fe78505090cd16

My empirical observations on a sample of 5 files with bad metadata are that it didn't make a difference.
"It didn't make a difference" as in both methods worked or as in both methods didn't work?
Post edited May 21, 2022 by mrkgnao
avatar
mrkgnao: "It didn't make a difference" as in both methods worked or as in both methods didn't work?
As in both methods didn't work.

avatar
phaolo: I frankly don't get these CDN tricks.. you're supplying a wrong address to a file (double slashes or extra parameters) and the CDN is supposed to return the correct one? How.. O_o
Yes, I thought that was weird too (if it had worked, it would have been a quirk of the cdn, but it didn't). The query string argument made more sense (I've seen it applied in the past to invalidate caches), but it didn't make a difference.

avatar
mvscot: So, neither of these methods is foolproof, not guaranteed to work in the future for any given web server, but for now they provide a means to implement a workaround to real problems which isn't very likely to stop working any time soon.
Well, so far, it's very inconclusive on my end. I think the main thing that would make a difference here is that you retry a lot as those bad metadata url have been shown to resolve at some point on their own. However, if it takes 5-10 minutes or more to resolve, a tool might as well just compute the checksum itself at that point.
Post edited May 21, 2022 by Magnitus
avatar
mrkgnao: "It didn't make a difference" as in both methods worked or as in both methods didn't work?
avatar
Magnitus: As in both methods didn't work.
Thanks. I tested the timestamp method on about 20 xml files that I currently have issues with. It didn't resolve a single one.
So far I just jump around the globe via VPN until it works for me, it's such a pain lol.... still better then manually going though galaxy to update them!
avatar
Starkrun: So far I just jump around the globe via VPN until it works for me, it's such a pain lol.... still better then manually going though galaxy to update them!
I record the ones that failed for me and retry them once a day (via a batch file I wrote). They tend to fix themselves at a rate of about one per day, give or take, which is faster than the rate of new ones appearing with game updates, so the list is getting shorter, although a few (e.g. Battle Brothers, Blade of Darkness, Ghostrunner) seem very stubborn.

That is, until the next full gogrepoc update, which will restart the cycle anew.

All is well in the state of Denmark, or rather Poland.
Post edited June 02, 2022 by mrkgnao
avatar
mrkgnao: They tend to fix themselves at a rate of about one per day, give or take, which is faster than the rate of new ones appearing with game updates, so the list is getting shorter, although a few (e.g. Battle Brothers, Blade of Darkness, Ghostrunner) seem very stubborn.
Dying Light: The Following (Enhanced Edition) is another that stubbornly resists.

The others on my list are now fixed, but a new one fails.
Here is my list of annoyingly stubborn ones and my notes:

aquanox_deep_descent - Extras?
bioshock_infinite - XML
cyberpunk_2077_game - XML
dying_light_the_following_enhanced_edition - XML
everspace_2 - XML
forgive_me_father - Connection issue?
ghostrunner - XML
metro_exodus - XML
sacred_2_gold - XML
stellaris - XML
the_bards_tale_iv_directors_cut - XML
Post edited June 04, 2022 by Starkrun