It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
avatar
timppu: But I do not think it ever had an option to add ALL your games to the download queue, with one click. If you had 1000 games, you had to click (more than) 1000 times to add them all into the download queue.
I don't think I would have used it if it was that much hassle. Was there an option to view only updated games or something similar? (I no longer have it set up on any of my easily-accessible older systems.)

I do remember it had the odd glitch, but nothing show-stopping.

Found an old screenshot: https://i.redd.it/2d0uosslbkmz.jpg

Can you remember what happened when you selected the "41 game updates" notification in that image? Pretty sure that's the route I took. I don't ever remember having hundreds of games in the queue (come to think of it, adding too many would trigger the glitches, IIRC).

I remember using it until GOG downright refused to fix broken gogdownloader links, then switching to gogrepo.py, which is of course great (especially since I don't primarily use Windows).

I do wish GOG would make the download API public and stable, at a minimum.

Even more pie in the sky, I wish several organisations (e.g. gog, itch, humble) would actively engage with the community (including folks like the playnite devs) to develop a standard open protocol (RFC-style), capable of meeting the needs of various stores (not just the smaller ones), along with an open source reference client (or separate "client" and "downloader", the latter being rather simpler). The end vision would be for any user to use either the client provided by the store they bought the game from (might be the same reference client for some), or any other implementation that the community wrote, or even another store's client (that's interoperability for you).

Is there any need for every store to have its own proprietary protocol and (uniquely buggy or inadequate) client? Organisations often find that they don't miss the "control" afforded by their own implementation when the development and maintenance can be shared, or even left, with others.

There already exist open shared user authentication schemes, so it would be possible for each user to decide whether to use the same identity with one or more stores (e.g. "log in with google", "log in with steam"), or separate identities, in whichever combination.

Note that while I strongly advocate DRM-free games, I wouldn't limit the protocol to DRM-free games only, as a pragmatic principle to get as many organisations on board as possible.

My own first priority is getting my installers backed up (DRM-free or otherwise) using a basic multi-platform downloader (command-line being superior for automation purposes), but I'm not averse to using an OS-specific client to install and play games conveniently, so long as it's optional. Oh, and achievements/multi-player should never be platform-specific (we also need a common API for those, rather than lazy devs directly targeting the Steam API).
Just my two cents worth of ideas regarding the current conversation.

Some people aren't that dedicated to their collection. Other people are. GOG is marketed to appeal to the people that want ownership of their games. Those who share a concern about preserving a personal collection. Those that would fill a closest with all the game boxes they use to be sold in or display them in their living room if they have the space to do so. People that would have a separate hard drive to keep updates so they don't need to download them again, or worse, risk the source website being lost to antiquity. For the former group of people, GOG usually falls behind more expedient though less ownership focused services.

I have that closet. I have a computer built just to download games and keep them updated. My Gogrepo folder takes up over 3 TB. I use to download my entire collection, one at a time, once a year to compare to the installers I had stored. Gogrepo certainly simplifies that by checking for updates and telling me which games have changes. Something that I found the updated notification in library failed to do reliably. That experience is why I still make a new manifest file periodically.

GOG Downloader didn't compare files. Unless I'm mistaken Galaxy doesn't either. Bulk downloading is a huge benefit. Though for me, the main issue Gogrepo addresses is the flawed updated flag which the alternatives are still dependent on.

The initial setup can be a challenge. It's unfortunate that there isn't a distribution of Gogrepo that does more of the configuring for the user. I recall suggesting some time ago that it would be nice if Gogrepo could update itself when something changes site side that breaks it. Also downloading 3 TB over a weekend would be excessive. I think there was some setting which addresses bandwidth usage by implementing delays if I recall correctly. Aside from that it would fall on the user to space out their downloads appropriately.
Post edited February 20, 2022 by SKARDAVNELNATE
avatar
SKARDAVNELNATE: GOG Downloader didn't compare files. Unless I'm mistaken Galaxy doesn't either.
Galaxy certainly didn't last I checked (quite some time ago). However, I believe gogdownloader went one better, and compared individual *chunks* of files for which separate checksums were stored, and so was capable of fixing partial downloads and corrupt files with minimal bandwidth use.

I don't think even gogrepo.py has access to that checksum data (unlikely it's even still generated).

Someone correct me if I'm misremembering this.
Id welcome a common download api.
The valve steam launcher is a giant behemoth and every other launcher also brings its own copy of chromium embedded framework or electron. This is duplication that isn't necessary.
I didn't know that about Downloader. Did that just apply to correcting the current download job or would it also compare past downloads to what is currently available? From what I recall Downloader only showed me the updates that were flagged in library. When doing yearly downloads I found several that hadn't been flagged during the year.

One downside of Gogrepo is when I only need certain files. The options are to use Gogrepo to download the entire set of installers for that title, or pick out just the files I need through my browser. It doesn't let me do both automated and file specific.
Post edited February 20, 2022 by SKARDAVNELNATE
avatar
SKARDAVNELNATE: I didn't know that about Downloader. Did that just apply to correcting the current download job or would it also compare past downloads to what is currently available?
Both.

avatar
SKARDAVNELNATE: From what I recall Downloader only showed me the updates that were flagged in library. When doing yearly downloads I found several that hadn't been flagged during the year.
That is true, but it has nothing to do with the downloader. It is just GOG's poor design of not flagging some games (unbundled games, kickstrated games, preordered games, goodies, etc.). You need to do the same "yearly download" (or "update") with gogrepo, browser, or any other download method you use. GOG's flagging, like everything else on GOG, simply cannot be depended on.

avatar
SKARDAVNELNATE: One downside of Gogrepo is when I only need certain files. The options are to use Gogrepo to download the entire set of installers for that title, or pick out just the files I need through my browser. It doesn't let me do both automated and file specific.
Gogrepo does allow it, but I can't say it's extremely friendly. See for example, the -skipfiles download option (in addition to the -lang and -os options, of course).

avatar
mvscot: Someone correct me if I'm misremembering this.
I believe you are correct.
Post edited February 20, 2022 by mrkgnao
avatar
Kalanyr: I'll keep an eye on this and fix the URL/encoding detection, when things have stabilised. Thanks for the reports.
Not knowing whether or not the query string is still sometimes used, and looking at the suggestions to simply append ".xml" to the url rather than look for the question mark (marker for the beginning of the query string), I devised the following patch:
@@ -668,7 +668,10 @@ def fetch_file_info(d, fetch_md5,updateSession):
file_ext = os.path.splitext(urlparse(response.url).path)[1].lower()
if file_ext not in SKIP_MD5_FILE_EXT:
try:
- tmp_md5_url = response.url.replace('?', '.xml?')
+ if '?' in response.url:
+ tmp_md5_url = response.url.replace('?', '.xml?')
+ else:
+ tmp_md5_url = response.url + '.xml'
md5_response = request(updateSession,tmp_md5_url)
shelf_etree = xml.etree.ElementTree.fromstring(md5_response.content)
d.md5 = shelf_etree.attrib['md5']
(Of course, the GOG forums don't like consecutive - or initial - whitespace, so the above patch is useless as-is.)
avatar
mrkgnao: You need to do the same "yearly download" (or "update") with gogrepo, browser, or any other download method you use.
Periodically starting a new manifest to compare against has alleviated that issue for me. Since it doesn't rely on the updated flag I don't need to re-download installers that I know are current. With the old method I didn't know if they were still current until after downloading and comparing them.
avatar
mrkgnao: You need to do the same "yearly download" (or "update") with gogrepo, browser, or any other download method you use.
avatar
SKARDAVNELNATE: Periodically starting a new manifest to compare against has alleviated that issue for me. Since it doesn't rely on the updated flag I don't need to re-download installers that I know are current. With the old method I didn't know if they were still current until after downloading and comparing them.
You don't need a new manifest or any manual comparison. Just use gogrepo update with the -full flag.
Post edited February 21, 2022 by mrkgnao
avatar
SKARDAVNELNATE: GOG Downloader didn't compare files. Unless I'm mistaken Galaxy doesn't either.
avatar
mvscot: Galaxy certainly didn't last I checked (quite some time ago). However, I believe gogdownloader went one better, and compared individual *chunks* of files for which separate checksums were stored, and so was capable of fixing partial downloads and corrupt files with minimal bandwidth use.

I don't think even gogrepo.py has access to that checksum data (unlikely it's even still generated).

Someone correct me if I'm misremembering this.
That chunk data is available and is how download resuming works. I've toyed around with using it for repairing files but there's synchronisation issues that I never really solved (you have to grab the data at the time you download the file (because otherwise you're potentially comparing to a different file or the data is gone) but once you do that, the file can become stale (there's no guarantee that for a bad chunk detected that a good chunk exists to download anymore)).
I tried to do a full update with the patched version for the XML issue and while the XML issue is indeed fixed, after a few minutes, it looks like the GOG server starts denying connections. So I added time.sleep(30) between processing of each game, but the same problem occurs.

So basically, it looks like GOG has added a request limit on their side and anyone with a substantial GOG collection will have trouble using this script. This is really annoying when they don't themselves offer a good alternative to sync your collection...
avatar
mrkgnao: You don't need a new manifest or any manual comparison. Just use gogrepo update with the -full flag.
Is there any advantage to that, compared to just renaming or deleting the old manifest, and running the "normal" gogrepo update?

In both cases it goes through each and every game in your GOG library and fetches all the relevant information for them, right?

I personally prefer starting from the clean table in such cases, in order to completely avoid all kinds of "oh this old manifest file is now incompatible this newer gogrepo manifest format" etc. Not my headache.
avatar
mrkgnao: You don't need a new manifest or any manual comparison. Just use gogrepo update with the -full flag.
avatar
timppu: Is there any advantage to that, compared to just renaming or deleting the old manifest, and running the "normal" gogrepo update?

In both cases it goes through each and every game in your GOG library and fetches all the relevant information for them, right?

I personally prefer starting from the clean table in such cases, in order to completely avoid all kinds of "oh this old manifest file is now incompatible this newer gogrepo manifest format" etc. Not my headache.
No advantage that I'm aware of. I personally prefer the -full method.
avatar
mrkgnao: No advantage that I'm aware of. I personally prefer the -full method.
Is full update still working for you? As I described earlier, for me it's no longer working correctly after 15 min or something, it's like the gog server is refusing the connections. Since my game collection is somewhat big, I no longer can get a full update. (I've applied the XML patch, so it's not because of that)
I had some errors with the full update but then it recovered. Then errors again after a while and so on. Let's see after the sale on my next monthly update.