It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
avatar
B1tF1ghter: ......
It's been a while since I have used any version of gogrepo.py, and I am certainly no expert of it or Python. I did however make a Windows GUI to run the original (last update) and Kalanyr's basic fork. Many months later, maybe over a year, I switched to gogcli.exe by Magnitus, and prefer that now.

Some things to note about gogrepo.py.

The original (last update) does not support Resume.

The fork by Kalanyr does support Resume, if you get the Developer version variant.

https://github.com/Kalanyr/gogrepoc/tree/dev

As you can see on that web page, you get a more up-to-date command list. To get a more precise or up-to-date one, use the help command.

I only ever briefly tried the Developer version. In fact my PC had issues with any Kalanyr fork when it came to downloading, no doubt due to my PC being underpowered and an older OS.
Post edited February 22, 2022 by Timboli
avatar
B1tF1ghter: Then there's this:
https://github.com/Magnitus-/gogrepoc
Consider mine an aborted attempt. I originally intended to refactor the codebase into something I could more comfortably work with and then decided to just start from scratch with something else.

I had just learned golang (in great part exactly for use-cases like this: to write self-contained clients that would just run across machines with nothing to install or external dependencies that could be different) and was more interested in practising that than refactoring a Python codebase at that particular point in time, plus I really wanted something that was self-contained and easy to run as a general principle (I've done enough tech support on others' machine when their libraries or version of the Python/nodejs interpreter didn't exactly match mine to know this is not the future of portable code that should run on end-users machines... for a while, I thought docker might be the answer to running things with extra dependencies portably on users' machines, but then realized that the majority of developers had cognitive issues ramping their docker skills, so it was even more hopeless for non-technical users and thus, at this point I'm just gunning for the simplicity of self-contained binaries compiled cross-platform).
Post edited February 22, 2022 by Magnitus
avatar
B1tF1ghter: Then there's this:
https://github.com/Magnitus-/gogrepoc
avatar
Magnitus: Consider mine an aborted attempt. I originally intended to refactor the codebase into something I could more comfortably work with and then decided to just start from scratch with something else.

I had just learned golang (in great part exactly for use-cases like this: to write self-contained clients that would just run across machines with nothing to install or external dependencies that could be different) and was more interested in practising that than refactoring a Python codebase at that particular point in time, plus I really wanted something that was self-contained and easy to run as a general principle (I've done enough tech support on others' machine when their libraries or version of the Python/nodejs interpreter didn't exactly match mine to know this is not the future of portable code that should run on end-users machines... for a while, I thought docker might be the answer to running things with extra dependencies portably on users' machines, but then realized that the majority of developers had cognitive issues ramping their docker skills, so it was even more hopeless for non-technical users and thus, at this point I'm just gunning for the simplicity of self-contained binaries compiled cross-platform).
yeah pyhton and ~3000 lines does not work well
avatar
Phin77: At the beginning of the script I populate a string variable and sub in spaces for line breaks and inject it right after the - skipfiles option in my download statement:

filter1=`cat filter.txt | tr '\n\r' ' '`

python3 gogrepoc.py download -os windows -lang en -skipfiles $filter1

Edit: I suppose you could simply use -skipfiles patch_*.*. I got too fancy I suppose...
This is perfect, I never thought of this and have PS1 script I'm already using for quick updates, I'll toss this in. Thanks a lot!
From kolhrak via chat:

i'd reply to the thread: i'd like to ask if anyone is working on a version of GOG repo that doesn't have memory leak issues. For some reason certain games in our libraries aren't playing nice with gog repo and it'll hang on them and overdeclare memory until it throws an exception (at least with kalanyr's version)

avatar
Orkhepaj: yeah pyhton and ~3000 lines does not work well
Ironically, I prefer Python as a programming language actually, but when picking a technological solution, there is more to consider than just the code.

If you always go with your favourite programming language no matter the problem you are trying to solve, you are more likely to come up with subpar solutions. Its ok when working on some personal project (it is your time, you do what you want with it), but when you're getting paid to produce work for an employer, its a selfish decision to make.
Post edited February 23, 2022 by Magnitus
avatar
blotunga: Nice idea with the filters, I only identified a handful (too lazy), but could you paste somewhere your full list, I think I might add some more :)
I'm using these at the moment: "patch_*.exe" "patch_*.bin" "patch_*sh" "*_(32bit)_*.exe" "*_(32bit)_*.bin"
Thanks much!

Mine mostly matches the one posted above but I'll post it here minus the line breaks. (I hated the way my last post looked):

patch_*.* *_?32bit?_*.* *_32_* *_osx.* *_mac.* *_linux* *_making_of* *_developers_diaries* *_devd.* *_behind_the_scenes* *_commentary* *_storyboards* *_interview* *_videos.* *_video.* *_trailer.* *_trailers.* *_voiceover_* *_commercials.* *_animated_short.* *_tv_spot* *_extras_pack* *_mocap_session* *_recording_session* *_deleted_scenes* *_behind_the_screens* *_documentary* *_footage* *_flac.* *_wav.* *_AAC.* *_wma.* *.mp4 *_handbuch.* *_manuel_* *_manuale* *_referenzkarte* *_de.* *_De.* *_DE.* *_fr.* *_Fr.* *_FR.*b*_ru.* *_Ru.*b*_RU.* *_it.* *_It.* *_IT.* *_es.* *_Es.* *_ES.* *_pt.* *_Pt.* *_PT.* *_pl.* *_Pl.* *_PL.* *_zog_* *_ita.* *_hu_* *_pl_* *_ru_* *_tr_* *_zh_* *_ch_* *_cz_* *_de_* *_es_* *_fr_* *_it_* *_jp_* *_esp_* *_ita_* *_german* *_deutsch* *_french* *_spanish* *_russian* *_polish* *_italian* *_euro.* *_geschichte* *_losung* *.tar.gz *.sh gog_*.sh *.dmg *.pkg

There's some more that I've left out here because it's mostly specific for certain games that duplicate bonus materials for every game in a series.. Like Quest for Glory or the Zork series.. I downloaded the materials for the 1st in the series and then added the file names to my filters.. Sure.. They won't ever get updated.. But I won't be downloading duplicate copies of sometimes bulky bonus files.

As for not downloading patch files at all, it's true that I may not have the most "up to date version" of some games but unless it's multi-player or heavily modable and require a specific version... I can pretty much get the full experience of the game. Heck.. Even the aforementioned 1.31 to 1.32 patch for the Witcher 3, only adds simplified Chinese as a language option for the game and mods are compatible between 1.32 and 1.31.. Likely why they have left it as a patch rather than updating the base installer.

That won't be the case for everything.. I'll miss a few things but I'm satisfied I'll have as much of my collection as I could humanly play given my backlog if anything ever happens to gog.

edit: fixed typos. Also.. That filter list looks be affected by the page justification.. Spacing is odd.
Post edited February 23, 2022 by Phin77
In case anyone is interested...

I ran a full update (3 hours) on my GOG library (1892 games) just now, using gogrepoc (Kalanyr) with the two recent xml fixes incorporated:
- this and
- this.

It passed without any issues, with the exception of one game that failed (and continued to fail on my attempting to update it again by itself), the game being "Mary Skelter: Nightmares" and the error messages being:
20:55:23 | xml parsing error occurred trying to get md5 data for setup_mary_skelter_nightmares_v1.00_(22308).exe
20:55:24 | xml parsing error occurred trying to get md5 data for setup_mary_skelter_nightmares_v1.00_(22308)-1.bin
20:55:59 | request failed: HTTPSConnectionPool(host='gog-cdn-lumen.secure2.footprint.net', port=443): Read timed out. (3 retries left) -- will retry in 5s...
20:56:40 | request failed: HTTPSConnectionPool(host='gog-cdn-lumen.secure2.footprint.net', port=443): Read timed out. (2 retries left) -- will retry in 5s...
20:57:20 | request failed: HTTPSConnectionPool(host='gog-cdn-lumen.secure2.footprint.net', port=443): Read timed out. (1 retries left) -- will retry in 5s...

I was able to download this game's files via the browser (they are not new).

I'm guessing this is something on GOG's side given that it's only affecting one game (in my library).

I then downloaded the files that have changed since my last update --- only 4 files (1GB), since I do regular updates daily and have done a full update a couple of weeks ago. No issues there either.
Post edited February 23, 2022 by mrkgnao
low rated
I have tried kalanyr's fork and I had a come back of an old issue and a new one.
When I tried the vanilla eddie3 half a year ago I remember it taking FOREVER to update the manifest (for like ~100-130 products) and downloading "quite some" data while doing so.

Just now I have tried doing a full manifest update with kalanyr's fork and it is even worse. Not only it seems to hang, at some point it starts downloading SOME data at max available net speed for seemingly indefinite amounts of time (I am on a metered connection, so I only left it for few dozen minutes before aborting for damage control).

I am trying to download the evil within (1).
Since I seemingly cannot get the manifest at all with kalanyr's fork (I tried the -ids cvar, didn't help, "xml parsing error" on the first chunk and THEN the "infinite download") I tried with eddie3, and I had an "xml parsing error" on EVERY file.
Btw eddie3 did NOT download excessive amounts of data while trying to update the manifest.

Is there another way to get md5's?
It's kind of worthless to me without the original checksums (as much as md5 is proven broken for "a while" now, it's still better than nothing)

edit:
fixed some spelling
Post edited February 23, 2022 by B1tF1ghter
avatar
B1tF1ghter: I seemingly cannot get the manifest at all with kalanyr's fork (I tried the -ids cvar, didn't help, "xml parsing error" on the first chunk and THEN the "infinite download") I tried with eddie3, and I had an "xml parsing error" on EVERY file.
I'm not sure why people are downvoting you..
Anyway, you started using gogrepo at the worst time.. It worked perfectly for years, until the last few days.
People have found some solutions\workarounds for now, see the posts above yours.
For the other questions, maybe others can reply.
Post edited February 23, 2022 by phaolo
avatar
mrkgnao: In case anyone is interested...

I ran a full update (3 hours) on my GOG library (1892 games) just now, using gogrepoc (Kalanyr) with the two recent xml fixes incorporated:
- this and
- this.

It passed without any issues, with the exception of one game that failed (and continued to fail on my attempting to update it again by itself), the game being "Mary Skelter: Nightmares" and the error messages being:
20:55:23 | xml parsing error occurred trying to get md5 data for setup_mary_skelter_nightmares_v1.00_(22308).exe
20:55:24 | xml parsing error occurred trying to get md5 data for setup_mary_skelter_nightmares_v1.00_(22308)-1.bin
20:55:59 | request failed: HTTPSConnectionPool(host='gog-cdn-lumen.secure2.footprint.net', port=443): Read timed out. (3 retries left) -- will retry in 5s...
20:56:40 | request failed: HTTPSConnectionPool(host='gog-cdn-lumen.secure2.footprint.net', port=443): Read timed out. (2 retries left) -- will retry in 5s...
20:57:20 | request failed: HTTPSConnectionPool(host='gog-cdn-lumen.secure2.footprint.net', port=443): Read timed out. (1 retries left) -- will retry in 5s...

I was able to download this game's files via the browser (they are not new).

I'm guessing this is something on GOG's side given that it's only affecting one game (in my library).

I then downloaded the files that have changed since my last update --- only 4 files (1GB), since I do regular updates daily and have done a full update a couple of weeks ago. No issues there either.
I have many games with the same issue: bound_by_flame, celtic_kings_rage_of_war, children_of_zodiarcs... None of them had problems a couple weeks ago but now they give me the same error:
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='gog-cdn-lumen.secure2.footprint.net', port=443): Read timed out.

I hope someone will be able to fix whatever problem the script is having
Post edited February 23, 2022 by Lhun Duum
avatar
Lhun Duum: I have many games with the same issue: bound_by_flame, celtic_kings_rage_of_war, children_of_zodiarcs... None of them had problems a couple weeks ago but now they give me the same error:
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='gog-cdn-lumen.secure2.footprint.net', port=443): Read timed out.

I hope someone will be able to fix whatever problem the script is having
Interesting. I have all three games you mentioned and had no problem with them.

Did you apply the two xml fixes I pointed to?

avatar
B1tF1ghter: Since I seemingly cannot get the manifest at all with kalanyr's fork (I tried the -ids cvar, didn't help, "xml parsing error" on the first chunk and THEN the "infinite download") I tried with eddie3, and I had an "xml parsing error" on EVERY file.
Did you apply the two xml fixes discussed in this thread? See here.
Post edited February 23, 2022 by mrkgnao
avatar
mrkgnao: Interesting. I have all three games you mentioned and had no problem with them.

Did you apply the two xml fixes I pointed to?
Some of those links (for the xml metadata) are just duds (I also validated them with curl on the command line by manually appending ".xml" to the final download link, its really the links). I don't think there is much you can do.

There aren't that many (last time I checked, I had 3), but they are out there.

One that was consistently bad for me was the "Clan of the Kraken" dlc for Northgard, though I haven't tried it for 2 days or so.
Post edited February 23, 2022 by Magnitus
avatar
mrkgnao: Interesting. I have all three games you mentioned and had no problem with them.

Did you apply the two xml fixes I pointed to?
Yes I re-downloaded a fresh copy of the script from github then apply the two changes you pointed to and had the issue when trying to update. I'm starting to think that maybe there is a problem from a CDN server only in some specific region. I'll try to use a VPN to see if updating from another continent solve the issue.
Post edited February 23, 2022 by Lhun Duum
With a fresh copy and the 2 mentioned edits it will run now better but there are still "heavy" errors with - for example - metro 2033 and witcher 3. I have a logfile of that session (I did interrupt the session after the witcher 3 errors.). Does anybody need that file? Or parts of of log?
avatar
B1tF1ghter: Is there another way to get md5's?
If you use my GOGPlus Download Checker program, you don't need to get the MD5 values, but it is a lot slower. It doesn't require gogrepo.py but does need Windows.

https://github.com/Twombs/GOGPlus-Download-Checker

--------------------------------------------------------

If you want to use gogrepo.py, then my GOGRepo Simple GUI program uses it but just to get the MD5 values for a game on the fly ... builds the manifest bit by bit as you need it, so not all downloaded in one go, just the game you are working with. It is not a downloader though, so you need to use your browser etc for that bit.

https://github.com/Twombs/GOGRepo-GUI/releases/tag/v1.5_SG

---------------------------------------------------------

Or just use gogcli by Magnitus, it doesn't require Python, and it will get the manifest for just one game.

https://github.com/Magnitus-/gogcli

And I also made a GUI for that.

https://github.com/Twombs/GOGcli-GUI