Posted July 22, 2017

Starkrun
Poops Darkmatter
Registered: Aug 2009
From United States

Kalanyr
Wyrdling
Registered: Nov 2010
From Australia
Posted July 23, 2017
Shouldn't be necessary now, I think I've fixed all.the points where the non-Gakaxy including layout would cause problems. Obviously the skipshared and skipgalaxy flags won't work on the old entries though and skipstandalone will skip what would be both standalone and shared for those entries.
I recommend doing a full update when you can though (there's no need to delete and rebuild from scratch or anything a full update will detect anything in your manifest no longer on gog and remove it, so a full update and creating from scratch should end exactly the same).
I recommend doing a full update when you can though (there's no need to delete and rebuild from scratch or anything a full update will detect anything in your manifest no longer on gog and remove it, so a full update and creating from scratch should end exactly the same).

Starkrun
Poops Darkmatter
Registered: Aug 2009
From United States
Posted July 23, 2017

I recommend doing a full update when you can though (there's no need to delete and rebuild from scratch or anything a full update will detect anything in your manifest no longer on gog and remove it, so a full update and creating from scratch should end exactly the same).
Thank you for keeping this alive, I know woolymethodman has had a lot of real life happening for him. I think I speak for many people when I say, we all truly appreciate your work on this!

timppu
Favorite race: Formula__One
Registered: Jun 2011
From Finland
Posted July 23, 2017

These are the problems I get, every time (for many many months already, maybe even a year) when running the full update. Or do you get some other kinds of errors? I don't get any error with Dreamfall Chapters (Season Pass) though.
00:59:24 | ( 8 / 1372) fetching game details for a_golden_wake...
00:59:31 | xml parsing error occurred trying to get md5 data for setup_a_golden_wake_2.0.0.4.exe
...
04:03:55 | ( 602 / 1372) fetching game details for kyn...
04:04:02 | request failed: HTTP Error 404: Not Found. will not retry.
04:04:02 | no md5 data found for setup_kyn_2.1.0.4.exe
04:04:07 | request failed: HTTP Error 404: Not Found. will not retry.
04:04:07 | no md5 data found for setup_kyn_2.1.0.4-1.bin
...
07:35:42 | (1297 / 1372) fetching game details for walking_dead_season_1_the...
07:35:58 | request failed: HTTP Error 404: Not Found. will not retry.
07:35:58 | no md5 data found for patch_walking_dead_season_1_2.1.0.5.exe
...
07:50:04 | (1343 / 1372) fetching game details for x3_terran_war_pack...
07:50:31 | xml parsing error occurred trying to get md5 data for patch_x3_terran_war_pack_2.2.0.5.exe
Post edited July 23, 2017 by timppu

timppu
Favorite race: Formula__One
Registered: Jun 2011
From Finland
Posted July 23, 2017
Now that I found the way to divide my gogrepo installer collection into two, that also speeds up my downloads and installer verification greatly.
Because I have two unused 2TB USB hard drives, but my GOG installer + extras collection (Windows/English only) is already bigger than 2TB, I've divided the GOG collection to those two drives so that GOG games which start with A-N (or numerals, like 1nsane, 7th Legion and so on) are on one USB drive, and O-Z are on the other.
After I've done a full update getting a full manifest file, I then make two copies of it and from one remove all the titles from #-N, and the other I remove O-Z. Using these two separate partial manifest files, I can run gogrepo download (and verification) on two separate PCs (laptops) at the same time, using separate internet connections. I have two internet connections I can utilize, my main 10Mbps cable modem connection, and my unlimited 4G mobile connection that gives me around 40Mbps at my home on a good day (It has reached around 120Mbps or so elsewhere, but not at my home unfortunately).
So right now I am downloading the new and updated #-N installers using the 10Mbps cable modem connection, and the O-Z installers with the 40Mbps mobile connection. GOG has changed lots of installers lately so for the first patch I have something like 75 GB to download, and for the second batch a whopping 200 GB to download. Very nice that I can now download them using a total 50Mbps speed, utilizing all my available bandwidth.
Earlier I did try some internet suggestions on how to merge two different internet connections into one in Windows (to get benefit of both bandwidths on the same device), but for some reason I never got them to work, I'd still get the bandwidth only from one of them.
Naturally also verification will speed up as now I can verify #-N and O-Z at the same time, running gogrepo twice at the same time.
Because I have two unused 2TB USB hard drives, but my GOG installer + extras collection (Windows/English only) is already bigger than 2TB, I've divided the GOG collection to those two drives so that GOG games which start with A-N (or numerals, like 1nsane, 7th Legion and so on) are on one USB drive, and O-Z are on the other.
After I've done a full update getting a full manifest file, I then make two copies of it and from one remove all the titles from #-N, and the other I remove O-Z. Using these two separate partial manifest files, I can run gogrepo download (and verification) on two separate PCs (laptops) at the same time, using separate internet connections. I have two internet connections I can utilize, my main 10Mbps cable modem connection, and my unlimited 4G mobile connection that gives me around 40Mbps at my home on a good day (It has reached around 120Mbps or so elsewhere, but not at my home unfortunately).
So right now I am downloading the new and updated #-N installers using the 10Mbps cable modem connection, and the O-Z installers with the 40Mbps mobile connection. GOG has changed lots of installers lately so for the first patch I have something like 75 GB to download, and for the second batch a whopping 200 GB to download. Very nice that I can now download them using a total 50Mbps speed, utilizing all my available bandwidth.
Earlier I did try some internet suggestions on how to merge two different internet connections into one in Windows (to get benefit of both bandwidths on the same device), but for some reason I never got them to work, I'd still get the bandwidth only from one of them.
Naturally also verification will speed up as now I can verify #-N and O-Z at the same time, running gogrepo twice at the same time.
Post edited July 23, 2017 by timppu

sbaylus
New User
Registered: Jun 2012
From United States
Posted July 29, 2017
Perfect application for wild cards in the -id option. Using real regular expressions you could use -id ^[0-9a-n]* and -id ^[o-z] with the same manifest file with two commands (I'll assume the manifest is open r/o for download commands). I'm working on this ...

timppu
Favorite race: Formula__One
Registered: Jun 2011
From Finland
Posted July 29, 2017
Yeah that would certainly be nice as that would drop the need for little manual work with the full manifest file (dividing it into two).

disi
Bandit
Registered: Jun 2012
From United Kingdom
Posted July 31, 2017
Ever since the Galaxy installers, I did not update any more :*(
@Kalanyr, I appreciate the work. Will check this out tomorrow, I think.
@Kalanyr, I appreciate the work. Will check this out tomorrow, I think.

timppu
Favorite race: Formula__One
Registered: Jun 2011
From Finland
Posted August 01, 2017
Why not? gogrepo downloads the classic installers by default, not the Galaxy-embedded installers. So you are as good to go as ever.
Be aware though that lots of classic installers have been updated recently as well. 1½ weeks ago I redownloaded around 270GB of installer files (most were those updated ones, but I had also bought some big new games), but this week there hasn't been similar flood of updated installers (I got only a few updated ones last week).
I just updated/downloaded my collection yesterday, went fine. I noticed though there was the same "two files with the same filename" problem as people have reported in this thread before with Sudden Strike 3, but today it was fixed. The glitch caused gogrepo to repeatedly redownload the said installer whenever one runs gogrepo download (only once per command).
Which reminds me:
timppu: Ok so for Michonne only the first three installer files are needed, and for Reigns only the first one? And we don't know why there are these duplicate files?
At least the download pages for both games are wrong then, as they suggest Michonne is in six parts (three parts in reality), and Reigns in two parts (when it is only in one). Yeah GOG needs to fix those download pages either way,
Kalanyr: Michonne and Reigns have different but similar problems.
Michonne is as you said.
Reigns has 64 and 32 bit versions , but the 32 bit version is present as both Part 2 and as a separate download (both with same name but different size / md5) , whereas Part 1 is the 64 bit version.
Unfortunately I have no idea which of the "duplicates" is the correct one. While GOG Galaxy exists it doesn't matter because it will repair the wrong one automatically by updating it. So yeah you only need 1 complete set of them but I have no idea which one that should be. Have these games (different files with identical filenames on the same download page) been reported to GOG, by the way?
I noticed the same problem with the "new" release of Sudden Strike 3: Arms for Victory, ie. it had two installer files (1 or 2, 2 of 2) which had identical filenames, causing gogrepo to redownload the installer whenever you run the gogrepo download command. I reported it to GOG support, and they fixed it within a day (now there is only one installer file).
I can't check whether those aforementioned games are fixed too as I don't have them.
Be aware though that lots of classic installers have been updated recently as well. 1½ weeks ago I redownloaded around 270GB of installer files (most were those updated ones, but I had also bought some big new games), but this week there hasn't been similar flood of updated installers (I got only a few updated ones last week).
I just updated/downloaded my collection yesterday, went fine. I noticed though there was the same "two files with the same filename" problem as people have reported in this thread before with Sudden Strike 3, but today it was fixed. The glitch caused gogrepo to repeatedly redownload the said installer whenever one runs gogrepo download (only once per command).
Which reminds me:

At least the download pages for both games are wrong then, as they suggest Michonne is in six parts (three parts in reality), and Reigns in two parts (when it is only in one). Yeah GOG needs to fix those download pages either way,

Michonne is as you said.
Reigns has 64 and 32 bit versions , but the 32 bit version is present as both Part 2 and as a separate download (both with same name but different size / md5) , whereas Part 1 is the 64 bit version.
Unfortunately I have no idea which of the "duplicates" is the correct one. While GOG Galaxy exists it doesn't matter because it will repair the wrong one automatically by updating it. So yeah you only need 1 complete set of them but I have no idea which one that should be.
I noticed the same problem with the "new" release of Sudden Strike 3: Arms for Victory, ie. it had two installer files (1 or 2, 2 of 2) which had identical filenames, causing gogrepo to redownload the installer whenever you run the gogrepo download command. I reported it to GOG support, and they fixed it within a day (now there is only one installer file).
I can't check whether those aforementioned games are fixed too as I don't have them.
Post edited August 01, 2017 by timppu

Kalanyr
Wyrdling
Registered: Nov 2010
From Australia
Posted August 01, 2017

Be aware though that lots of classic installers have been updated recently as well. 1½ weeks ago I redownloaded around 270GB of installer files (most were those updated ones, but I had also bought some big new games), but this week there hasn't been similar flood of updated installers (I got only a few updated ones last week).
I just updated/downloaded my collection yesterday, went fine. I noticed though there was the same "two files with the same filename" problem as people have reported in this thread before with Sudden Strike 3, but today it was fixed. The glitch caused gogrepo to repeatedly redownload the said installer whenever one runs gogrepo download (only once per command).
Which reminds me:

Michonne is as you said.
Reigns has 64 and 32 bit versions , but the 32 bit version is present as both Part 2 and as a separate download (both with same name but different size / md5) , whereas Part 1 is the 64 bit version.
Unfortunately I have no idea which of the "duplicates" is the correct one. While GOG Galaxy exists it doesn't matter because it will repair the wrong one automatically by updating it. So yeah you only need 1 complete set of them but I have no idea which one that should be.

I noticed the same problem with the "new" release of Sudden Strike 3: Arms for Victory, ie. it had two installer files (1 or 2, 2 of 2) which had identical filenames, causing gogrepo to redownload the installer whenever you run the gogrepo download command. I reported it to GOG support, and they fixed it within a day (now there is only one installer file).
I can't check whether those aforementioned games are fixed too as I don't have them.

African_wildlife
Overdose — the best Painkiller
Registered: Feb 2014
From Finland
Posted August 18, 2017
When movie support is added, I hope that excluding films is easy. I have a couple of (free) 10 GB films on my account and I'd prefer not to have them on my hard drive, in addition to not straining GOG servers.
Post edited August 18, 2017 by African_wildlife

solar_dome
Linux user
Registered: Nov 2008
From United States
Posted August 19, 2017
I tried to see if I could login with gogrepo if my cookies file were gone.
I disabled two-step authentication in my gog.com account.
I am getting this message now in gogrepo:
"cannot continue, gog is asking for a reCAPTCHA :( try again in a few minutes."
EDIT: ok, I waited roughly 15 minutes. I can now login with gogrepo.
I disabled two-step authentication in my gog.com account.
I am getting this message now in gogrepo:
"cannot continue, gog is asking for a reCAPTCHA :( try again in a few minutes."
EDIT: ok, I waited roughly 15 minutes. I can now login with gogrepo.
Post edited August 19, 2017 by solar_dome

solar_dome
Linux user
Registered: Nov 2008
From United States
Posted August 19, 2017
I am now getting these errors:
python3 gogrepo.py update -os windows linux mac -lang en
23:35:53 | loading local manifest...
23:35:53 | fetching game product data (page 1)...
23:35:54 | fetching game product data (page 2 / 2)...
23:35:54 | found 150 games !!!
23:35:54 | ( 1 / 150) fetching game details for 1nsane...
23:36:05 | error
Traceback (most recent call last):
File "gogrepo.py", line 928, in cmd_update
item.downloads = deDuplicateList(item.downloads,{})
File "gogrepo.py", line 454, in deDuplicateList
dummy_item = copy.copy(update_item)
File "/usr/lib/python3.4/copy.py", line 97, in copy
rv = reductor(2)
File "gogrepo.py", line 199, in __getattr__
return self[key]
KeyError: '__getstate__'
If I let it continue, possibly more of the same.
python3 gogrepo.py update -os windows linux mac -lang en
23:35:53 | loading local manifest...
23:35:53 | fetching game product data (page 1)...
23:35:54 | fetching game product data (page 2 / 2)...
23:35:54 | found 150 games !!!
23:35:54 | ( 1 / 150) fetching game details for 1nsane...
23:36:05 | error
Traceback (most recent call last):
File "gogrepo.py", line 928, in cmd_update
item.downloads = deDuplicateList(item.downloads,{})
File "gogrepo.py", line 454, in deDuplicateList
dummy_item = copy.copy(update_item)
File "/usr/lib/python3.4/copy.py", line 97, in copy
rv = reductor(2)
File "gogrepo.py", line 199, in __getattr__
return self[key]
KeyError: '__getstate__'
Post edited August 19, 2017 by solar_dome

timppu
Favorite race: Formula__One
Registered: Jun 2011
From Finland
Posted August 19, 2017
Just a shot in the dark: do you already have an old manifest file? Since you are apparently trying to get all file details all over again (not using -skipknown or -updateonly options), delete any old manifest file.
I always redownload the whole manifest file all over again just so that I am guaranteed to catch any changes, so I always also delete the old manifest file before running gogrepo update.
Also, are you running the original gogrepo, or one of the recent forks by different authors? For now I am just using the vanilla gogrepo.
I always redownload the whole manifest file all over again just so that I am guaranteed to catch any changes, so I always also delete the old manifest file before running gogrepo update.
Also, are you running the original gogrepo, or one of the recent forks by different authors? For now I am just using the vanilla gogrepo.
Post edited August 19, 2017 by timppu

solar_dome
Linux user
Registered: Nov 2008
From United States