It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
Thank you. I found the items:

fallout_4_game_of_the_year_edition/setup_fallout_4_game_of_the_year_edition_1.10.163.0_(64bit)_(66700)-1.bin
fallout_4_game_of_the_year_edition/setup_fallout_4_game_of_the_year_edition_1.10.163.0_(64bit)_(66700)-4.bin
grim_dawn/patch_grim_dawn_1.1.9.7_(60763)_to_1.1.9.8_(65199).exe
the_night_of_the_rabbit/setup_the_night_of_the_rabbit_1.2.3.0389_(18473)-1.bin

Now just need to play around with your suggestion to resolve it.

Thanks
avatar
mrigolli: Thanks how can I tell which items are failing? Currently logging in remotely to my NAS and I do not see to get any fail message beyond the summary of MD5 mismatch. (Since you have posted I have marginally updated my previous message)

Thanks
avatar
mrkgnao: That's weird.

For every MD5 mismatch you should receive an info message reading:
'mismatched md5 for ...'
unless the MD5 failures are historic and gogrepoc is not retrying them since nothing has changed (in which case you might want to check past logs).

If you do not see these messages, you could try:
- run verify with "-clean" which cleans (moves to !orphaned) failing files, see what gets moved, then update those games
- run verify with "-delete" which deletes failing files, then redownload
- run verify with "-forceverify" which reverifies everything, but that could be a very long process, depending on how many games you have

Otherwise, I guess you will have to wait for Kalanyr to assist you.

EDIT: ninja'd
Just as a note, GOGrepoc doesn't "remember" failures* only successes, so any reported MD5 error is with the current files ( with the debatable exception of the thing where files got updated in between the manifest update and the download without a name change as an exception, that could be viewed as a "remembering" error I suppose )

*except in the sense of removing any previous "success" marker.
avatar
mrkgnao: That's weird.

For every MD5 mismatch you should receive an info message reading:
'mismatched md5 for ...'
unless the MD5 failures are historic and gogrepoc is not retrying them since nothing has changed (in which case you might want to check past logs).

If you do not see these messages, you could try:
- run verify with "-clean" which cleans (moves to !orphaned) failing files, see what gets moved, then update those games
- run verify with "-delete" which deletes failing files, then redownload
- run verify with "-forceverify" which reverifies everything, but that could be a very long process, depending on how many games you have

Otherwise, I guess you will have to wait for Kalanyr to assist you.

EDIT: ninja'd
avatar
Kalanyr: Just as a note, GOGrepoc doesn't "remember" failures* only successes, so any reported MD5 error is with the current files ( with the debatable exception of the thing where files got updated in between the manifest update and the download without a name change as an exception, that could be viewed as a "remembering" error I suppose )

*except in the sense of removing any previous "success" marker.
That's what I thought. So why didn't mrigolli see the "MD5 mismatch" messages on screen alongside the summary he has posted?
I suspect I did not see the error message as currently I am logging in remotely, with limited history for the bash shell, and over 2500 items. (So I had to cat the log files)

I am happy to report that verify -clean and re-downloading has resolved all issues.

Thank you for your help diagnosis the issue and resolving it !
avatar
mrigolli: I suspect I did not see the error message as currently I am logging in remotely, with limited history for the bash shell, and over 2500 items. (So I had to cat the log files)

I am happy to report that verify -clean and re-downloading has resolved all issues.

Thank you for your help diagnosis the issue and resolving it !
All's well that ends well.

And we have all learned a bit more about gogrepoc, which is an added bonus.
Post edited October 10, 2023 by mrkgnao
On the topic of the 30 MB limit for the log files, I'll probably increase that a bit on a future update. Games have gotten larger generally and for linux in particular the GOG servers are slower and the entire game is often one huge file, so 30 MB isn't necessarily enough for a standard full update / download / verify loop anymore. I'll play around with the values a bit to see what's a good number but I anticipate somewhere between 90 MB and 300 MB per log file (and 11 x for the 11 files in total )
Oh god, they've included the entire 1.63 version of Cyberpunk as an extra which means it's downloading at a snail's pace and it's a 100 GB zip file. Why ?
avatar
Kalanyr: ... it's a 100 GB zip file...?
Wow, that's a good one :D Who cares for corrupted downloads anyway. Since I still have the old version of the installer, I'll make sure to add an entry in the file exclude filter.

At least they left some way to download the old version, there's some people complaining about v2.0 (as usual)
avatar
Kalanyr: Oh god, they've included the entire 1.63 version of Cyberpunk as an extra which means it's downloading at a snail's pace and it's a 100 GB zip file. Why ?
Is that slow speed something related to gogrepo? I just tried downloading the zip file with the Firefox browser at work, and I was getting a healthy 42 MBytes/s speed, quite fast (estimated download time at around 43 minutes or so if that speed would maintain; I cancelled the download).

EDIT: Hmm you are in Australia, and I've seen quite many Aussies complaining about slow download speeds from GOG in general... Some have allegedly fixed it by using US VPN...
Post edited October 10, 2023 by timppu
avatar
Kalanyr: Oh god, they've included the entire 1.63 version of Cyberpunk as an extra which means it's downloading at a snail's pace and it's a 100 GB zip file. Why ?
Some of the patches are ridiculously big too.

I started trimming my patches away (on a game per game basis, because 32 bits installers and some newest updates are only found in patches so you need to look really carefully) and I estimate I freed about 500GB so far and I still have ways to go (I trimmed the patches for about 1/3 of the list so far).

The takeaway from all this for me is that they tried to shoehorn evolving needs into the existing format without updating its structure.
Post edited October 10, 2023 by Magnitus
avatar
Magnitus: Some of the patches are ridiculously big too.
That's a whole different story. That depends on the games' file structure.
The patches contain the diffferent files from one version to the next. Patches made by an SVN on file base do not change existing files but replace them with the altered ones. This way it's possible to keep a complete history of all versions. But a sometimes even with small changes, many game files or - if it's organized that way - one big archive get updated. In that case the patch becomes quite big. For many games it does not pay off to patch at all, Dying Light being an example. Every patch is almost as big as the entire game.

But patches are still distributed in 4GB chunks. The problem with Cyberpunk 2077 is, that it's one huge file of 100GB with a much higher chance to fail once in a while.
Post edited October 10, 2023 by neumi5694
avatar
neumi5694: That's a whole different story. That depends on the games' file structure.
The patches contain the diffferent files from one version to the next. Patches made by an SVN on file base do not change existing files but replace them with the altered ones. This way it's possible to keep a complete history of all versions. But a sometimes even with small changes, many game files or - if it's organized that way - one big archive get updated. In that case the patch becomes quite big. For many games it does not pay off to patch at all, Dying Light being an example. Every patch is almost as big as the entire game.

But patches are still distributed in 4GB chunks. The problem with Cyberpunk 2077 is, that it's one huge file of 100GB with a much higher chance to fail once in a while.
Well, there are storage considerations too. I already have ~12TBs of games here just for Windows and Linux installers in English, French, Japanese and Spanish.

Once you factor in redundancy, add at least 33% to that size if you use erasure coding, or straight up double it if you just duplicate.

And this is for offline storage. Once GOG is gone (hopefully after I'm dead, but I like to keep things real), I'll need cloud storage too to have an offsite backup and at my current size, that's a good 50$-60$ per month for the cheapest dependable online offerings I can find.

Truly redundant (both in terms of physical devices and location) dependable storage ain't cheap.

If I kept a backup of all previous installer versions, I'd have to at least double the size of my collection if not more (some games get updated a lot and not all of them are small).

So yeah, size matters and not just when you are downloading.

Now, if GOG gave me the guarantee that patches only serve to patch previous versions of the installers that are not the latest, I could just delete all my patches without thinking about it, except that they don't.

They jam in 32 bits installers in there (ok, arguably I could probably delete those nowadays), other special version installers and latest patches that never make their way into the main installer (including one for Witcher 3 for the latest French language patch). So I need to review patches on a per-game basis very carefully to figure out what I can delete.
Post edited October 10, 2023 by Magnitus
For the logs, I have gone with 180 MB per log, and reduced it to 9 rotating backups (so 10 total files). 120 MB was probably enough for now, but leaving some padding for the future this time. Overall this means you should need about an extra 1.5 GB of extra storage for the log files after that update happens (needs some local testing first), that's a pretty small amount compared to the kind of collection you need to go through that many logs quickly but I figure I'll give people a headsup in case they are running close to the edge on storage space.
avatar
Kalanyr: For the logs, I have gone with 180 MB per log, and reduced it to 9 rotating backups (so 10 total files). 120 MB was probably enough for now, but leaving some padding for the future this time. Overall this means you should need about an extra 1.5 GB of extra storage for the log files after that update happens (needs some local testing first), that's a pretty small amount compared to the kind of collection you need to go through that many logs quickly but I figure I'll give people a headsup in case they are running close to the edge on storage space.
You could probably reduce what gets logged (or make it an option). I don't think it's entirely useful to have so many updates of the download progress for example.
avatar
Kalanyr: For the logs, I have gone with 180 MB per log, and reduced it to 9 rotating backups (so 10 total files). 120 MB was probably enough for now, but leaving some padding for the future this time. Overall this means you should need about an extra 1.5 GB of extra storage for the log files after that update happens (needs some local testing first), that's a pretty small amount compared to the kind of collection you need to go through that many logs quickly but I figure I'll give people a headsup in case they are running close to the edge on storage space.
avatar
nightstrike_gog: You could probably reduce what gets logged (or make it an option). I don't think it's entirely useful to have so many updates of the download progress for example.
It's extremely useful for the console output because it lets people know that stuff is happening but it's likely true that the frequency could be cut way down for the logs.