Posted October 12, 2020
[Update:] Issue is solved.
Original post:
Hi!
I noticed with several games, using the GOG installer for that game, mostly Version 2 of the installer, multi-part, the file integrity check is extremely slow. To give an example: I just downloaded ELEX. It's got 7 .bin files, 26.84 GB in total, and with a <2 MB/s (average of about 1.7 MB/s) line this took me > 4 hours. The integrity check runs now for about 3 hours and it's ~75% through. How is this possible?!?
It would have been easier to include a md5sums or a sha1sums file, so I could at least use this external tool to check the files' integrity. And I am certain that it will be way faster.
My system is a Ryzen 1800X, 8 cores running at 3.7 GHz.
This shouldn't take sooo long!
As for my reasons to do the file integrity check: I archive the downloaded installers. If they were corrupted I wouldn't want to archive them, so I have to check them somehow. I only know of the installer method for this.
Quintessential this is my request:
1. Provide means to check if the downloaded files are not corrupted.
2. Without the need to actually install the game. (Currently not possible. Whishlist.)
3. If the installer is used, enhance the performance significantly! Please! (It wasn't the installer. Issue fixed.)
Thanks.
[Edit]
I checked the Task Manager stats: "Setup/Uninstall (32 Bit) (2)" has two sub-processes: 1. ELEX Setup and 2. Setup. The main task is using 0.1% CPU and reading about 1.3-3.5 MB/s (so an average somewhat >2 MB/s, maybe) from the HDD. I wonder what's the limiting factor. It's not the CPU, and it's also not the HDD - it can be read much much faster (USB 3.x external drive).
[Edit #2]
Okay, installation has about the same stats and the same issue: it takes ages. In Task Manager I get the same ~0.3% CPU and <4 MB/s read from the drive. Installation will take approximately the same >4 hrs. It might be a different issue then. I have to check what's wrong. Playing games isn't at all an issue (but not from the external USB 3.x drive). Copying files around to/from the USB drive is also faster. I don't get it.
[Edit #3]
Just for the confirmation: I used robocopy on two USB drives, and I reach >95 <100 MB/s transfer rates. While that was running I used Explorer to copy the GOG Installer for No Man's Sky 3.01 64352 from the USB 3.x external HDD to the internal NVMe SSD and, while both copy operations are running simultaneously (robocopy and Explorer), I still get > 25 MB/s (while not once below 10 MB/s) displayed from the Explorer copy dialog.
Big surprise: I run the installer for NMS from the NVMe and it's way faster. It has ~6.5% CPU utilization and read rates >100 MB/s (which was expected from an NVMe). (As a sidenote: I wanted to start the integrity check and, once I see how fast or slow it works, wanted to close/cancel the installation. First, there is no cancel button. Second, the X to close the Installation window was completely ignored. So I installed 3.01 over 3.03. No big problem though, as the two patches were easily restored as well....)
[Solution:]
So now I'm copying the ELEX installer over, to see the results. And here the biggest surprise. The external HDD is performing very poorly when copying specifically the ELEX installer. It's a 4 TB drive filled to > 95 % (only 200 GB free) and there you go. It's not the installer. It's the drive. But not at every random position, only at those last GBs.
Weird. But not the Installer to blame either, so I apologize for the wrong accusation. :-)
robocopy also only got ~4 MB/s copying the ELEX installer files to the other external HDD. So, long story, short, I reinstalled from the second USB 3.x HDD and it reaches the ~100 MB/s from it, as it's supposed to, the integrity check is now way faster, and the limiting factor is only the read performance (CPU at around 2% or so).
Original post:
Hi!
I noticed with several games, using the GOG installer for that game, mostly Version 2 of the installer, multi-part, the file integrity check is extremely slow. To give an example: I just downloaded ELEX. It's got 7 .bin files, 26.84 GB in total, and with a <2 MB/s (average of about 1.7 MB/s) line this took me > 4 hours. The integrity check runs now for about 3 hours and it's ~75% through. How is this possible?!?
It would have been easier to include a md5sums or a sha1sums file, so I could at least use this external tool to check the files' integrity. And I am certain that it will be way faster.
My system is a Ryzen 1800X, 8 cores running at 3.7 GHz.
This shouldn't take sooo long!
As for my reasons to do the file integrity check: I archive the downloaded installers. If they were corrupted I wouldn't want to archive them, so I have to check them somehow. I only know of the installer method for this.
Quintessential this is my request:
1. Provide means to check if the downloaded files are not corrupted.
2. Without the need to actually install the game. (Currently not possible. Whishlist.)
3. If the installer is used, enhance the performance significantly! Please! (It wasn't the installer. Issue fixed.)
Thanks.
[Edit]
I checked the Task Manager stats: "Setup/Uninstall (32 Bit) (2)" has two sub-processes: 1. ELEX Setup and 2. Setup. The main task is using 0.1% CPU and reading about 1.3-3.5 MB/s (so an average somewhat >2 MB/s, maybe) from the HDD. I wonder what's the limiting factor. It's not the CPU, and it's also not the HDD - it can be read much much faster (USB 3.x external drive).
[Edit #2]
Okay, installation has about the same stats and the same issue: it takes ages. In Task Manager I get the same ~0.3% CPU and <4 MB/s read from the drive. Installation will take approximately the same >4 hrs. It might be a different issue then. I have to check what's wrong. Playing games isn't at all an issue (but not from the external USB 3.x drive). Copying files around to/from the USB drive is also faster. I don't get it.
[Edit #3]
Just for the confirmation: I used robocopy on two USB drives, and I reach >95 <100 MB/s transfer rates. While that was running I used Explorer to copy the GOG Installer for No Man's Sky 3.01 64352 from the USB 3.x external HDD to the internal NVMe SSD and, while both copy operations are running simultaneously (robocopy and Explorer), I still get > 25 MB/s (while not once below 10 MB/s) displayed from the Explorer copy dialog.
Big surprise: I run the installer for NMS from the NVMe and it's way faster. It has ~6.5% CPU utilization and read rates >100 MB/s (which was expected from an NVMe). (As a sidenote: I wanted to start the integrity check and, once I see how fast or slow it works, wanted to close/cancel the installation. First, there is no cancel button. Second, the X to close the Installation window was completely ignored. So I installed 3.01 over 3.03. No big problem though, as the two patches were easily restored as well....)
[Solution:]
So now I'm copying the ELEX installer over, to see the results. And here the biggest surprise. The external HDD is performing very poorly when copying specifically the ELEX installer. It's a 4 TB drive filled to > 95 % (only 200 GB free) and there you go. It's not the installer. It's the drive. But not at every random position, only at those last GBs.
Weird. But not the Installer to blame either, so I apologize for the wrong accusation. :-)
robocopy also only got ~4 MB/s copying the ELEX installer files to the other external HDD. So, long story, short, I reinstalled from the second USB 3.x HDD and it reaches the ~100 MB/s from it, as it's supposed to, the integrity check is now way faster, and the limiting factor is only the read performance (CPU at around 2% or so).
Post edited October 13, 2020 by Andtha
No posts in this topic were marked as the solution yet. If you can help, add your reply
