It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
Community
General discussionOffline Game Installers - why use C:\Temp?(54 posts)(54 posts)
(54 posts)
Pages:
1
2
3
4
This is my favourite topic
Posted November 04, 2021
high rated
I'm not sure if people new to the thread skipped over coffeecup's posts here but the reason why the newer GOG installers cannot extract directly to the game install folder is the Galaxification of them. Instead of files being store as files with real file-names (like in a zip file), they are stored as a Galaxy "stream" with obfuscated names based on MD5 checksums. These then need reassembling into proper files via an included "manifest". So even if the tmp folder is on the same partition as the destination game folder, and even if you have 128GB RAM, it's possible that it will still need to write everything twice (once from installer to a mess of bits in tmp, then a second time to actually reassemble that mess of bits into files in the destination folder).

Read also immi's (InnoExtract author) posts on the other previous thread for more background information.
Posted November 04, 2021
avatar
AB2012: Instead of files being store as files with real file-names (like in a zip file), they are stored as a Galaxy "stream" with obfuscated names based on MD5 checksums. These then need reassembling into proper files via an included "manifest".
Does this offer some real tangible advantage? Being faster? Better compressed? Acting as a diff for identical parts of a previously installed version so you can skip files?
Posted November 04, 2021
avatar
rtcvb32: Does this offer some real tangible advantage? Being faster? Better compressed? Acting as a diff for identical parts of a previously installed version so you can skip files?
It doesn't seem to offer users any advantages. Compression is no better, if anything I've found it slower vs some older installers (and certainly much slower than if you zipped up the game folder yourself then unzipped it or repacked it using the same InnoSetup GOG use as measured in post 17 here). GOG seem to be doing it possibly because it might make it easier to automate building them, but even then we still seem to have outdated offline installers for a number of games.
Post edited November 04, 2021 by AB2012
Posted November 04, 2021
low rated
avatar
rtcvb32: It may be standard to a degree. But using a tmp directory is pointless if you have enough ram to do the job. If you were unpacking an 8Gb game and had 2Gb ram, sure. But if you are unpacking an 8Gb game and have 32Gb, unpacking to a temp directory may mean nothing. Worse is if the installation is to a different drive than the temp is located, then it unpacks, then has to copy. An awful lot of extra work.

On the other hand if more memory becomes common, people will start using ramdrives for temp directories and the problem goes away.... mostly...
avatar
Orkhepaj: sounds like a garbage outdated installer if that is the case
avatar
Magmarock: This is done during the extracting process of larger games. It's part of how inno works. you can change the temp directory in your Windows settings.
avatar
Orkhepaj: hah why do they use temp anyway when they have access to the final folder and could just do their job there?:O
You'd have ot ask the auther of inno, but it's pretty common. Winrar does the same thing.
Posted November 04, 2021
avatar
Orkhepaj: sounds like a garbage outdated installer if that is the case

hah why do they use temp anyway when they have access to the final folder and could just do their job there?:O
avatar
Magmarock: You'd have to ask the author of inno, but it's pretty common. Winrar does the same thing.
Hmmm... I'm having trouble seeing how it's useful.

If you look at GIT (a version management program by Linus) files are hashed using SHA-1 i believe, and it adds several protections among other things. But if the files in inno uses MD5 while similar it doesn't look like it's meant to be used with GIT for patches or other things.

I don't know. I'd prefer if it was just simple. Though the advantage of the 'blender' effect is that detection of file(s) to be copyrighted works by things like Google Drive and others is broken up. Seen something similar of breaking a game into 100Mb chunks of individual rar files which then is packed in a single rar, makes hashing and looking for said game blocks a bit harder i suppose, or maybe people have tons of zip drives they backup their games with...
Posted November 04, 2021
avatar
AB2012: I'm not sure if people new to the thread skipped over coffeecup's posts here but the reason why the newer GOG installers cannot extract directly to the game install folder is the Galaxification of them. Instead of files being store as files with real file-names (like in a zip file), they are stored as a Galaxy "stream" with obfuscated names based on MD5 checksums. These then need reassembling into proper files via an included "manifest". So even if the tmp folder is on the same partition as the destination game folder, and even if you have 128GB RAM, it's possible that it will still need to write everything twice (once from installer to a mess of bits in tmp, then a second time to actually reassemble that mess of bits into files in the destination folder).

Read also immi's (InnoExtract author) posts on the other previous thread for more background information.
To be perfectly honest I find it all rather confusing, so I don't know what to believe.

For instance, I can test the Offline Installers with InnoExtract, and it shows file names and MD5 values. I've not actually looked into what it does during testing. Does it extract each file to a temp location (or RAM) and then compare MD5, then delete the extracted file?

Or maybe I've just not come across a newer type of GOG installer file.

And I've never tried extracting using InnoExtract ... maybe I should.
Posted November 04, 2021
avatar
AB2012: Instead of files being store as files with real file-names (like in a zip file), they are stored as a Galaxy "stream" with obfuscated names based on MD5 checksums. These then need reassembling into proper files via an included "manifest".
avatar
rtcvb32: Does this offer some real tangible advantage? Being faster? Better compressed? Acting as a diff for identical parts of a previously installed version so you can skip files?
I always assumed the benefit is that they can (probably) effortlessly and automatically create the offline installers from the same "Galaxy streams" (or whatever they are called) that Galaxy users are already using.

In theory that should help offline installers stay at the same version as the Galaxy versions of games, without having to use processor time constantly to repack Galaxy streams into more basic offline installers. That last part (transforming the Galaxy streams into the actual game installation) will be performed by the end users who install their games with those offline installers.

I don't know if that is the reason and true in the first place, but to me that sounded the most logical and plausible explanation why the new offline installers are the way they are.
Post edited November 04, 2021 by timppu
Posted November 04, 2021
avatar
Timboli: To be perfectly honest I find it all rather confusing, so I don't know what to believe. For instance, I can test the Offline Installers with InnoExtract, and it shows file names and MD5 values. I've not actually looked into what it does during testing. Does it extract each file to a temp location (or RAM) and then compare MD5, then delete the extracted file?
InnoExtract merely extracts the files from the GOG installer. Older GOG installers stored them as files, newer ones store them as Galaxy chunks. The newer versions of InnoExtract that are aware of this automatically reconstruct those into proper files by default. However if you use InnoExtract with the switch --no-gog-galaxy, you'll get to see what the GOG installer really contains which resembles what Adamhm posted on the other thread, ie, a load of "tmp/53/4c/534c4f4b866e9544b57c638f388623f3" etc files that are gibberish until they are reconstructed from a manifest file GOG includes. I'm pretty sure this is part of the reason why newer installers need to unpack to tmp first before installing and why offline installers include files like goggame-xxxxx.hashdb, etc.
Posted November 05, 2021
avatar
AB2012: InnoExtract merely extracts the files from the GOG installer. Older GOG installers stored them as files, newer ones store them as Galaxy chunks. The newer versions of InnoExtract that are aware of this automatically reconstruct those into proper files by default. However if you use InnoExtract with the switch --no-gog-galaxy, you'll get to see what the GOG installer really contains which resembles what Adamhm posted on the other thread, ie, a load of "tmp/53/4c/534c4f4b866e9544b57c638f388623f3" etc files that are gibberish until they are reconstructed from a manifest file GOG includes. I'm pretty sure this is part of the reason why newer installers need to unpack to tmp first before installing and why offline installers include files like goggame-xxxxx.hashdb, etc.
Okay, thanks for that info.
I'll certainly check it out soon, and see whether it is possible to extract straight to the final destination game folder, using InnoExtract, and then run something(s) therein to finish the install.

If that can be done, then it should be quick and simple to knock up a helper program (maybe a floating drag & drop box) to do that for you ... or something CLI for those who would prefer that. One can hope anyway.
Pages:
1
2
3
4
This is my favourite topic
Community
General discussionOffline Game Installers - why use C:\Temp?(54 posts)(54 posts)
(54 posts)