It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
OK, so I want to make an offline backup of my GOG collection.

The first rough estimation: I may need ~1.7TB for ~800 games. I'm a new user of lgogdownloader. It's impressive how simple and convenient the script is, but still – I'd like to ask you for some good practices or your experience. I can see some problems, especially:

a) size of library and required storage
b) vol increased by extras, patches etc.
c) keeping win installers if you're linux user
d) keeping translated vs. original language
e) updating and adding new games

I'd be grateful for your help. If you maintain your offline backup, perhaps you can just answer some of these questions:

1. What is the size of your library? (number of games vs GB/TB)
2. What's your storage? (external drive? cloud?)
3. Do you download all games owned? (or some specific subset? how do you select?)
4. What do you download? (installers? Win and/or Linux? extras?)
5. How do you handle updates? (auto, specific command passed periodically? manually? how often?)
6. Do you use any additional management system for your game library? (spreadsheet with all games listed, csv list of games, online services, tags on GOG etc.)

Any other tips or insights? How do you reduce the volume and manual workload?
Regarding volume, from what I've discovered so far:
– a library size may be reduced by excluding demos, goodies' packs and some games you're completely not interested in (e.g. some freebies or games you don't like and you're not going to replay)
– it seems to be a good idea to exclude some extras, as I rather doubt I'll ever come back to use wallpapers, artbooks etc.
– the backup size is increased by saving both linux and windows installers (which I guess is still a preferred strategy for Linux users)
– the backup size is also increased by translation delivered as separate installers (as I prefer to keep also an original version of a game)

My pool of downloaded games is a bit unrepresentative for now, but after collectiong and analyzing full library I'll come back here with some interesting statistics on the size of individual components and estimates how assumptions mentioned above affects an overall backup size.
avatar
ciemnogrodzianin: 1. What is the size of your library? (number of games vs GB/TB)
2. What's your storage? (external drive? cloud?)
3. Do you download all games owned? (or some specific subset? how do you select?)
4. What do you download? (installers? Win and/or Linux? extras?)
5. How do you handle updates? (auto, specific command passed periodically? manually? how often?)
6. Do you use any additional management system for your game library? (spreadsheet with all games listed, csv list of games, online services, tags on GOG etc.)
1. Almost the same as yours. ~850 games / 1.6 TB (Yes, A LOT of oldies).
2. External drive.
3. Yes, all of them.
4. All main executables (english or multi-language) and the very few that have portuguese-only executables. Only Windows. All extras except redundant ones (Only english manuals or if a game has FLAC/MP3 soundtrack, I only keep the best quality files).
5. Manually. I keep track of the updates in a spreadsheet and download them when I'm in the mood. Sometimes I don't, like the SW: Battlefront update that removed 4:3 resolutions, I still keep an old version.
6. Only a spreasheet for all my games owned and another for the games I need to re-download (updates).
Post edited October 06, 2020 by Glaucos
1. What is the size of your library? (number of games vs GB/TB)
10k games, over 10gb

2. What's your storage? (external drive? cloud?)
See:

https://www.gog.com/forum/general/offline_backup_general_thread

3. Do you download all games owned? (or some specific subset? how do you select?)
Yes, every time I buy something I download immediately. I rely on nothing internet related.

4. What do you download? (installers? Win and/or Linux? extras?)
Everything. I do not rely on the internet to store anything, only supply.

5. How do you handle updates? (auto, specific command passed periodically? manually? how often?)
I check periodically, asses wether it’s worth downloading a new version or patch. Mostly no for old games it’s not worth downloading as they only add in their drm platform. For new games, once in a while.

6. Do you use any additional management system for your game library? (spreadsheet with all games listed, csv list of games, online services, tags on GOG etc.)
I have used various. Do not use excel, your data will corrupt at some point. I use offline collectorz game library just as a dB and front end, am working on my own version. Again, do not rely on the internet.

End of the day, I use the internet as a service, nothing else.
avatar
ciemnogrodzianin: 1. What is the size of your library? (number of games vs GB/TB)
2. What's your storage? (external drive? cloud?)
3. Do you download all games owned? (or some specific subset? how do you select?)
4. What do you download? (installers? Win and/or Linux? extras?)
5. How do you handle updates? (auto, specific command passed periodically? manually? how often?)
6. Do you use any additional management system for your game library? (spreadsheet with all games listed, csv list of games, online services, tags on GOG etc.)
1. ~830GB / 316 games (from GOG. I have another separate backup for non-GOG games)
2. External drive. 4TB, USB3. Another external drive that holds older version of the backup (I copy or mirror with Toucan).
3.-4. All games+patches, no extra, no other lang, only Windows.
5. Manually once a month (I feel like I have to be part of it sometimes); Single Windows batch file to update:
python gogrepoc.py update -lang en -os windows
python gogrepoc.py download -skipextras F:\gogrepo\
python gogrepoc.py clean F:\gogrepo\
python gogrepoc.py verify F:\gogrepo\

6. Hate anything to do with spreadsheet related. If I really need a list I can just do a directory list and pipe it out into a txt file or something.

I do plan to expand my backup to include Linux also, and move it all to Linux later on. If there is some extra I really want I download them separately / manually.
Post edited October 07, 2020 by sanscript
avatar
ciemnogrodzianin: 1. What is the size of your library? (number of games vs GB/TB)
2. What's your storage? (external drive? cloud?)
3. Do you download all games owned? (or some specific subset? how do you select?)
4. What do you download? (installers? Win and/or Linux? extras?)
5. How do you handle updates? (auto, specific command passed periodically? manually? how often?)
6. Do you use any additional management system for your game library? (spreadsheet with all games listed, csv list of games, online services, tags on GOG etc.)

Any other tips or insights? How do you reduce the volume and manual workload?
1. 127 titles, 299 GB (including a few non-GOG titles).
2. Internal HDD, backed up roughly monthly on external HDD... But I just did that this evening and hadn't done it in 4 months. Anyway, neither is dedicated to just games, though they take up the vast majority of the space, both being 500 GB HDDs, with the internal one having a separate 50 GB partition and the external one two 50 GB ones.
3. Yes, as soon as I get them, unless they're freebies I already finished or clearly abandoned (not the GOG version in those cases, obviously). Once I finish or clearly abandon a game, I delete the installer as well. Otherwise, so far kept all, but with those HDDs filling up, I'll soon most likely delete all those filed under not interested (already separated them).
4. Initially Windows installers and any applicable patches that may be relevant to me (no reason to download separate language support or multiplayer stuff) if the installer version isn't the latest one, and all extras. Then go through the extras and usually after a quick look delete stuff like artbooks, avatars, most wallpapers, nearly all soundtracks and some other odds and ends, and the handful of wallpapers and couple of soundtracks I kept are moved in the respective wallpapers and music folders, not kept with the games. What's left alongside the installers are usually manuals, maps and other informational stuff, plus in-game extras that are for some reason separate. In the few cases where things like novellas or comics are included, I keep those with the game until I read them (which may take a long time, not something I really keep in mind to do, unlike if they'd be "proper" books, so maybe it happens when I stumble into them again someday), then delete.
5. Manually, whenever I happen to check my library and notice an update flag or want to install a new game and check to see whether the installer version matches mine. If an update exists, I check to see whether it may be relevant and download if so. Won't download if it says (either in changelog or what I can find in the updates thread) no change in actual game files, or it's multiplayer only, or only Galaxy related or such things.
6. Nothing but tags, matched by folders locally. My tags on GOG are Installed, Started (these two are the only non-exclusive ones on GOG, and installed takes priority - locally an installed kit goes in the Installed directory, whether started or not, Started being just for games not reinstalled since I dropped them despite intent to continue), Backlog, Uncertain, Not Interested, Abandoned, Completed (last 3 being hidden on GOG, and as I said last 2 not existing locally as I don't keep those kits, and most likely soon enough Not Interested will vanish as well).
If you could tinker with the installer files you could get better compression by optimizing some files take it up less space. But as that's unlikely.

But it would take roughly 80 Blueray discs, or 220 dual layer DVD discs.

You could also do a new dedicated external, but depending on bitrot and whatnot....
Since the downloader was murdered, I use a browser to download the small games and Galaxy to download the big games. I take the latest Windows installer at the time and all the extras and bonuses. They get downloaded into one folder per game, which I copy on two external hard drives. It looks like over 100 games, 847 GB.

I would download the patches, but I always miss a few, so it's often necessary to download the complete new version anyway, which I rarely do. I don't think games should have so many updates all the time.

I used to do the same thing with game folders on Steam, with the appropriate crack in case something happens to my account or to Steam, but it got too big.
avatar
ciemnogrodzianin: 1. What is the size of your library? (number of games vs GB/TB)
2. What's your storage? (external drive? cloud?)
3. Do you download all games owned? (or some specific subset? how do you select?)
4. What do you download? (installers? Win and/or Linux? extras?)
5. How do you handle updates? (auto, specific command passed periodically? manually? how often?)
6. Do you use any additional management system for your game library? (spreadsheet with all games listed, csv list of games, online services, tags on GOG etc.)

Any other tips or insights? How do you reduce the volume and manual workload?
1. 1400+ games (I count unbundled games as one game, not multiple games --- GOG thinks I have 1650+ games), 3+ TB
2. Portable external HDD
3. All, but not demos
4. Windows English + extras, but not multiple copies of basically the same thing (e.g. only MP3, if game has both MP3 and FLAC soundtracks)
5. Manually, daily (i.e. as soon as a game changes). Until recently MaGog + DownThemAll, now looking into other options (gogrepo, lgogdownloader).
6. Until last week, MaGog. Starting this week, spreadsheet with data exported from MaGog.

A lesson: Two weeks ago, my external HDD died and I lost all my GOG backups. Since I had only one copy, it's all gone (including a dozen or so extras that GOG had removed over the years from our libraries, and thus cannot be recovered). I plan to buy a new HDD next week. Theoretically, I should learn from this and maintain two copies, but it's too much for me, so I'll risk it again.
Post edited October 07, 2020 by mrkgnao
avatar
ciemnogrodzianin: ...
I will not say what backup I have atm because I basically have none.
Instead I will write my take on how I plan on doing it and what I suggest to you all.

This will be in few parts as GOG has char and rate limiting.
-----
PART 1/2
-----

First of all, you should EXACTLY calculate how much it's going to take. NOT "roughly". Be exact or you will regret it sooner or later. Go and count every single file you are interested in backing up. Those 1-10 MiB files too.
Those small files (below 1 GiB, with median being probably below 50 MiB) alone were total of around 20 GiB for me.

I made fairly accurate (not 100%) draft estimate after seeing the "EGS GOG deal" thread.
(I spent "large amount of time" and manually calculated total by taking ALL files into consideration with their exact GOG reported sizes)
My results were:
I have 87 library positions right now.
If I would go and download EVERY single file (that assumes 1.ONLY English installers 2.Only latest collective versions) it would take around 647 GiB.
Out of those 87 I have PAID for 33 GAMES (around 36 or so positions in library).
Rest is demos, "goodie packs" (not to be confused with extras), and stuff obtained for free (purchase bonuses, stuff free on store, things from giveaways).

I use Linux as my primary system (for gaming too) and will do so eternally (well at least I will use *NIX derivative, because I may MAYBE consider BSD someday).
So that means that I care about A.Linux ports B.Windows ports (even if there is a Linux port available).

I never saw Mac as a viable gaming platform, I didn't ever own Mac and never will (for very good reasons, that company is BS) so ports for that platform are not in my interest AT ALL.
So removing Mac ports from total makes it slimmer by about 124 GiB. And should I even care about them then only about 24 GiB is for stuff that I actually paid for.

I have spent less than 140 eur on this platform in 5 years I am here.
It's not terribly a lot but I am pretty poor so I don't want to have this go to waste.
And I do own quite some games here that I don't own elsewhere.
So at the very least I need to backup the stuff I paid for.

The 600-ish GiB of data isn't terribly lot PER SE.
But it gets super inconvenient in the long run. Namely:
1.My internet is ancient and atrocious. If it works FINE I can download around 50 GiB per day IF I would do nothing but download for entire day. And that is IF it works. Upload speed is a complete joke and so Cloud storage is a big NO (I have top of like 0.4 Mbps upload. Also having 500+ GiB of Cloud storage is REALLY expensive with NO guarantee of it working in case of some "major world events").
2.We are not yet in "40 TiB drives popular on the market" situation so no matter how you look at it with time it means having to put up with a stash of drives instead of for example single one (or few at best, for safety and redundancy).
3.Platter based hard drives are a burden. They need replacement every few years. Even with enterprise class you are still going to have to throw them out of the window after 10 years max.
4.Ssd drives are a different burden. Suspectible to solar flares, degausing (most don't even come in metal cases, let alone have any shielding), memory cells demagnetising over time when not connected to power for many months. TLDR they need maintenance at least once a year even if totally unused. Plus they are still too expensive.
5.Flashdrives. Overpriced. Quality is generally worthless. Generally the worth they represent is temporary just like the files stores on them longterm. (it adds to the insult that close to nobody sells MLC ones nowadays)
6.Optical storage is prone to physical damage. So right off the bat it filters off all normal grade disks.
DVD can be obtained in "better" quality but it's still trash longetivity compared to BD.
7.The 100 GiB BD disks are not cost effective yet. 50 GiB ones are unjistifiably price gouged (btw pressing BD at factory costs basically same as DVD just so you know, so it's not like it costs manufacteur more). 128 GiB are close to unseen (and EVEN MORE price gouged) on the market. BD disks above 128 GiB (they DO EXIST) are not currently available on the market which adds to the insult (and alleged prototype 1 TiB BD disks would allegedly require different laser wave length so could not be used without replacing drive which adds to the cost).

Storing even as much as 1 TiB more data PERMANENTLY is troublesome these days.
We are progressing technology and storage but Cloud storage prices don't go down (because of some responsible people's greed).

So naturally it would be desired for me to reduce the size of this backup as much as possible.
Right off the bat I am "ASSUMING" that the files in question cannot be further compressed.
I don't know that. But I'd rather to prepare for "raw dump" and have more free space in case I manage to compress it instead of underestimating and getting into dissappointment and trouble.
IF I would be to compress the files I would likely TRY to use ZSTD (on custom settings).

So I started flexing the math and figured:
-"witcher goodie pack" is a whopping 45 GiB and I am not very interested in these games so I am likely to drop it
-the Mac ports drop 124 GiB off the total
-Linux ports (including EVERYTHING, not just the stuff I paid for) weight around 94 GiB
-all possible extras (including stuff I didn't pay for) weight around 32 GiB
-cyberpunk goodie pack weights almost 8 GiB
-free stuff (INCLUDING their mac ports) is around 302 GiB (that includes Shadow Warrior 2, Everspace and Witcher 1 among other things)
-demos (including their mac ports) are around 15 GiB, not a whole lot but not great (and it's not like I saved every possible demo to my library)

Removing mac ports, "cdpr marketing tr**h", free stuff and demos it goes down to around 254 GiB.
That is a TOTAL MINIMUM I have to backup.
I generally don't care about demos (and should I want them they would fit on 3 DVD) in this case. They are most useful as benchmarks if anything and since they "should" remain free this isn't much of a concern.
I want all extras for games I paid for. Period. I will not pick as that could lead to a regret years later.
Free stuff is not a direct financial loss so it's not a priority.
But there are some I do care about so there is unknown amount for that.
I haven't calculated yet but I would want to preserve things like Higurashi and Soma.
So REAL total would still fall below 400 GiB.

(I will add remaining part of post in like 10-15 m because of GOG char and rate limiting)
avatar
ciemnogrodzianin: ...
-----
PART 2/2
-----

Now, my plan:
my internet is a** so it will take "<2 weeks" total to just download it.
Then I will checksum everything (I am yet to figure if I can dig out checksums out of the API somewhere).
As for the storage. I am rather poor so I cannot get new drives any time soon.
I have a stash of "unused" (awaiting data transfer) old ones, namely:
-around 9 year old CCTV grade 2 TB Seagate
-unknown age 1 TB Samsung
-unknown age 500 GB Samsung

I can MAYBE squeeze less than 400 GiB into new Seagate Ironwolf (COLD STORAGE backup of system) for "prolonged temporary" store.
If so then this would be first copy (non used one, cold storage).

Then I would store "working copy" (to be accessed) on another Seagate Ironwolf that I use in daily system.

Now, the main cold storage copy is kind of a pain in my situation.
I don't want another drive as I would have to throw it into the sunset in few years. Not to mention they are super prone to mechanical damage when unused (it never happened to me but write heads can get stuck on platters when unused [even when properly parked] and that means game over).
So generally I am planning obtaining a batch of Japanese-made 25 GiB high quality BD disks. The Japanese-made ones specificly have high longetivity.
For the bare minimum that means 11 disks (around 23.3 GiB usable) for this gold master style backup copy.
The 11 disks would cost me around 10 euro total (including shipping) because I found somebody domestically who imports them (if I would be to import them myself it would cost A LOT more due to shipping).
The thing is, I would need realistically more than 11, I would prefer to buy like 30 at once (for a number of reasons). But I am in not so great financial situation now so I'm in a bit of a dilema.
I could suck it up (and be pretty unhappy about it) and buy those.
Or perhaps I could store one copy and HOPE no shit happens in few months before I would get into better financial situation.
I don't know. I need to think that through.

What I do know for sure is these:
1.I would put applicable receipt on each disk (so that NOBODY EVER tries to "accuse" me of anything; This is especially important when changing country of residence).
2.I would likely store files unencrypted on those disks. I would normally never do that for PRIVATE DATA backup. But for concerns resolving around long term password retention (we are talking about 10 years at least here, these disks would likely last even 20) and few other things I would likely store this SOFTWARE unencrypted.
3.I would store complete spreadsheet (NOT in exel, just open format, stored in multiple locations, including cloud btw) containing exact filenames and checksum values * for sha512 and sha256 (having two is better because possibility of two files with same even sha256 checksum is EXTREMELY low, but for 2 different checksum types it's close to IMPOSSIBLE for 2 different files to have same checksums for different checksum types).
Since I use Linux I can heavily automate checksumming so I could for example run batch check for entire disk at once.

* even the best backup is useless if you don't know if files are unchanged (even random neutrinos can lead to bit flips [on MAGNETIC storage], and there is no way on earth that would get corrected given it would be in cold storage)

Btw I would record the disks at minimal possible write speed fyi (to lower risk of a bad burn).
I would likely use the 23.3 GiB and not entire 25 GiB (it is possible to use full on Linux by bypassing some stuff as that "unusable" space is normally for error correction data but "can" be used with overrides).

A suggestion to everybody:
add 5-10% of total size to your estimate (different filesystems with different settings may result in file sizes differing between them).

As for the patches:
Almost all games I have here currently are older than few years so they are unlikely to get any.
But I would check maybe like every 2 weeks manually.
I would use GogDB as a whole lot of file changes info ISN'T shown in library at all (at least in GogDB I can tell there was a change).

Another tip:
When "adding" data to backups I suggest you lock partition with older data to read only and add new data on new partition (or use containers but I guess you cannot do that on Windows).
Oh, and when prepping drive make it so that there will be recovery GPT header (and better yet, you yourself backup GPT headers of your drives and store them separately safe, for example within password manager "add file" functionality, these headers can break from A LOT of things, especially on Windows).

Forgot to say:
my BD disks cost estimate doesn't include jewel cases (which are a must).
Spindle ("cake") cases are a big NO (high possibility of SOME disk damage due to friction, close to no physical protection, risk of damage when taking out single disk, questionable quality plastic [usually very low quality] leading to plasticiser possibly shorting disks longetivity).
Also, store your disks and drives protected from sun, heat and humidity.

So here is my short plan. Hope I didn't forget anything (I possibly did).
I wonder if anybody will even read this :P

L.
1. What is the size of your library? (number of games vs GB/TB)

I have the following computer games - no consoles - no handhelds - no mobile games:

007 Games in original big box/CD Jewel Case (floppy to CD) from 1986 to 1997
002 Games from Desura (Company Dissolved)
095 Games From Humble Bundle DRM-Free
085 Games on GOG
330 Games on itch.io
001 Game on 3Drealms
002 Games on BeamDog
008 Games from IndieBox
015 Games on Steam
002 Games on Stadia

547 Games
23 of them are duplicates due to bundles, remasters and free giveaways

524 Total

My wife and I share accounts for all of these except for Steam and Stadia.

2. What's your storage? (external drive? cloud?)

I use an internal/external 2 TB Hard drive that has all of my games along with configs and save files.
I also have a home server with 6 GB of storage for my wife and kids to have access to the games.
I've also transferred of few of these games into USB drives for when I do LAN events.

a) size of library and required storage

My library is small but at one point, I use to save all the mods, models and maps I've found for games like Doom (1993), UT99 and Quake. I stopped doing that about 5 years ago and only kept the ones that my friends and family always play.

3. Do you download all games owned? (or some specific subset? how do you select?)

I've back up all of my games at least 1x a year. Most of the games I play are no longer getting updates or patches. Those that do I back up to my local, NAS and cloud storage using Rsync it's auto the minute I get the game.

The games that I always have at the ready are my fighting, brawlers and local-coop games because that's what my family mostly plays.

4. What do you download? (installers? Win and/or Linux? extras?)

I'm a Linux user. So I download the Linux binaries. I have a very small amount of Windows games that never got a Linux binary so I download those also. If the manuals have lore, I download those also.

b) vol increased by extras, patches etc.

I keep a few versions of the games until it gets to be 10+ then I just remove them and keep the last 3.

5. How do you handle updates? (auto, specific command passed periodically? manually? how often?)

The only client I use is Steam so it handles those updates but I do back up those DRM FREE titles from there. Currently, I own 2 games on Steam that do not work without Steam and Steamplay for me. So those do not get backed up at all. My itch.io library is large but that was from 2 bundles. I currently play 3 games from them and when those get updated I download them manually and they get Rsync;ed.

I use some scripts I have via Rsync to back up the games once I manually download them to a pre-selected folder. Lately the only game that I have that gets updated a lot of 'Jupiter Hell' the rest have not been updating in months or years.

6. Do you use any additional management system for your game library? (spreadsheet with all games listed, csv list of games, online services, tags on GOG etc.)

I have a lot of adult children and we often buy games for each other. So we have a shared Google Sheet with what we own and what we are looking for. My wife and I have completed our collection but If I see any bigbox games that my kids want when I go shopping, I check that list to see if they got it yet or not.

c) keeping win installers if you're linux user

I have 10 games in my entire collection that do not have Linux binaries. I keep those.

Ultimate Body Blows (Amiga CD 32), Virtua Fighter PC, Virtua Fighter 2 PC, Fatal Fury 3 PC, The King of Fighters 98 Ultimate Final Edition, The King of Fighters 2002 Unlimited Match, The King of Fighters XIII Galaxy Edition, Garou: Mark of the Wolves, Street Fighter Alpha PC and Street Fighter Alpha 2 PC

d) keeping translated vs. original language

I have one game in Japanese the rest are in English. I keep as is.

e) updating and adding new games

I have a few games on my wishlist here but I'm not looking for any new games anytime soon.
Post edited October 16, 2020 by Yucaju
avatar
ciemnogrodzianin: 1. What is the size of your library? (number of games vs GB/TB)
2. What's your storage? (external drive? cloud?)
3. Do you download all games owned? (or some specific subset? how do you select?)
4. What do you download? (installers? Win and/or Linux? extras?)
5. How do you handle updates? (auto, specific command passed periodically? manually? how often?)
6. Do you use any additional management system for your game library? (spreadsheet with all games listed, csv list of games, online services, tags on GOG etc.)
1. 1987 games = around 5TB or little over at this point (English Windows versions only)

2 Since it recently went close or over 5TB, now it is divided into two (5TB + 2TB) external 2.5" USB HDDs.
Games starting with #-S go to the 5TB drive, and games starting with T-Z go to the 2TB drive.

3. All.

4. Installers + extras, Windows + English only at this point. I might want to download Linux versions as well, but it seemed at some point that they are quite often just WINE + Windows version anyway, so why download twice...? If I want to play the game in Linux, I guess I will try how the Windows version works with WINE. For instance with Two Worlds, I actually had better success getting the Windows version running in Linux WINE, than running the "Linux version" which apparently came with its own WINE IIRC.

5. Maybe once a month or two months I manually run the gogrepoc.py tool which downloads any new or changed games/files in my library, and moves old obsolete files to trashbin folder. If I want to install and play some GOG game in the meantime that I haven't downloaded yet, I just download it separately with a web browser.

6. Not really. gogrepoc.py does good enough job to put game installers in subfolders so it is easy to see with File Explorer what games I have and find the game I want to install. I don't do any "re-organizing" of the downloaded files either, besides keeping some of the "obsolete" files I feel are worth keeping, like the older non-ScummVM versions of Myst 2: Riven.

I would like to exclude all patches and non-Windows/English versions from the extras to reduce the archive size, but currently that is not possible because
a) There are some cases where the main installer is not the newest version, but you do need a newer patch on top of that.
b) In some cases the "update patch" is basically a DLC, new content.

So I can't just blindly exclude downloading all "patch_*" files, unfortunately.

Also excluding all extras is not an option as with many games there is some alternative/classic/original version of the game available in the extras that I want to keep.

I guess I will just keep downloading all extras, and then sometime (far?) in the future if GOG is not providing updates anymore and my GOG game archive is "frozen", I will go through the games with time and manually remove files that are unneeded. Right now such weeding out is useless as the removed files would just be redownloaded the next time I run gogrepoc.py.
Post edited October 07, 2020 by timppu
1. 1976 games about 4.5TB
2. Two 10TB HDDs in RAID1 (mirror) in a NAS.
3. Yes. I hide only demos and such.
4. Windows, english only, extras, skipping patches.
5. I've set up gogrepoc to automatically update everything once every two weeks.
6. I have a spreadsheet with all games to have an overview, but I don't buy anywhere else anymore.

avatar
timppu: I would like to exclude all patches and non-Windows/English versions from the extras to reduce the archive size, but currently that is not possible because
a) There are some cases where the main installer is not the newest version, but you do need a newer patch on top of that.
b) In some cases the "update patch" is basically a DLC, new content.

So I can't just blindly exclude downloading all "patch_*" files, unfortunately.
I think there are only very rare the cases where there is no installer that contains the patch also. So far I think I have found one (Trine). Maybe we could pester GOG update the standalone instances for those rare cases properly.
Post edited October 07, 2020 by blotunga
avatar
ciemnogrodzianin: 1. What is the size of your library? (number of games vs GB/TB)
Currently out of 6 TB drive it takes about 5 TB, explanation for size follows.
avatar
ciemnogrodzianin: 2. What's your storage? (external drive? cloud?)
External USB hard drive.
avatar
ciemnogrodzianin: 3. Do you download all games owned? (or some specific subset? how do you select?)
Yes. I see no reason not to download everything. I suppose if it ever came to choosing, I would leave out some free games that I never actively wanted, but that's a bit foolish thing to do. You never know if you want to play something completely different in the future.
avatar
ciemnogrodzianin: 4. What do you download? (installers? Win and/or Linux? extras?)
Absolutely everything.
All installers, all OS's, all language versions, all extras, and I even save the store page locally.

I keep installers, but in addition them, also the game folders as the installer unpacks them. Not for every version obviously, but for the versions that I play.

Again, you never know what you might need in the future. You can always delete something that is not needed, but with storage space being relatively cheap, I don't see that being necessary.
avatar
ciemnogrodzianin: 5. How do you handle updates? (auto, specific command passed periodically? manually? how often?)
Manually, every now and then.
I don't need to have every version released, so I can check updates for previously downloaded games after a few years, I recently downloaded updated versions for some games that I had stored five years ago. To me, it's more important to have some version of the game, having the latest update is rarely needed, especially with games that use ScummVM or DOSbox.

When downloading updates, I decide whether to keep or delete older versions. Usually I delete them, but in some rare cases I also keep the older ones, like in some cases where the older downloads come with DOSbox and later ones with ScummVM, I want to keep the DOSbox version to have the game exe files.
avatar
ciemnogrodzianin: 6. Do you use any additional management system for your game library? (spreadsheet with all games listed, csv list of games, online services, tags on GOG etc.)
I use an Excel file, where I have one spreadsheet as the mainlist, and I have genre-specific spreadsheets which are linked from the mainlist. In theory, if I want to know how many let's say racing games I have on GOG, I can go to racing games spreadsheet and filter additional information column for "GOG" and there is the answer.
avatar
ciemnogrodzianin: Any other tips or insights? How do you reduce the volume and manual workload?
To quote Sierra: "save early, save often".

It's much easier to download at least some version of the game as soon as you get it, rather than trying to download everything in one massive download.

I see no real reason to reduce volume, but if you must, don't download languages that you don't speak. Most probably that French Mac version is never needed, so you can skip it. But in case you somehow happen to get involved with a Mac and some French speaking person, it would be nice to have that just in case.

Manual workload can be reduced by using download tools, but unless you want to have every version immediately, there's really no reason to use those. Especially if you use your own DOSbox/ScummVM/other third party tools, you only need some game files and manage all settings yourself, all those cloud save/Wine repack/etc. updates are unnecessary.