It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
This is wonderful, I have been manually keeping my offline collection up to date for years. I would love to throw this on a Linux server (or raspberry pi) as a cron job and never have to worry about manually updating again.

I few things spring to mind:

1. It would be highly desirable for me to be able to filter what content is downloaded. To save space, I tend to lean towards mp3 soundtracks over FLAC. I also choose not to download video bonus material such as developer diaries and such. And why bother downloading foreign manuals/bonus content if I'm only downloading english games already? Being able to pick and choose what type of bonus material I want would be preferable.

2. Being able to add certain titles to an ignore list would help as well. Why download Darksiders II when I already have DS2 Deathfinitive Edition stored locally? I can understand some people would want it all.. but my library in approaching 700 titles... my local backup folder is nearly 1TB and growing.

3. Currently, I download the game's box art and name it folder.jpg which makes browsing my backup folder almost like browsing my library shelf on the gog.com site. In addition to !info.txt and !serial.txt, would it be possible to scrape the box art as well? The only place I can still find the boxart for the titles is on the gog mixes creation page... Here's how my backup folders look with folder.jpg files... http://i.imgur.com/bdR1V7M.png

I understand some of this may not be possible due to lack of naming conventions of bonus material. Rather than tell it what I DO want.. perhaps the better approach would be to parse a file which includes terms I wish to filter out... [Fr],[Du],[Ru],[It],FLAC,video,videos,etc... .. any filenames that include these terms would be skipped.
avatar
Phin77: 1. It would be highly desirable for me to be able to filter what content is downloaded.
...
perhaps the better approach would be to parse a file which includes terms I wish to filter out...
I suggested earlier that there would be an ability to skip downloading files which include certain listed strings. I presume you are suggesting something similar? Then it would be up to the user to check how the files are named he wants to skip.

avatar
Phin77: 2. Being able to add certain titles to an ignore list would help as well. Why download Darksiders II when I already have DS2 Deathfinitive Edition stored locally? I can understand some people would want it all.. but my library in approaching 700 titles... my local backup folder is nearly 1TB and growing.
I think you can do that already now, by hiding the titles in your GOG account that you do not wish to download with gogrepo.

https://www.gog.com/forum/general/gogrepopy_python_script_for_regularly_backing_up_your_purchased_gog_collection_for_full_offline_e/post506

Yeah I also suggested that as GOG has started listing demo versions of games, which I obviously don't want to download anymore esp. after buying the full game (e.g. the demo for the forthcoming System Shock remake). So I'll just hide the demo versions in my account so that gogrepo will not add them to the manifest file later.
Post edited April 12, 2017 by timppu
avatar
Phin77: perhaps the better approach would be to parse a file which includes terms I wish to filter out...
avatar
timppu: I suggested earlier that there would be an ability to skip downloading files which include certain listed strings. I presume you are suggesting something similar? Then it would be up to the user to check how the files are named he wants to skip.
Yes. Exactly. Apologies... I haven't read all 36 pages of this thread.. :(

avatar
Phin77: 2. Being able to add certain titles to an ignore list would help as well.
avatar
timppu: I think you can do that already now, by hiding the titles in your GOG account that you do not wish to download with gogrepo.

https://www.gog.com/forum/general/gogrepopy_python_script_for_regularly_backing_up_your_purchased_gog_collection_for_full_offline_e/post506

Yeah I also suggested that as GOG has started listing demo versions of games, which I obviously don't want to download anymore esp. after buying the full game (e.g. the demo for the forthcoming System Shock remake). So I'll just hide the demo versions in my account so that gogrepo will not add them to the manifest file later.
Yeah but again gog is not consistent with how they do this.. In my example of Darksides 2... DS2 and DS 2 Deathfinitive editions are separate titles in my library... Darksiders 1 "legacy edition" is packaged with the Warmastered Edition... I suspect some demos are the same.
avatar
Phin77: Yeah but again gog is not consistent with how they do this.. In my example of Darksides 2... DS2 and DS 2 Deathfinitive editions are separate titles in my library... Darksiders 1 "legacy edition" is packaged with the Warmastered Edition... I suspect some demos are the same.
In the case of Darksiders, I guess you would have to use the first suggested feature, ie. blacklist by filename all the files from within Darksiders (title) that you want to exclude. E.g. exclude all files whose name include the string "setup_darksiders_2.2.0.9".

I am unsure if there is any other sensible way to exclude them. As you said, GOG is not quite consistent where it includes the "alternate" or "classic" versions of games, or even how they are named. Sometimes the extra installer is in the main game download section (like with Darksiders), sometimes it is in the Extras/Goodies section (Stronghold), sometimes it is a completely separate title in your library (Darksiders 2, Pathologic).

So depending how the version of the game you don't want is included, you'd have to use a different way to exclude it, either hiding the separate title in yourl library, or using filename keyword exclusion. The keyword string exclusion list would also allow excluding e.g. Mac and Linux versions which are sometimes added to the Extras section (which is common also for Windows users), sometimes there are fan-translated versions (e.g. in Planescape: Torment) etc. You'd just have to go through your whole library to find the files (and filename strings) to include to the exclusion list, but at least you'd have to do it only once, unless those names change at some point (not probable).
Post edited April 12, 2017 by timppu
I suggest you get the same filters GOG provides natually at most and that is it.

Genre, System, Language, Function.
We already have System and Language.

You can update/download specific titles.
You can only download Installers
you can only download Extras.

Do not make it too complicated.

If you only want to download or look for one specific game, it is easier to use the web browser than an automated script?

p.s. I mean it is up to Wooly where he wants to go, but I know that some people are just louder than others and can ruin things. It may become an eierlegende Wollmilchsau, many features but not good at anything.
Post edited April 12, 2017 by disi
avatar
disi: Do not make it too complicated.
I agree, and the keyword exclusion (by filename) shouldn't be too complicated to implement, and the good thing about it is that is is usable for lots of different "special cases". E.g. some people earlier requested a feature not to download patch installers (because the main installers are already probably up to date so the patches are not needed), someone wouldn't possibly want to download the Russian Windows/Mac/Linux translations that are included in the Planescape Torment game extras (that is almost 3GB of extra Russian stuff in that one game alone, even though you wanted to download only the English Windows versions), the guy above doesn't want to download the classic version of Darksiders installer, etc.

As far as I can tell, the keyword exclusion would cover all those cases, it would then be up to the user to define which filenames to exclude. The main problem with the filters is that GOG doesn't always use them, like e.g. when they add those Russian Windows/Mac/Linux localizations to games like Planescape. They get downloaded even if you use the Windows and English filters, because the Extras are common for all. Or then you would have to exclude all extras, for all games, just to avoid re-downloading those extra Russian installers over and over again.

In fact, I wouldn't be surprised that at some point GOG revises the whole offline installer download system and gets rid of those filters altogether, so if you intend to download offline installers (and not download and installl with Galaxy), you have to hand-pick the correct installers and files with your web browser.

But of course it is up to wooly to decide what he wants to add there and how feasible it is.

avatar
disi: If you only want to download or look for one specific game, it is easier to use the web browser than an automated script?
The request was not for downloading only a specific title/file, but to exclude it (from the hundreds of other games/files you intend to download with gogrepo).

Sure you could manually remove those unneeded big files afterwards... but then they would be redownloaded over and over again each time you run gogrepo update & download (or alternatively you'd have to edit the manifest file before download each and every time, removing all the entries you don't want).

(I guess one could make an automated script that always removes unwanted files from the manifest file, before you run download...).
Post edited April 12, 2017 by timppu
avatar
Ebany: Game OST's make up a large percentage of the background music I play, Game artwork often ends up as slideshows in my screensaver.
Normally I download everything as backup, so I have a lot of OSTs already locally stored. But until now I haven't listen to much of the music...
So can you give some suggestions for great OSTs? Thanks in advance.
high rated
Hi folks, just FYI some progress made but not ready yet. Hope everyone had a nice holiday.
avatar
woolymethodman: Hi folks, just FYI some progress made but not ready yet. Hope everyone had a nice holiday.
I'm just glad you're still working on it. Thanks a ton for your efforts.
avatar
woolymethodman: Hi folks, just FYI some progress made but not ready yet. Hope everyone had a nice holiday.
+1
I did an update, got a local manifest file, imported a bunch of files but didn't download new ones because it would take too long. I did a dry-run of the download which worked successfully, I then logged out of my Windows account, went to bed and when I came back the local manifest file wouldn't load. I entered the command to download, but it wouldn't use the manifest file I previously generated even though it's in exactly the same place as I did the dry-run the night before along with the gogrepo.py script file. I tried updating the repo file, but it then fetches each and every game details again even with -skipknown.

Edit: When the update finished, the new manifest file was placed in C:\Windows\SysWOW64 rather than in the same directory as the script file, so it seems the working directory was incorrectly set there.

Edit2: Now I have issues with some things only downloading at 0.1MB/s while others go at full speed.
Post edited April 23, 2017 by Undesirable
I hope the new version will have the fast manifest downloading.
This deserves the occasional bump I think, to remind people to back-up their games when they get the chance - that is the point of GOG after all (depending on how optional Galaxy stays in the future, but for the present I'm not too worried.)
This is a new one to me:
00:01:20 | {legend_of_heroes_trails_in_the_sky_the_3rd_the}
00:01:20 | fail setup_the_legend_of_heroes_-_trails_in_the_sky_the_3rd_2017.05.03_(11701).exe has incorrect size.
00:01:20 | download setup_the_legend_of_heroes_-_trails_in_the_sky_the_3rd_2017.05.03_(11701).exe
00:01:20 | fail setup_the_legend_of_heroes_-_trails_in_the_sky_the_3rd_2017.05.03_(11701)-1.bin has incorrect size.
00:01:20 | download setup_the_legend_of_heroes_-_trails_in_the_sky_the_3rd_2017.05.03_(11701)-1.bin
...
00:01:23 | chunk request has unexpected Content-Range. expected '0-733215/733216' received '0-732783/732784'. skipping.
00:01:23 | chunk request has unexpected Content-Range. expected '0-3506750125/3506750126' received '0-3506750125/3508678715'. skipping.
00:01:23 | --
00:01:23 | total time: 0:00:04.081782
00:01:23 | exiting...
Will try again tomorrow, but I don't know if there's anything I can do.
avatar
paladin181: This is a new one to me:

00:01:20 | {legend_of_heroes_trails_in_the_sky_the_3rd_the}
00:01:20 | fail setup_the_legend_of_heroes_-_trails_in_the_sky_the_3rd_2017.05.03_(11701).exe has incorrect size.
00:01:20 | download setup_the_legend_of_heroes_-_trails_in_the_sky_the_3rd_2017.05.03_(11701).exe
00:01:20 | fail setup_the_legend_of_heroes_-_trails_in_the_sky_the_3rd_2017.05.03_(11701)-1.bin has incorrect size.
00:01:20 | download setup_the_legend_of_heroes_-_trails_in_the_sky_the_3rd_2017.05.03_(11701)-1.bin
...
00:01:23 | chunk request has unexpected Content-Range. expected '0-733215/733216' received '0-732783/732784'. skipping.
00:01:23 | chunk request has unexpected Content-Range. expected '0-3506750125/3506750126' received '0-3506750125/3508678715'. skipping.
00:01:23 | --
00:01:23 | total time: 0:00:04.081782
00:01:23 | exiting...
avatar
paladin181: Will try again tomorrow, but I don't know if there's anything I can do.
This happens if the game has been updated between when you updated the manifest and when you downloaded. Delete the files giving you this error then force an update on the game in question, then download again and it should be fine.