It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
low rated
avatar
Timboli: @GameRacer - Thanks.
Have a few questions if you have some time(and so I don't have to read through a ton of older posts):

1. Will the program be(or is it now) able to be portable if one wants or needs such functionality?
(As in running from a folder one can move around to a memory stick or such)

2. Does the program need any dependencies to be installed to function(on windows I mean)?

Thanks in advance for any answers you can provide.
@GameRacer - You really should go back and read all the relevant posts in this thread, which should answer any further questions you might have, including the ones you've asked.

Just this once however, I will answer your two questions, but be aware that all will be explained when my program is ready for release and has its own thread. Until then, do as I suggested in my first paragraph above.

My program is a self contained frontend or graphical user interface (GUI) for the gogrepo.py script.
So my program requires the gogrepo.py script, and must share the same folder.
The gogrepo.py script in turn requires that Python be installed, with the Python location set as a system PATH.
You can place the folder containing GOGRepo GUI.exe and the gogrepo.py script, wherever on your Windows PC, so it is portable in that sense.
So long as the gogrepo.py script runs from wherever it is located, you are cooking with gas.
Python also requires a specific library be installed, which is dealt with by GOGRepo GUI and an Internet connection.
Post edited July 06, 2020 by Timboli
All up, for 978 games in my GOG library, it took 5 hours and 50 minutes to get all the required detail via UPDATE ALL to build the manifest file on my PC. This was for both Windows and Linux plus all game extras. It means I now have all the detail required for downloading any of those games. It is also a process I should never have to repeat, as from now on I only need to do an UPDATE ALL for new games or changes, which should barely take any time at all. I can also selectively update a game.

The manifest file (gog-manifest.dat) is now almost 5 Mb, and was around 4 Mb before the update. So around 1 Mb accounts for the missing Windows games file info.

I can now carry on with the next phase of testing, tweaking and fine tuning etc.

NOTE - The manifest file not only contains a list of all game files (Windows & Linux), but also all the game extra files. It also contains file dates and sizes, plus links to image files and downloads, and full Changelog information. In fact, it is the Changelog information that takes up the most space. Checksums are also listed, along with other information about the game (genre, rating, series, OS, language, etc). You also get store URL and forum URL.

By selecting a game title on the list in GOGRepo GUI, and then clicking the MORE button, you get to see a web page representation of the manifest information for one game. File sizes and dates are not in a user friendly form at the moment, but I intend to eventually convert them so they are.
Post edited July 06, 2020 by Timboli
low rated
avatar
Timboli: @GameRacer - You really should go back and read all the relevant posts in this thread, which should answer any further questions you might have, including the ones you've asked.
Sorry for that. It's just that you post so much "technical information" and your posts (no offense by this, as I do it too) can be a bit long and searching them a bit hard**.

That said thanks for the information, and I am glad to also hear you will be answering all this sort of stuff when the program is finished.

(**=For one like me....don't know if I am explaining this right, but essentially: my mind, when it sees long technical texts and such, wants to mentally skip chunks of it and skim it for needed info...and if I cannot find it quickly enough my mind becomes averse to continuing to try to find such information)
Post edited July 06, 2020 by GameRacer
A few things have occurred to me that you may, or may not, have covered. Just in case you don't I'll mention them:

1. Do you have an option to select which type of installers to update/download, i.e. standalone, Galaxy or both? These are choices available with Kalanyr's gogrepo.pv when running the "update" command as (i) -installers standalone, (ii) -installers galaxy and (iii) -installers both. When actually doing a "download" they are (i) -skipgalaxy, (ii) -skipstandalone and (iii) -skipshared (although the latter is a bit more of a special case). I can't recall if the version of gogrepo.py that you're using includes this, but being able to differentiate the type of installers to download is very important.
2. Are you running the gogrepo.py clean command after the download and before any verification?
avatar
ikrananka: A few things have occurred to me that you may, or may not, have covered. Just in case you don't I'll mention them:

1. Do you have an option to select which type of installers to update/download, i.e. standalone, Galaxy or both? These are choices available with Kalanyr's gogrepo.pv when running the "update" command as (i) -installers standalone, (ii) -installers galaxy and (iii) -installers both. When actually doing a "download" they are (i) -skipgalaxy, (ii) -skipstandalone and (iii) -skipshared (although the latter is a bit more of a special case). I can't recall if the version of gogrepo.py that you're using includes this, but being able to differentiate the type of installers to download is very important.
2. Are you running the gogrepo.py clean command after the download and before any verification?
Thank you very much for those, as I was not aware yet of either.

Do you or anyone else happen to know if the original gogrepo.py script defaulted to a -skipgalaxy scenario?
I am very close now to providing a release candidate.

Still a few loose ends to tidy up though, including full on testing.
I've also got to check Kalanyr's version of gogrepo.py, and make additions or amendments for that. I may release something before that though.

----------------------------------------------------------------

The FIND button now has a second choice, if you hold down CTRL while clicking. You can search with 'contains' rather than 'starts with'.

I've also added a LAST button, which does a similar find process, working backwards through latest games added to the manifest, and jumping to each with a click. If you hold down CTRL while clicking, it starts over.

I finally found time and remembered to develop some code to sanitize game names, to prevent errors in the INI database I use. I use an INI file because it is quick and simple and easy to troubleshoot. But even if I had gone XML or SQL instead, I would still have needed to do some sanitizing of entries. That removes characters like trademark sign and other troublesome ones, including converting foreign ones to English. Hopefully that last won't impact users from non English countries. All 976 game titles in my GOG library are now dealt with properly, and hopefully that will be the same for other users of GOGRepo GUI.

I've made some other changes to the various windows, especially the Queue window, to give proper support for verifying after each download. I haven't enabled it yet, but you will eventually be able to set verification options on a per game basis.

Auto Start is disabled for my version of DOWNLOAD ALL, but you are immediately taken to the Queue window, where you can click START. The program also enters a DOWNLOAD ALL state, after you populate that option, and remains that way, even after a program restart, until you have completed DOWNLOAD ALL or clear the list on the Queue window. You can still use some options on the main program window, including adding single titles to the download list, in case you have removed any etc.

Main Window

Download ALL Window

Queue Window

Some of you may note the UPDATE Selected button has been removed on the Queue window, and replaced by a relocation of the Languages field. The 'Game Files' checkbox is also relocated and renamed to just 'Game'. The OS field now shows what can be downloaded per game, so it is wider. NOTE - The language and OS options are set when you do an UPDATE, and you cannot change them for downloading via the download options, so I only have them displayed for informative reasons. You can however, elect to download Game Files and/or Game Extras ... though I have yet to enable being able to do that on an individual game basis (currently it uses what is set for all).

As you can also see with the Queue window, I have reduced the height of the Download lists, to accommodate the Verify options, which as I said above, you will eventually be able to change on an individual game basis.

Mostly what I have left to do now, is adding more information to dialogs and making sure all errors are dealt with. I also need to add a check for an Internet connection every time 'gogrepo.py' is used. And of course, I need to check if the queuing works as planned.

I'm also nursing a long time big toe injury on my left foot, which has been plaguing me a lot lately ... not been having it elevated enough, and alas I am not setup to use this PC while it is elevated, so it has been slowing me down in recent times. Still, we are very close to the end now.
Post edited August 03, 2020 by Timboli
Something I wasn't sure about, but had my suspicions, and now definitely seems to be so, is that when I use gogrepo.py to do an update, it messes with the sort order in my game library web page settings at GOG. In other words, it annoyingly puts it into sorted by Title, which is not what I want when I have just purchased a game. It also appears, which I hadn't realized until now, to clear notifications of what has been updated recently, which is even more annoying as you cannot restore them.

I am hoping that doesn't occur with Kalanyr's version ... or it restores those two things after extraction.
Well today was the moment of truth, and while I had to fix a few little issues, the queuing worked as intended.

Before that test, I had added in the code for web connection checks where needed, and the code to deal with no connection found or a loss of connection.

I also coded a backup method for the manifest file at the start of each Update, so that up to five cycled backups are kept in a Backups folder, for the manifest file and other related files. A backup can be used by those savvy enough, to restore things if something went wrong with an Update, etc.

I've spent a good amount of time adding more information and advice to some dialogs.

I've implemented a right-click menu for the 'Games' title list, with three options.
(1) 'Go to Store page' for selected game.
(2) 'Go to Forum page' for selected game.
(3) 'Go to Library page'.

I still have more tests to do, and a couple of niggling issues to resolve, one more concerning than the other, but things appear to be mostly working as they should.

I did two types of queuing tests. One with 'Auto Start', where the first game added to the download list started downloading immediately, then I added a second game to the list. The second without 'Auto Start', where I added two game titles to the download list and clicked START when I was ready. Both methods worked well, except for some smallish issues.

One of those issues, was not clearing up all settings upon completion. Another was a wrong value sent to the progress bar. Another was reactivating some things sooner than should be. The biggest issue was the overlooked use of the same variable twice, which meant game extras were skipped.

All issues were easily fixed, and I did two more tests where I just downloaded the game extras. I then discovered another issue during that, during the Verify stage, where game files were being tested again, wasting time. I've implemented a fix for that, where the MD5 check is not enabled when only game extras are downloaded, plus also no Zip file checking when game extras are not downloaded.

Another issue which seems to happen now and then, is the game cover image fails to download for some unknown reason. A simple exists check and a repeat of the download if not existing, may solve this random seeming issue, that may just be due to a slow server response.

I'm fairly pleased with results, and now need to do more testing.

I will hopefully release a working version for others to test tomorrow.

P.S. During testing, I used one thread only to download, and it worked well enough at 5.xx Mb/s for me for game files. When downloading game extras, I noted it was downloading at about one fifth of that speed, so very much similar to what the old GOG Downloader used to often do with game extras ... probably based on server usage at any given time.

Using a third party downloader with browser links, like Free Download Manager 5, which doesn't use the GOG SDK, doesn't seem to suffer from that slowdown ... neither does Galaxy I seem to recall, but I may be wrong about that.
Post edited July 08, 2020 by Timboli
Did a bit more fine tuning, plus created an icon for the program.

Icon

Then I did 4 more game downloads, using Auto Start, and all worked fine except for two issues.

(1) Cover image for first game failed, even though I had it set to check and repeat up to two more times. Afterward I manually used the program option to download the cover image, with no issue, so no idea what causes this seemingly random issue as the covers for the other three games downloaded fine.

(2) For some reason the Progress Bar showed 20% instead of 25% after the first download. However it correctly showed 50% and 75% and then 100% ... though I cheat for that last value. Perhaps the program initially thought there was an extra file for the total ... a flaw in my checking logic at the start maybe.

Anyway, aside from those two minor issues, it worked well.

There are some issues I need to investigate and probably ask about, but I will leave them until after my sleep.
One very important one of those, is whether an error file ever gets created by the Verify option, as my GUI does not read the STDOUT to detect anything ... an inevitable consequence of having such a nice queuing process, so game entries can be added, removed or modified on-the-fly on the download list. Normally I do read STDOUT, but the programming language I use is not multi-thread aware, so I have a limited choice of options. It is also why I have a simple Progress Bar that works on a percentage of files downloaded. That last might seem pointless to have, but if you are at a distance away from your PC, a green bar is easier to see than a small number.
Post edited July 09, 2020 by Timboli
low rated
avatar
Timboli: I am very close now to providing a release candidate.
Sweet

avatar
Timboli: I'm also nursing a long time big toe injury on my left foot, which has been plaguing me a lot lately ... not been having it elevated enough, and alas I am not setup to use this PC while it is elevated, so it has been slowing me down in recent times. Still, we are very close to the end now.
You have my sympathies....I once tore off an entire toenail or close to it and was in pain for a good long while.

That said if you feel like taking some time off i'm sure no one would mind....your health comes first, after all. :)

And once again, thanks for all this.....you are a kind kind soul, and I wish you the best.

avatar
Timboli: It is also why I have a simple Progress Bar that works on a percentage of files downloaded. That last might seem pointless to have, but if you are at a distance away from your PC, a green bar is easier to see than a small number.
I love this bit, as I find those swirling circles and such to be confusing and vague(they don't even tell you how long you have to wait for something, etc) & numbers can be too small to see or read as you said.
Post edited July 09, 2020 by GameRacer
Thanks.

-------------------------------------------------------------------

Several more tests done, more code done for improvements, fixes and overlooked elements.

Still having that failed cover download issue, and now seems to only be for the first game, and perhaps only when in Auto Start mode. I've mainly been focusing on other things, so I just note that in passing.

I've belatedly remembered to add code to populate the Games Downloaded list on the Queue window. I've also modified the Main program window, with smaller SETUP and LOG buttons now sharing the previous location of the SETUP button. Where the LOG button was, is now an 'Added' field with a count for games added to the Download list, and either a RED background (downloading has started) or GREEN (populated list and ready to start downloading) or BLACK (not populated). I did that because I felt there was a need for some feedback on the Main program window, especially after adding a game to the download list.

Main Window (screenshot)

The progress bar on the Queue window, is now showing the correct percent at all times. However, when finished downloading, it is not reset until the program closes or a new download session started. The Games Downloaded list will persist after a program close, but is also cleared when a new downloading session starts.

For those of you who feel up to the task and are keen to try the latest GOGRepo GUI out, you can now do so.

BE WARNED - It has still had very little overall testing ... minimal in fact for most aspects ... some not at all.

GOGRepo GUI (download)

I STRONGLY SUGGEST that you read what I have written in earlier posts in this thread, to gain required insight and understanding, though some of it is now dated. However, several in-program dialogs give explicit information.

Constructive FEEDBACK welcome.

Enjoy!

P.S. This is provided for FREE as is, and while I have gone to significant measures to make sure the program is safe, ultimately you use at your own risk. Any virus related issues, please submit and check at VirusTotal. The 'gogrepo.py' script has been developed by others, unrelated to myself.
Post edited July 09, 2020 by Timboli
avatar
Timboli: It also appears, which I hadn't realized until now, to clear notifications of what has been updated recently, which is even more annoying as you cannot restore them.

I am hoping that doesn't occur with Kalanyr's version ... or it restores those two things after extraction.
I can confirm that Kalanyr's version has this effect as well. I don't believe there is a way around this as it is a GOG thing. I believe that simply polling for info on the updated game files results in the GOG update flag being reset/cleared. The same behaviour happens on the website, if one clicks on the cover image of a game in your library that has an update flag then the update flag is cleared.

You should also be aware that often there are games that have updates and GOG fails to set the update flag on them. As such Kalanyr and others recommend that a full manifest update be run using gogrepo.py on a regular basis (say once per month) as this identifies all changes/updates independent of the GOG update flag setting.
Post edited July 09, 2020 by ikrananka
avatar
Timboli: Do you or anyone else happen to know if the original gogrepo.py script defaulted to a -skipgalaxy scenario?
As far as I know, the original gogrepo.py script was developed in the early days of Galaxy before GOG really started messing things up with both Galaxy and Standalone installers. As such, the original gogrepo.py didn't include any code for selecting the installer type. So, running it today I have no idea which one it would default to. However, as you're running GOGPlus with the original gogrepo.py you should be able to tell by looking at the installers that it downloads.
avatar
ikrananka: I can confirm that ...........
Thanks for both replies, and I suspected that was going to be the likely case.
I guess you just have to live with some things ... two steps forward, one step back ... or is it three steps forward. :)

P.S. I do wonder if it is possible to get the games that currently show the update flag, and save the list of those to a text file in the script folder?
Post edited July 09, 2020 by Timboli