It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
low rated
avatar
kohlrak: A little late jumping in here, but this is why i switched from C++ to C: While i loose fancy things like operator overloading, and this, i gain alot more in knowing the code i wrote won't be broken due to deprecation of language features (it's coming, thanks to Google, and the failure of their language GO to really catch as much as they wanted). Another thing I do is homebrew what I can or pull in FOSS sources that aren't likely to need frequent updates (i think the LZMA libs are pretty well established, for example). Unfortunately, that might be a bit much for gogrepo, however.
avatar
Magnitus: Thing is, C++ is mostly a superset of C (or was a decade ago when I was doing a lot of C++ anyways). Python 3 is not a superset of Python 2. They didn't just add stuff on top of Python 2. They broke things with Python 2.

You have to put a lot of "if" and "catch" statements to support both versions and equally importantly, you have to validate what you are doing across both python versions (unless you have automated pipelines which themselves will require time to implement, that's a lot of manual overhead).

I remember about 6 years ago when I was working on the Insomnia Sale script with phoeniixz. I believe she was using Python 3 and I was using Python 2 at the time. We were breaking each other's stuff a lot.

Otherwise, for Golang, I don't think its the end-all solution for everything (what is?). Its too low level to get things done yesterday (Python and Javascript still shine for that), but for portable binaries and operational cloud solutions (where it carved itself as the dominant player), it really shines.

I mean, right now, I have a self-contained binary in Golang that is less than 20mb, that will run by itself cross-platform on Windows (both 32 bits and 64 bits), MacOS, and Linux. The only way I could approach that with Python/Javascript is with Docker and even some of the developers struggle with that, nevermind non-technical users.
You seem to be missing the big picture of what I'm saying, largely because you must not be aware of what's going on.

C++ is being updated under ISO, and the representative from Google is trying to change the language to suit Google's needs. The most recent controversy is demands to break the ABI. Of course, this is a big deal for GOG customers, but trying to explain how this could affect people here is not likely to be well understood. I understand C++ works by jumping on top of C (something else they're desparately trying to change). Python2 vs Python3 is more like C++ of 2001 vs C++ as of the next update or two. The reason, for this, is obviously GO never caught on, and since C++ is still way bigger, they're needing to change the language to force people to use GO (by turning C++ into GO). Supposedly GO is more popular, but that's outside of the scope of my point.

My point is, we, as developers, need to start supporting things that are far, far more stable and reliable for long-term purposes. Projects like GOG repo can't be breaking (or putting the very busy dev under heavy stress) every 2 or 3 years (yes, i know python's cycle is longer than that, but for how long?) because someone thinks they have a way to change a language syntax to force coders to "make better choices." While this is great for making busy-work for the coders so that they can keep jobs on "completed projects," this means people like Kalanyr need to spend more time porting the code than working on other goals that he had. I mean, here's we're talking about python2 vs python3, but here i'm looking into whether or not these Python4 projects are real or if people are trying to pre-emptively say "Don't you even think about it." (Of course, i'm leaning towards the latter.)
avatar
kohlrak: You seem to be missing the big picture of what I'm saying, largely because you must not be aware of what's going on.
I haven't done any C++ in close to 10 years so no, I'm blatantly unaware of what is going on. C++ is great for gpu intensive games, but its no longer a big driver in networked applications.

avatar
kohlrak: C++ is being updated under ISO, and the representative from Google is trying to change the language to suit Google's needs. The most recent controversy is demands to break the ABI. Of course, this is a big deal for GOG customers, but trying to explain how this could affect people here is not likely to be well understood. I understand C++ works by jumping on top of C (something else they're desparately trying to change). Python2 vs Python3 is more like C++ of 2001 vs C++ as of the next update or two. The reason, for this, is obviously GO never caught on, and since C++ is still way bigger, they're needing to change the language to force people to use GO (by turning C++ into GO). Supposedly GO is more popular, but that's outside of the scope of my point.
Go is extremely popular in the cloud. If you're devops and want to extent Kubernetes or write Terraform providers, it really pays to know Go.

Overall, I don't think Go and C++ are really competing. They occupy different niches.

C++ is still the winner for lower level things that need to draw every inch of performance. Golang has taken a lot of the system level things above that. Its just a lot faster to write something correct in Go.

avatar
kohlrak: My point is, we, as developers, need to start supporting things that are far, far more stable and reliable for long-term purposes. Projects like GOG repo can't be breaking (or putting the very busy dev under heavy stress) every 2 or 3 years (yes, i know python's cycle is longer than that, but for how long?) because someone thinks they have a way to change a language syntax to force coders to "make better choices." While this is great for making busy-work for the coders so that they can keep jobs on "completed projects," this means people like Kalanyr need to spend more time porting the code than working on other goals that he had. I mean, here's we're talking about python2 vs python3, but here i'm looking into whether or not these Python4 projects are real or if people are trying to pre-emptively say "Don't you even think about it." (Of course, i'm leaning towards the latter.)
If you want stable and reliable, give people a completely self-contained binary, ideally one for each platform supported with as few assumptions as humanly possible (ideally, a web page for the gui, almost everyone has a browser). Anything else will give you pains.

I know, I've been writing tooling for devs for nearly a decade now. I've seen it all.

Wrong version of node. Wrong version of Python (sometimes, differing only by minor version). Doesn't install well in Windows because of x. Doesn't install well in MacOs because of y. Etc.

And of course, everybody just loves the "He tried to install Python 3 besides Python 2, with pip for both and some libraries and now, its all a jumbled mess... nobody is quite sure what he did exactly, but he really needs to format his machine, because nobody wants to troubleshoot this mess... formatting is quicker".

For a bit, I though Docker was the magical solution for all that, until I realised most people are not motivated enough to absorb that bit of complexity (even though it will rock their world and open up countless possibilities for them and make them way better devs than they were before, they just will not learn it).

Try to avoid, as much as possible, writing anything that assumes something (the correct version of an interpreter, a java vm, linked libraries, whatever) is installed on your user's machine. If you do, there will be pain.

Sometimes, you have no choice, but if you have a choice, avoid it.

PS: Python is still awesome for server-side and pipeline scripting, where you have tight control on the environment. I would just not use it anymore for a client that I pass around to users.
Post edited March 25, 2021 by Magnitus
low rated
avatar
kohlrak: You seem to be missing the big picture of what I'm saying, largely because you must not be aware of what's going on.
avatar
Magnitus: I haven't done any C++ in close to 10 years so no, I'm blatantly unaware of what is going on. C++ is great for gpu intensive games, but its no longer a big driver in networked applications.
I'd argue C++ isn't even relevant for games anymore, because of their attempts to "keep it relevant."
avatar
kohlrak: C++ is being updated under ISO, and the representative from Google is trying to change the language to suit Google's needs. The most recent controversy is demands to break the ABI. Of course, this is a big deal for GOG customers, but trying to explain how this could affect people here is not likely to be well understood. I understand C++ works by jumping on top of C (something else they're desparately trying to change). Python2 vs Python3 is more like C++ of 2001 vs C++ as of the next update or two. The reason, for this, is obviously GO never caught on, and since C++ is still way bigger, they're needing to change the language to force people to use GO (by turning C++ into GO). Supposedly GO is more popular, but that's outside of the scope of my point.
Go is extremely popular in the cloud. If you're devops and want to extent Kubernetes or write Terraform providers, it really pays to know Go.

Overall, I don't think Go and C++ are really competing. They occupy different niches.

C++ is still the winner for lower level things that need to draw every inch of performance. Golang has taken a lot of the system level things above that. Its just a lot faster to write something correct in Go.
Well, here's the irony: the arguments from Google's rep are based on the idea that C++ is no longer able to keep up with other languages because the ABI's inefficient (i would agree, but not on the basis that it's because of the C++ specific things). The big picture, now, is to break the ABI to make a C++ specific version. Eseentially, this would end up breaking all the DLLs out there that have functions that begin with _Z (prefix for C++ imports).
avatar
kohlrak: My point is, we, as developers, need to start supporting things that are far, far more stable and reliable for long-term purposes. Projects like GOG repo can't be breaking (or putting the very busy dev under heavy stress) every 2 or 3 years (yes, i know python's cycle is longer than that, but for how long?) because someone thinks they have a way to change a language syntax to force coders to "make better choices." While this is great for making busy-work for the coders so that they can keep jobs on "completed projects," this means people like Kalanyr need to spend more time porting the code than working on other goals that he had. I mean, here's we're talking about python2 vs python3, but here i'm looking into whether or not these Python4 projects are real or if people are trying to pre-emptively say "Don't you even think about it." (Of course, i'm leaning towards the latter.)
If you want stable and reliable, give people a completely self-contained binary, ideally one for each platform supported with as few assumptions as humanly possible (ideally, a web page for the gui, almost everyone has a browser). Anything else will give you pains.

I know, I've been writing tooling for devs for nearly a decade now. I've seen it all.

Wrong version of node. Wrong version of Python (sometimes, differing only by minor version). Doesn't install well in Windows because of x. Doesn't install well in MacOs because of y. Etc.

And of course, everybody just loves the "He tried to install Python 3 besides Python 2, with pip for both and some libraries and now, its all a jumbled mess... nobody is quite sure what he did exactly, but he really needs to format his machine, because nobody wants to troubleshoot this mess... formatting is quicker".

For a bit, I though Docker was the magical solution for all that, until I realised most people are not motivated enough to absorb that bit of complexity (even though it will rock their world and open up countless possibilities for them and make them way better devs than they were before, they just will not learn it).

Try to avoid, as much as possible, writing anything that assumes something (the correct version of an interpreter, a java vm, linked libraries, whatever) is installed on your user's machine. If you do, there will be pain.

Sometimes, you have no choice, but if you have a choice, avoid it.

PS: Python is still awesome for server-side and pipeline scripting, where you have tight control on the environment. I would just not use it anymore for a client that I pass around to users.
That's a similar conclusion i came to, but now we're looking at threats of damaged binaries. Making binaries that are completely self contained is untenable, because, at the end of the day, something must shoot up to the OS. What I've opted for (and argued for above) is self-contained code: while i can't prove C is safe, it's also the hardest thing to change without completely breaking everything out there. While I wished i could use some of the automatic features of C++ to help me (constructors and destructors), I can't rely on it. Instead, I've been making my own versions of things like "smart pointers" in C, just basically making a C-friendly library of things I use from C++, because I can't trust anything, anymore. I seem to be slowly finding alot of loose people out there that are about sick of this crap, too, especially in regards to languages like PHP: people are sick of having to write code from scratch for simple projects like gogrepo that shouldn't need much more than minor security and feature updates.

While alot of the time these things are said to be for improving the security of code written by coders to force them to code with better practices or something, usually these updates end up causing coders to just rush development, and abandon plans to improve certain algorithms, because they also have other things to be coding.

As for docker, i've been very unimpressed. Don't get me wrong, it's a cool idea, but i foresee it used in horrid ways. How long until the IDE on your screen is only 30% of your screen at one extreme, and at another your boss locks your docker and you have to ask you manager for permission to open google to search a problem at the other extreme. I think docker has potential from a user customization perspective, but my experience with the average user is that they spend less time learning how to use this. As for coders, this has the potential to be a nightmware if they find ways (assuming they haven't) to lock it for reasons stated above.

When looking up magical solutions, I think we need to take a step back and look at how these always turn out. I think OOP is easily the clearest example. Object oriented programming was supposed to be the big thing that ended all complexity, because you would just only worry about your current level of abstraction. There were a number of problems with this, and it could easily be explained in terms of "the double free bug." You see, object oriented programming was attempting at coming up with a simpler way to address a common problem, but not actually the magic solution to said problem. Fundamentally, at the end of the day, someone needs to free the resource, but the question reasonably becomes who's job that really is, and the fundamental problem that C++ was to address was to make it so you didn't ahve to think about such things, but that required someone actually did and declared that in the object description. So, then, someone catches that there is a memory leak, so the coder using the object then frees the memory, and then the library's updated and the maker of the class has decided that he needs to solve that bug himself. Now you have a double-free issue, which resulted from the opposite issue: not freeing. Prior to OOP, the question of "who's job is it, anyway?" was always present, and OOP aimed to solve that, but it actually failed to do so. However, it looks really nice in it's attempts. I'd argue that OOP's actual gain wasn't the things it set out to do, but the things it managed to accomplish without setting out to do them (better documentation of available APIs, via the nice little drop-down menues in IDEs after putting a dot), automatic constructors and destructors (most people don't realize that C++ actually does have a garbage collector if anyone would ever actually use it). On the flip side, then we got the abstraction hell, which, due to how we compile code, often ends up in alot of RAM hogging (because, polymorphism usually results in function pointers to function pointers to function pointers to function pointers to function pointers ad inifnitum well beyond cache misses). (I did some googling and some people are complaining specifically about encapsulation, but i believe this is the result of the fundamental issue i stated above, not the lack of encapsulation [because encapsulation would happen naturally].)

That said, contrary to what i written above, I actually like OOP. OOP provides wonderful tools for encapsulation, once everyone crosses their Ts and dots their Is. To that end, i'm sure docker's great, but we should also ask ourselves what it'll really cost. That goes for any ideas.
Sorry to interrupt the discussion, but what flavor of gogrepo.py is the most up to date and used now?
avatar
coffeecup: Sorry to interrupt the discussion, but what flavor of gogrepo.py is the most up to date and used now?
Kalanyr's dev branch, I believe:
https://github.com/Kalanyr/gogrepoc/tree/dev
Is there a way to get a list of all gameids when updating?

Of course they are written on screen amont with other info, but is it possible to get just the game id? maybe logged in a text file?
This way this file could be used to also download the games that have been updated. So far I wrote them down manually in order to do that.
avatar
neumi5694: Is there a way to get a list of all gameids when updating?

Of course they are written on screen amont with other info, but is it possible to get just the game id? maybe logged in a text file?
This way this file could be used to also download the games that have been updated. So far I wrote them down manually in order to do that.
There isn't one that I know of but you could search the manifest for them. On Linux you can use grep:

grep "'id':" "gog-manifest.dat" | sed -e "s/ 'id': //" -e "s/,//" > IDlist.txt

Will (should, it works for me) give you a file called "IDlist.txt" with just the game IDs, one per line.
avatar
HappyPunkPotato: grep "'id':" "gog-manifest.dat" | sed -e "s/ 'id': //" -e "s/,//" > IDlist.txt
Thanks. This way I got the numbers, but not the plain text game ids, but that's ok. In a similar way I should be able to get the other value as well, I'll experiment a bit.

I was specifically interested in getting only the IDs of the games that were updated during the last run of "python gogrepo.py update -os windows -lang en de -updateonly" for example. I only want do download these.
Thanks for the answer, maybe we'll get an option for that at some point.
avatar
neumi5694: maybe we'll get an option for that at some point.
It would be useful if it could give some abbreviated output with just and changes and errors!

If you know how to do any scripting you could do it yourself. I wrote a little Python script that creates a backup of the old manifest before running gogrepo to create a new one. Then it loads both as json and compares them for any changed or new games and prints out the information I want.

Alternatively you could make a backup of the old manifest yourself and use something like diif or kompare to compare it with the new one but then you would have to look through for the IDs yourself.
avatar
neumi5694: maybe we'll get an option for that at some point.
avatar
HappyPunkPotato: It would be useful if it could give some abbreviated output with just and changes and errors!

If you know how to do any scripting you could do it yourself. I wrote a little Python script that creates a backup of the old manifest before running gogrepo to create a new one. Then it loads both as json and compares them for any changed or new games and prints out the information I want.

Alternatively you could make a backup of the old manifest yourself and use something like diif or kompare to compare it with the new one but then you would have to look through for the IDs yourself.
I'm more of a Java programmer, didn't look into python yet. But yes, that would be an option, although it sounds a bit like anal eye surgery.
I might just look into Python :) After all there obviously already exists a list of all "todos" when downloading the infos of updated games, just need to save it somewhere.
Post edited April 30, 2021 by neumi5694
avatar
Kalanyr: GOG Servers are overloaded by the sale (as per usual), so this is a bad time to do updates / downloads and I would recommend avoiding such until it's over.
avatar
Timboli: I've never understood making something redundant, just because you can. It should always be on a needs basis.
avatar
Kalanyr: I'm still maintaining both 2/3 but support for 2 isn't zero cost, there's a reasonable divergence in syntax and functionality at this point. I'll probably provide a pure Python 2 branch when I decide to drop support for it, so someone else can fork / maintain Python 2 support if they want (and I can provide any needed major bug fixes).
Hello Kalanyr, I've got my new PC and tried to use your gogrepoc.py for the first time, and I am getting this error. I did not get anything like that with the gogrepo.py. Can you help me out what might be the issue? Thank you.

I've used this command to run it (I have installed python 3.9.7):

gogrepoc.py update -os windows linux -lang en

17:11:46 | ( 1 / 212) fetching game details for ad_2044...
17:11:50 | error
Traceback (most recent call last):
File "D:\Backup\GOG\gogrepoc.py", line 255, in __getattr__
return self[key]
KeyError: 'galaxyDownloads'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\Backup\GOG\gogrepoc.py", line 1383, in cmd_update
handle_game_updates(gamesdb[item_idx], item,strict)
File "D:\Backup\GOG\gogrepoc.py", line 570, in handle_game_updates
_ = olditem.galaxyDownloads
File "D:\Backup\GOG\gogrepoc.py", line 257, in __getattr__
raise AttributeError(key)
AttributeError: galaxyDownloads
Post edited June 10, 2021 by MMLN
avatar
Kalanyr: GOG Servers are overloaded by the sale (as per usual), so this is a bad time to do updates / downloads and I would recommend avoiding such until it's over.

I'm still maintaining both 2/3 but support for 2 isn't zero cost, there's a reasonable divergence in syntax and functionality at this point. I'll probably provide a pure Python 2 branch when I decide to drop support for it, so someone else can fork / maintain Python 2 support if they want (and I can provide any needed major bug fixes).
avatar
MMLN: Hello Kalanyr, I've got my new PC and tried to use your gogrepoc.py for the first time, and I am getting this error. I did not get anything like that with the gogrepo.py. Can you help me out what might be the issue? Thank you.

I've used this command to run it:

gogrepoc.py update -os windows linux -lang en

17:11:46 | ( 1 / 212) fetching game details for ad_2044...
17:11:50 | error
Traceback (most recent call last):
File "D:\Backup\GOG\gogrepoc.py", line 255, in __getattr__
return self[key]
KeyError: 'galaxyDownloads'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\Backup\GOG\gogrepoc.py", line 1383, in cmd_update
handle_game_updates(gamesdb[item_idx], item,strict)
File "D:\Backup\GOG\gogrepoc.py", line 570, in handle_game_updates
_ = olditem.galaxyDownloads
File "D:\Backup\GOG\gogrepoc.py", line 257, in __getattr__
raise AttributeError(key)
AttributeError: galaxyDownloads
Ahh yes, this is a consequence of AttrDicts behaving differently now than they did at the time I coded this and I didn't notice because this is a one off error (it only happens when converting manifest entries from gogreco to gogrepoc which I was well and truly past at the point this change happened). If you're game collection is tiny it's probably easiest to just delete your manifest and let it run from scratch, otherwise yoou should be able to search for

KeyError
[code]
try:
_ = game.galaxyDownloads
except KeyError:
game.galaxyDownloads = []
[/code]

and notice how in the try statement there's a . in game.galaxyDownloads <- in each instance where that pattern of
try (something.something) except KeyError occurs (and only there, because there's some real KeyErrors) you can replace KeyError with AttributeError and it should work.

Or feel free to send me a friend request and I'll send you a theoretically fixed file to test and we can go back and forth until I get it working and when it's actually working I'll upload it as a bugfix.
Post edited June 10, 2021 by Kalanyr
I've sent you a friend request, I can test out the file, if you'd like to make the fix for the new users of gogrepoc. :) But I need to run to some meeting right now, so I can get back to it maybe in two or three hours again.
avatar
MMLN: Hello Kalanyr, I've got my new PC and tried to use your gogrepoc.py for the first time, and I am getting this error. I did not get anything like that with the gogrepo.py. Can you help me out what might be the issue? Thank you.

I've used this command to run it:

gogrepoc.py update -os windows linux -lang en

17:11:46 | ( 1 / 212) fetching game details for ad_2044...
17:11:50 | error
Traceback (most recent call last):
File "D:\Backup\GOG\gogrepoc.py", line 255, in __getattr__
return self[key]
KeyError: 'galaxyDownloads'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\Backup\GOG\gogrepoc.py", line 1383, in cmd_update
handle_game_updates(gamesdb[item_idx], item,strict)
File "D:\Backup\GOG\gogrepoc.py", line 570, in handle_game_updates
_ = olditem.galaxyDownloads
File "D:\Backup\GOG\gogrepoc.py", line 257, in __getattr__
raise AttributeError(key)
AttributeError: galaxyDownloads
avatar
Kalanyr: Ahh yes, this is a consequence of AttrDicts behaving differently now than they did at the time I coded this and I didn't notice because this is a one off error (it only happens when converting manifest entries from gogreco to gogrepoc which I was well and truly past at the point this change happened). If you're game collection is tiny it's probably easiest to just delete your manifest and let it run from scratch, otherwise yoou should be able to search for

KeyError
[code]
try:
_ = game.galaxyDownloads
except KeyError:
game.galaxyDownloads = []
[/code]

and notice how in the try statement there's a . in game.galaxyDownloads <- in each instance where that pattern of
try (something.something) except KeyError occurs (and only there, because there's some real KeyErrors) you can replace KeyError with AttributeError and it should work.

Or feel free to send me a friend request and I'll send you a theoretically fixed file to test and we can go back and forth until I get it working and when it's actually working I'll upload it as a bugfix.
Post edited June 10, 2021 by MMLN
In the interests of not tearing my hair out in future, I've integrated the fix that MMLN helped tested and rolled dev to master. You'll have to update any support scripts/bats you have, but hey as a bonus dev has been tested for like 2 years now and has several advantages.

Let me know if you have problems.
I'm away on my Mint laptop and haven't updated my backup for almost a year, but I keep getting this with the new (today) and with old version (10-2020) of the dev script:


| Could not initialise the systemd session inhibitor: QDBusReply value is invalid
| fatal...
Traceback (most recent call last):
File "gogrepoc.py", line 2778, in <module>
wakelock.take_wakelock()
File "gogrepoc.py", line 2690, in take_wakelock
self.inhibitor.inhibit()
AttributeError: 'NoneType' object has no attribute 'inhibit'

Any ideas?