It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
avatar
Yepoleb: I think 2,147,483,647 should be the highest possible ID. It's equal to 2^31 - 1, which is the upper limit for a 32 bit signed integer type. Anything higher than that would be a huge waste of storage space.
Hmmm, never thought about that, but I guess it makes sense for the id range to map to an actual variable type range in a programming language and/or database storage engine.

Do also note that GOG is not yet using any id (except ids 1 to 10) below 1094368489, which gives them a bit of a downward wiggle room. The highest id I'm currently aware of is: http://api.gog.com/products?ids=2130376219, so your hypothesis may yet be confirmed.

My scans still haven't reached that high, though I'm getting close to it. I'll keep you posted.
Post edited February 11, 2017 by WinterSnowfall
avatar
Yepoleb: I think 2,147,483,647 should be the highest possible ID.
I've started scanning above this limit and haven't found anything yet. The highest id I've currently found is: http://api.gog.com/products?ids=2146639323. I'll leave the scan running for a while longer, just to prove your point, but to be honest I think you are right.
high rated
I hereby declare the first mapping of the entire available products public API range complete.

Let's all celebrate with a safety dance!

As promised, here is the data I've collected:
-> in a not-so nicely packed CSV format: https://dl.dropboxusercontent.com/u/1845258/gog_products_api_18012017_14022017.csv
-> full JSON response dumps: https://dl.dropboxusercontent.com/u/1845258/gog_products_api_18012017_14022017.json

I'll come back with some comments and a partial analysis in a few hours.
Post edited February 24, 2017 by WinterSnowfall
high rated
A few highlights, deductions and comments based on the scan results:

-> I've queried ALL ids from 1 up to 2216270000
-> First 10 ids are there just for testing most likely, they don't appear to be actively used any longer
-> There's a big void of unused ids between 10 and 1094368489
-> 1207658645 to 1207667273 is a very tightly packed range of mostly populated ids, which makes me think they were assigned/reassigned during the galaxy data cleanup and/or unbulding
-> 1420630484 to 1467789790 is also a very busy area, though not as busy as the one above
-> the highest populated id I've detected is 2146639323, which I think confirms Yepoleb's assumption that the id range is in fact stored as a 32bit signed integer, as there is a second vast empty range above 2146639323, all the way up to my scan's stopping point of 2216270000 at least
-> new ids have no discernible (as of yet) assignment logic, so don't expect newer games to have higher ids, though sometimes this seems to be the case

One more thing to mention is that there are probably some games missing from my data dumps - they were released and assigned an id below the range I was sweeping at that time.

To be honest, I haven't had too much time to look over the dumps yet, so there may still be a few gems hidden in the ocean of ids. Feel free to look around, the links to the dumps are posted above.

I plan a more refined second mapping in the near future. Watch this space.
Post edited February 16, 2017 by WinterSnowfall
Oh, and one more potentially interesting thing (for some people, at least)... here's a full export of all the companies (developers/publishers) currently listed by GOG: https://dl.dropboxusercontent.com/u/1845258/gog_companies_15022017.csv

I've generated it using one of the new scripts I'm working on for the second full mapping run. I plan on adding the associated author/publisher to each of the ids that I detect, though this is not something that's returned by the products API and I have to do some side-ways maneuvering and some other queries in order to get it.
avatar
WinterSnowfall: Oh, and one more potentially interesting thing (for some people, at least)... here's a full export of all the companies (developers/publishers) currently listed by GOG: https://dl.dropboxusercontent.com/u/1845258/gog_companies_15022017.csv

I've generated it using one of the new scripts I'm working on for the second full mapping run. I plan on adding the associated author/publisher to each of the ids that I detect, though this is not something that's returned by the products API and I have to do some side-ways maneuvering and some other queries in order to get it.
Where do you get the developer information from? I'm currently using the gogData variable on store pages, but maybe there's a better way.
avatar
Yepoleb: Where do you get the developer information from? I'm currently using the gogData variable on store pages, but maybe there's a better way.
If there's a better way I couldn't find it either - so I'm also parsing gogData :) and storing all the company data in a neatly ordered sqlite table which I plan to cross-references for each mapped id I detect on my second scan run.

Where am I going to get the author/publisher data for each game? I'll use the product API value for the "product_card" to load the product page and parse the two values from there. Obviously this is not possible for unreleased or removed games, but for all the regular games at least it works.
I've started my second scan and all I can say so far is that GOG's been busy... some of the newer games have ids below the minimum I've detected in the last scan (1094368489). I guess they're trying to expand into lower id ranges as well.
Post edited February 17, 2017 by WinterSnowfall
avatar
WinterSnowfall: -> First 10 ids are there just for testing most likely, they don't appear to be actively used any longer
I actually have games 1 through 5 installed. 1-3 are games that are no longer sold on gog, but remain in buyers libraries.

gog games that are installed show up in the windows registry with their id under :
'HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\' as '[id]_is1'
Post edited February 18, 2017 by bernstein82
avatar
bernstein82: I actually have games 1 through 5 installed. 1-3 are games that are no longer sold on gog, but remain in buyers libraries.
Oh yeah, you're right. Forgot that ids 1-3 are in fact the old pre-Bethesda Fallout game entries. Well then some of the first 10 ids might still be actively used, but I still think they were left in their current positions mainly for testing/convenience purposes.

avatar
bernstein82: gog games that are installed show up in the windows registry with their id under :
'HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\' as '[id]_is1'
That's good to know, but of course I don't use Galaxy for any of this :), though I suspect other parties keep a close watch on it to derive real-time game-related information. Never had it installed and probably never will. Not that I have anything against it, I'm just not a fan of clients of any sort.
Post edited February 18, 2017 by WinterSnowfall
Using the GOG APIs is always... fun... a blast really... like the blast you'd get when shooting yourself in the leg. But there are perks!

Here's a patented easy-peasy way to bring the GOGBear out of its den: https://api.gog.com/products/1207667273?expand=downloads,expanded_dlcs,description,screenshots,videos,related_products,changelog

What did I just do? Why, I ran an expanded product query on an id linked to a movie - apparently the bear doesn't like that at all.

Ah well, since I've structured my entire second run on expanded API calls, don't expect my scans to include any movies from now on. I'm just going to ignore them, along with the atlas GOGBear and the HTTP 500 error code it returns. Stop trying to mess with my code and just hibernate for a few more months, ok?
That explains why I got an error trying to query https://api.gog.com/products/1691391100?expand=downloads,expanded_dlcs,description,screenshots,videos,related_products,changelog (Double Fine Adventure Definitive Edition). It's a movie that somehow found its way into the games section. If you remove downloads it works, at least in this specific case. Good to know why it failed.
Just so you know, you can get a response as long as you drop 'downloads' as one of the parameters. =\
avatar
Yepoleb: If you remove downloads it works, at least in this specific case. Good to know why it failed.
avatar
SCPM: Just so you know, you can get a response as long as you drop 'downloads' as one of the parameters. =\
Thanks guys! I was trying to stay away from recursive functions, but I might as well include one now for all those pesky movie entries :). Personally I blame the trained monkeys that, as we've recently learned, work at GOG on various API coding tasks under the supervision of the GOGBear overlord.

avatar
Yepoleb: It's a movie that somehow found its way into the games section.
Trust me, there's more than one.
Post edited February 20, 2017 by WinterSnowfall
avatar
SCPM: Just so you know, you can get a response as long as you drop 'downloads' as one of the parameters. =\
avatar
WinterSnowfall: Thanks guys! I was trying to stay away from recursive functions, but I might as well include one now for all those pesky movie entries :).
Recursive functions? It should be a simple if response.status == 500 then retry without downloads.

avatar
Yepoleb: It's a movie that somehow found its way into the games section.
avatar
WinterSnowfall: Trust me, there's more than one.
It's the only one that brought out the gogbear for me so far.
Post edited February 20, 2017 by Yepoleb