It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
avatar
tag+: Wondering if the use of Deutsch links (as reported on post #636) is already on your list
gog.com/forum/general/gog_database_a_website_that_collects_data_on_gog_games/post636
Yes, sorry I missed that post, but I have discovered it myself as well. GOG returns wrong data here, I am not too fond of fixing that stuff in the frontend, but I'm probably gonna have to.

avatar
tag+: Also to reask if would be possible to add the System requirements info to the gogdb
Maybe, but it's very low in priority.
avatar
Yepoleb: GOG returns wrong data here, I am not too fond of fixing that stuff in the frontend, but I'm probably gonna have to.
What do you mean? Both the products and v2 APIs return the "en-US" local links for me... maybe it's a geolocation problem?

In which case, does the gog_lc cookie help?

COOKIES = {
'gog_lc': 'BE_EUR_en-US'
}

This is what I use.
Post edited July 13, 2022 by WinterSnowfall
avatar
WinterSnowfall: What do you mean? Both the products and v2 APIs return the "en-US" local links for me... maybe it's a geolocation problem?
I am using both the "&locale=en-US" query parameter and "gog_lc=US_USD_en-US" and it is still returning German links.
avatar
Yepoleb: I am using both the "&locale=en-US" query parameter and "gog_lc=US_USD_en-US" and it is still returning German links.
Du bist Deutscher, du kannst nicht entkommen... :P. Weird then.
Post edited July 13, 2022 by WinterSnowfall
avatar
Yepoleb: Good news for everyone waiting for new updates: I've been approached by GOG to join their affiliate program and have done so a month ago! This means I have an incentive to work on GOG DB again and I'll try to spend a few hours every month updating the site. The first coming update will fix the random price tracking holes and change the main store link to an affiliate link. I hope everyone's okay with that, if you have any concerns please leave a comment! The downside might of course be that I'll be a bit less critical of GOG, but there are probably still enough angry people around to take care of my part.
Congratulations, it's a well-deserved achievement given all the effort you put into creating the site.

It occurred to me a possible quality-of-life improvement if it's not too complicated: many games have the price chart staggered because of the $99.99 placeholder that GOG puts for upcoming games, would it be possible to remove it?
And for future features, price tracking in euros would certainly be a welcome addition. Thank you!
avatar
Alexim: Congratulations, it's a well-deserved achievement given all the effort you put into creating the site.

It occurred to me a possible quality-of-life improvement if it's not too complicated: many games have the price chart staggered because of the $99.99 placeholder that GOG puts for upcoming games, would it be possible to remove it?
And for future features, price tracking in euros would certainly be a welcome addition. Thank you!
Thank you! Something with the price tracking is fucked in general, I have to put more work into figuring that out, but that should fix the $99.99 price as well.
Thanks a lot for this project and to all who contribute to it. Steamdb is the most used steam-related site for me hands down, so project like this is of great benefit for the userbase.

I only wonder (sorry if it is stated somewhere and I missed it), whether it's possible to download old builds via some 3rd party software using galaxy api, i.e. something akin to what steam allows via console, downloading depots, etc. I'm mainly interested in builds that were removed from the rollback option in galaxy client itself for whatever reason. Cheers!
avatar
Spectrum_Legacy: I only wonder (sorry if it is stated somewhere and I missed it), whether it's possible to download old builds via some 3rd party software using galaxy api, i.e. something akin to what steam allows via console, downloading depots, etc.
I don't know, have you given it a try? You can get unlisted manifests IDs from GOG DB, you just need to find the right kind of downloader.
avatar
Yepoleb: Text changelogs are now linked at the top of the changelog tab when they're available. I called the feature release notes to avoid confusion.
Good one, thank you and welcome back!
low rated
avatar
tag+: Wondering if the use of Deutsch links (as reported on post #636) is already on your list
gog.com/forum/general/gog_database_a_website_that_collects_data_on_gog_games/post636
avatar
Yepoleb: Yes, sorry I missed that post, but I have discovered it myself as well. GOG returns wrong data here, I am not too fond of fixing that stuff in the frontend, but I'm probably gonna have to.

avatar
tag+: Also to reask if would be possible to add the System requirements info to the gogdb
avatar
Yepoleb: Maybe, but it's very low in priority.
All right, I hope you figure the DE thing to let me use your affiliated links.
Or meanwhile please tell me the suffix to use to achieve the affiliated goal.

And no hurries on the sys reqs, just a nice to have.

Many thanks again for openly sharing your work and its very useful result :)
avatar
tag+: All right, I hope you figure the DE thing to let me use your affiliated links.
Or meanwhile please tell me the suffix to use to achieve the affiliated goal.
Just click the link and change the language back to English, attribution lasts for 7 days (this is public information). I just discovered I can make language neutral links, so I will apply that filter to all URLs soon.

Edit: This is now implemented.
Post edited July 14, 2022 by Yepoleb
high rated
I just released a big crawler update that I've been working on for the last 3 days! This changes the default sorting of the products page to match that of the "Bestselling (all time)" sorting option. The old "popularity" sorting is unfortunately no longer available in the new design, although I quite liked it. The main reason for the update was the unreliable price tracking, this should now be much better. No more gaps in the price data or even missing price history. Celebrating this improvement sadly also means admitting that a lot of price changes were recorded with a delay, usually under 24h, but for some games much longer. I only noticed the extent of the problem a few days ago, otherwise I would have fixed it much earlier, sorry about that :(.

The following is a technical explanation for the programmers following this thread. I have switched from the www.gog.com/games/ajax/filtered to the catalog.gog.com/v1/catalog API to get a list of all the store items. This switch was not urgent, but it had to be done at some point because the old API will probably be shut down soon. Thanks to the person who kept it alive long enough to give me time to switch! I like that the new interface has country, currency and language as parameters, this should be more convenient to debug and more reliable. It also has a better price value format and some properties I'll implement soon. I think whoever designed it did a pretty good job (besides the date format, who the hell uses 2020.07.16, you were so close to doing this in proper ISO8601).

api.gog.com/products/prices was removed as part of the migration. Apart from the usual reliability issues of all GOG APIs (random server errors, connection errors) it often failed to return values for requested IDs. This made the endpoint unusable for my purpose because randomly missing entries with no error indication are really hard to deal with. Instead prices are now collected using the regular store crawl. I originally liked the concept of getting prices separately because of the higher performance and flexibility, but losing it does not have too much of an impact.
avatar
Yepoleb: I have switched from the www.gog.com/games/ajax/filtered to the catalog.gog.com/v1/catalog API to get a list of all the store items.
Slight problem here though, because catalog.gog.com/v1/catalog is NOT in sync with what you get on GOG's main page from (no matter what sort order you select, as far as I can tell):

https://www.gog.com/discover_games/upcoming
https://www.gog.com/discover_games/new

I'm mainly referring to ordering, because the catalog API does appear to actually list all the new arrivals, but definitely not in the actual order of release on GOG (there's no option for that, I guess?), only the declared game release date.

avatar
Yepoleb: This switch was not urgent, but it had to be done at some point because the old API will probably be shut down soon.
Great, I was using it too... guess I will have to switch to a combination of the above to scrape new IDs. Thanks for the heads-up!
Post edited July 16, 2022 by WinterSnowfall
avatar
WinterSnowfall: Slight problem here though, because catalog.gog.com/v1/catalog is NOT in sync with what you get on GOG's main page from (no matter what sort order you select, as far as I can tell):
I don't care about the main page, I follow the order from /en/games. And I can do GOG release date sorting based on collected data.

avatar
WinterSnowfall: Great, I was using it too... guess I will have to switch to a combination of the above to scrape new IDs. Thanks for the heads-up!
An extra note on that: I do 3 passes for the store listing. First with alphabetical sorting and then one for bestselling (recently) and bestselling (all time). Then I combine all the results. This is because games might jump around in order with the other sorting methods (not sure if due to people buying games or they just use non-deterministic sorting and have not noticed that problem yet).
Thanks for creating and maintaining your great project and congratulations to being made an affiliate! I've been using your dumps on a weekly basis for some time now, so it's good to know that they won't go anywhere anytime soon.

I've noticed that on Sunday, 537 new product ids were added to the dump with no info about them whatsoever, only containing an empty prices.json with a timestamp. Usually I expect there to be either a product.json or a product_removed.json, but they're both missing. Some examples are 1074003868, 1625557603 and 2147483074. Also, 1540875525 is the only one to include both files for some reason.

Do you have any idea what they are about, will that be fixed? Otherwise I'd just ignore those discrepancies in my scripts.

On that topic, you may want to think about checking api.gog.com/v1/bannedProducts for ids, too. There are some products listed that aren't on GOG DB yet, like the Japanese versions of Horizon Zero Dawn (1811181314) and Mad Max (1086074200). There's also 1767680123, the GOG Galaxy Integration Demo, which isn't listed anywhere as far as I know. I'm not sure if you can fetch any meaningful info for the DB about it, though.