It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
Community
General discussiongog.com API(34 posts)(34 posts)
(34 posts)
Pages:
1
2
3
This is my favourite topic
Posted March 04, 2016
avatar
mrkgnao: I was unable to find the video, but I believe you.
avatar
Gydion: Or shmerl in this case. I never actually watched the video myself...
Well, Shmerl is a very trustworthy name where I come from, alongside Berl and Cherl.
Posted March 04, 2016
A while back i requested information on the API so i could build a little trading page for people to use, mostly that checked and ensured games on a gift coupon existed and weren't expired without giving the codes away (unless both agreed to the trade(s)), never got any feedback. Something like a week later they changed to using move Java and JQuery making it more difficult for me.

Oh well.
Posted March 17, 2016
Is there any way to request the data of a promo page, like the list of the games and the prices?
Posted March 17, 2016
avatar
eiii: Is there any way to request the data of a promo page, like the list of the games and the prices?
libcurl + libtidy
Posted March 17, 2016
avatar
eiii: Is there any way to request the data of a promo page, like the list of the games and the prices?
I'll try and remember to come back to this and give you some tips to get the Angular data from the promo page so you could get the info you want.
On my phone now so I can't use the console to figure out what you'd need to do.
Posted March 17, 2016
avatar
eiii: Is there any way to request the data of a promo page, like the list of the games and the prices?
First, if you don't use my script you might need to enable the Angular JS debugging for this to work. To do that run;
angular.reloadWithDebugInfo();
Which will refresh the page and enable debugging to make the .scope() function work.

You need to be on the promo page itself (either in the console, injecting a userscript or using an AJAX call), then run this function:
var scope = angular.element(document.querySelectorAll('.promo-columns-wrapper')).scope();

This will return an object with quite a few properties, the ones you'd probably be interested in are;
scope.data.products which contains a list of all the products (owned and unowned) which you can then do what you like with.
scope.notOwnedProducts which is an array of just the products you don't own.
scope.ownedProducts which is just the games you do own.


Hope that helps :)
Posted March 17, 2016
avatar
adaliabooks: Hope that helps :)
I hoped more for something outside of a browser, like a URL to a JSON file with the data or something like that. But I'll try to understand what you have written there and see if it helps me to extract the data. Thanks!
Posted March 17, 2016
avatar
eiii: I hoped more for something outside of a browser, like a URL to a JSON file with the data or something like that. But I'll try to understand what you have written there and see if it helps me to extract the data. Thanks!
Unfortunately such a thing doesn't exist (yet). But you could probably whip up a script that would do what you wanted and host it somewhere easily enough.
What exactly were you looking to do? Scrape the promos to create a list of titles and prices?
Posted March 17, 2016
avatar
adaliabooks: What exactly were you looking to do? Scrape the promos to create a list of titles and prices?
Yes, a list of the game titles (or better the game card names as the titles are often written differently) and the prices to store them in a file where I can add other info and remarks to decide which games I'm going to buy at the end of the promo. The old promo pages allowed me to copy and paste the info directly from my browser, but that does not work anymore. And doing it manually is rather tedious, especially for promos with dozens of titles.
Posted March 17, 2016
avatar
eiii: Yes, a list of the game titles (or better the game card names as the titles are often written differently) and the prices to store them in a file where I can add other info and remarks to decide which games I'm going to buy at the end of the promo. The old promo pages allowed me to copy and paste the info directly from my browser, but that does not work anymore. And doing it manually is rather tedious, especially for promos with dozens of titles.
Hmm, something like that shouldn't be too difficult. If I get a chance I'll try and whip something up for you.
A userscript would probably be easiest to make (you could just run it and then paste the output where you wanted it) but I could put a page on my server with the rest of my script stuff that would do it for you and return a JSON output (or whatever format you wanted really). All you'd need to do is go to the url and maybe add the promo page url as a variable..
Posted March 17, 2016
avatar
adaliabooks: Hmm, something like that shouldn't be too difficult. If I get a chance I'll try and whip something up for you.
A userscript would probably be easiest to make (you could just run it and then paste the output where you wanted it) but I could put a page on my server with the rest of my script stuff that would do it for you and return a JSON output (or whatever format you wanted really). All you'd need to do is go to the url and maybe add the promo page url as a variable..
Thanks for the offer! But my goal is also to learn a bit how this browser and web stuff works. I'm good enough to put things together at the shell, to parse the MaGog data and make my lists from it. I just don't understand how all this dynamic web stuff works and I would like to change that at least a bit. Pure HTML is easy, you can get the file from the URL and work with it. But these dynamic, scripted pages are quite complex and I somehow do not find a good entry.
Posted March 17, 2016
avatar
eiii: Thanks for the offer! But my goal is also to learn a bit how this browser and web stuff works. I'm good enough to put things together at the shell, to parse the MaGog data and make my lists from it. I just don't understand how all this dynamic web stuff works and I would like to change that at least a bit. Pure HTML is easy, you can get the file from the URL and work with it. But these dynamic, scripted pages are quite complex and I somehow do not find a good entry.
I can understand that. Javascripts really not all that complicated. GoGs site can be a bit odd sometimes but once you figure out how things work it's easy enough.
I suppose for what you want you needn't even mess around with the Angular JS as you could just pull the values you want from the html using Javascript or JQuery.
If you need a hand just let me know and I'll see if I can help :)
Posted March 18, 2016
avatar
eiii: (...)
Paste this in your browsers console on promo page:

for (product of gogData.promoProducts) {
console.log(product.title + ': ' + product.price.amount);
}

I don't know how well that fits your needs, but maybe it will somehow guide you. :) You can retrieve that data from the HTML source code too - search for gogData.

edit: Adalia's way is also good for owned/not owned games. :)
Post edited March 18, 2016 by Johny.
Posted March 18, 2016
avatar
Johny.: Paste this in your browsers console on promo page:

for (product of gogData.promoProducts) {
console.log(product.title + ': ' + product.price.amount);
}

I don't know how well that fits your needs, but maybe it will somehow guide you. :) You can retrieve that data from the HTML source code too - search for gogData.

edit: Adalia's way is also good for owned/not owned games. :)
Ah, forgot to check the gogData variable on the promo page. That's handy to know too.
Thinking about it I overcomplicated it way too much, if all you want is the name and the price (as if you had copy and pasted it) there are much easier ways to do it then what I originally suggested.
My way was more for if you wanted to do some further processing on it in code (or pass the info back to the page, not really useful or relevant in this case).
Posted March 18, 2016
avatar
Johny.: You can retrieve that data from the HTML source code too - search for gogData.
Indeed, it's already embedded into the original URL. What a disgrace! I was just too blind. Not sure where I was looking the last time, probably only in the two .js files.

avatar
adaliabooks:
Thanks for your help, both of you!

A few lines of Python do the job for me:

import json
from lxml import html
import re
import requests
page = requests.get('https://www.gog.com/promo/weekly_staff_picks_git_gud_140314')
tree = html.fromstring(page.content)
script = tree.xpath('//script/text()')
m = re.match('^\n *var gogData = ({.*});\n', script[0])
j = json.loads(m.group(1))
for g in j['promoProducts']:
print(g['price']['amount'], g['slug'])
^^ That last line should be indented, but the forum software eats the leading spaces.

I just have started to learn Python so there's probably an easier way and the matching is ugly and not very robust, but at least it works. :)

Now I have to find out how to construct the cookies to change the language and currency and what the difference between price amount and price finalAmount is. But that's for tomorrow or so.

Edit: Cookie construction solved. :)

avatar
Johny.: Paste this in your browsers console on promo page:

for (product of gogData.promoProducts) {
console.log(product.title + ': ' + product.price.amount);
}
That also works, nice. And it's much easier.

I still prefer the "browser free" version as I can combine it with other scripts when needed, but I definitely have to learn more about how to hack around in the browser. :)
Post edited March 18, 2016 by eiii
Pages:
1
2
3
This is my favourite topic
Community
General discussiongog.com API(34 posts)(34 posts)
(34 posts)