Posted September 03, 2015
Over time whenever I've read reviews for games on GOG.com and looked at game ratings I've always felt like they were heavily skewed. We don't have access to the full underlying database or statistics but by observing the ratings and reviews over time I get the distinct impression that a lot of games get rated 5 stars by people for no other reason than that they are enthusiastically ecstatic to see the particular game even show up here on GOG and they just need an outlet to express their glee, so they go rate a game 5 stars perhaps because there is no "thumbs up" or "I think this is cool" or "I'm happy about this" button to click on. This is done for all kinds of games, both truly popular high-demand games with a big following that arguably deserve a high rating, and also for lesser known games that aren't really that great but have some type of historical or nostalgic value.
Additionally, people seem to rate the game based on how good it was when it first came out and their memories of the game from 20 years ago or whatever, and are not rating the actual GOG version of the game which may or may not be very different, may not run as well under modern Windows or may have other issues and they aren't necessarily even aware of it as they may have not even purchased the GOG version of the game. They're just purely excited about their memory of the game and to finally see it here so saying "5 stars: This game is fucking awesome!" even though they haven't bought or played the GOG version yet is their way of saying "thank you GOG, thank you game publisher, I'm happy you brought this here." rather than an actual reflection on the game's quality and the GOG build of the game.
The GOG review system does not currently indicate whether a person owns the game or not (as it does on Steam), nor how many hours they've played of the game if they do own it on GOG (as it does on Steam), both of which are important things to known when trying to gauge a individual person's review and assign a mental weight to it while contemplating whether or not to be interested in or buy a given game.
Additionally, people will purposefully pump up a game for a variety of reasons whether or not they own it or have ever played it or even care about it, such as the case in the Jack Keane 2 game in the Insomnia sale where people who never owned or played the game before gave the game great reviews because the game was selling too slowly causing the sale to grind to a halt and they hoped their good reviews would help the game sell faster so the promo would move along to another game. Nobody has any way of determining this, so the reviews are devalued greatly by being gameable like this.
Another thing I've observed is that people will down-rate a game and give it terrible reviews out of having anger or frustration or other negative emotions because they bought the game and experienced problems downloading or installing it, whether it was due to temporary server overload during a sale, their Internet being unreliable, the Great Firewall of China or some other reason that has nothing to do with the quality of the given game and how fun it is. Thus their review/rating just skews the data on the game rather than being a useful factual feedback about the actual game itself. It's like eating at a restaurant in a city and your food takes 10 minutes longer to get to the table so they rate the food as the worst food they've ever had, badmouth the restaurant, the owner, the city, the mayor, then burn the whole city to the ground because they're pissed off. It doesn't help other people to know whether that particular dish is tasty and enjoyable or not.
I've seen multiple people downrate various games simply because the game did not have their own local language supported, where their language is not even a language commonly supported by games at all and not in the top 10 or so of the most highly spoken languages, but they give the game a 1 star rating and complain because it doesn't support for Ancient Sumerian language during install or whatever.
People may have their own reasons they justify to themselves out of anger and frustration for making such crappy reviews and giving low ratings to games, or for giving overly good reviews/ratings to games they have not actually played or have not bought or played the GOG version, but the result is the same - all of these behaviours in the rating system end up devaluing the rating system itself and making game ratings/rankings/reviews on GOG.com next to useless for people making purchasing decisions.
I seriously hope that GOG has plans in the next 12-24 months to gut the review/rating system completely and throw it in the trash and implement a more modern system that lets people know whether someone has bought the GOG game or not and if so how many hours they've played of it (if they've chosen to opt into making that information available, or indicate "game time not available" otherwise), and completely ditch the useless 5 star rating system. A thumbs up or down system with a textual review indicating the person's ownership and time playing the game makes it easier to judge whether their words are of value to me when making a purchase decision. There are other things that can bring greater value to people's reviews and rankings also, but that's a start.
Until there is a better system in place, those of us who seek truly useful information on deciding whether or not we're going to purchase or wishlist or play a particular game are just going to have to seek that information elsewhere such as some combination of Steam, Metacritic and other online review systems and apply our own level of weight and trust to each piece of data.
What changes would others like to see happen with the GOG.com game review/rating system to make it more useful and less gameable?
Additionally, people seem to rate the game based on how good it was when it first came out and their memories of the game from 20 years ago or whatever, and are not rating the actual GOG version of the game which may or may not be very different, may not run as well under modern Windows or may have other issues and they aren't necessarily even aware of it as they may have not even purchased the GOG version of the game. They're just purely excited about their memory of the game and to finally see it here so saying "5 stars: This game is fucking awesome!" even though they haven't bought or played the GOG version yet is their way of saying "thank you GOG, thank you game publisher, I'm happy you brought this here." rather than an actual reflection on the game's quality and the GOG build of the game.
The GOG review system does not currently indicate whether a person owns the game or not (as it does on Steam), nor how many hours they've played of the game if they do own it on GOG (as it does on Steam), both of which are important things to known when trying to gauge a individual person's review and assign a mental weight to it while contemplating whether or not to be interested in or buy a given game.
Additionally, people will purposefully pump up a game for a variety of reasons whether or not they own it or have ever played it or even care about it, such as the case in the Jack Keane 2 game in the Insomnia sale where people who never owned or played the game before gave the game great reviews because the game was selling too slowly causing the sale to grind to a halt and they hoped their good reviews would help the game sell faster so the promo would move along to another game. Nobody has any way of determining this, so the reviews are devalued greatly by being gameable like this.
Another thing I've observed is that people will down-rate a game and give it terrible reviews out of having anger or frustration or other negative emotions because they bought the game and experienced problems downloading or installing it, whether it was due to temporary server overload during a sale, their Internet being unreliable, the Great Firewall of China or some other reason that has nothing to do with the quality of the given game and how fun it is. Thus their review/rating just skews the data on the game rather than being a useful factual feedback about the actual game itself. It's like eating at a restaurant in a city and your food takes 10 minutes longer to get to the table so they rate the food as the worst food they've ever had, badmouth the restaurant, the owner, the city, the mayor, then burn the whole city to the ground because they're pissed off. It doesn't help other people to know whether that particular dish is tasty and enjoyable or not.
I've seen multiple people downrate various games simply because the game did not have their own local language supported, where their language is not even a language commonly supported by games at all and not in the top 10 or so of the most highly spoken languages, but they give the game a 1 star rating and complain because it doesn't support for Ancient Sumerian language during install or whatever.
People may have their own reasons they justify to themselves out of anger and frustration for making such crappy reviews and giving low ratings to games, or for giving overly good reviews/ratings to games they have not actually played or have not bought or played the GOG version, but the result is the same - all of these behaviours in the rating system end up devaluing the rating system itself and making game ratings/rankings/reviews on GOG.com next to useless for people making purchasing decisions.
I seriously hope that GOG has plans in the next 12-24 months to gut the review/rating system completely and throw it in the trash and implement a more modern system that lets people know whether someone has bought the GOG game or not and if so how many hours they've played of it (if they've chosen to opt into making that information available, or indicate "game time not available" otherwise), and completely ditch the useless 5 star rating system. A thumbs up or down system with a textual review indicating the person's ownership and time playing the game makes it easier to judge whether their words are of value to me when making a purchase decision. There are other things that can bring greater value to people's reviews and rankings also, but that's a start.
Until there is a better system in place, those of us who seek truly useful information on deciding whether or not we're going to purchase or wishlist or play a particular game are just going to have to seek that information elsewhere such as some combination of Steam, Metacritic and other online review systems and apply our own level of weight and trust to each piece of data.
What changes would others like to see happen with the GOG.com game review/rating system to make it more useful and less gameable?
