<\/p>\n
Our fundamental assumption is that the userscore is the real evaluation of video games, while the metascore is the score altered by the influences of the mega corporations that make games. In this article we will show how we obtained the information from Metacritic and then how we evaluated and statistically tested the "generosity of reviewers."<\/p>\n
The Available Data<\/h3>\n
As I said before, the site has average scores and details. For our purposes we will download for PS4 games the following data:<\/p>\n
- \n
- MetaScore<\/li>\n
- UserScore<\/li>\n
- Company that publishes the game<\/li>\n
- MetaScore detail (score given by each reviewer)<\/li>\n<\/ul>\n
For example, for the case of FIFA 17, we would see something like this on the page:<\/p>\n
![\"\"](\"https:\/\/i0.wp.com\/www.geekosas.com\/wp-content\/uploads\/2017\/02\/fifa-300x121.jpg?resize=937%2C378\")
<\/p>\nGenerating for each game a record with this information (I hid the key I will use to cross\u2011reference with the detail table):<\/p>\n
![\"\"](\"https:\/\/i0.wp.com\/www.geekosas.com\/wp-content\/uploads\/2017\/02\/tabla-userscore-300x43.jpg?resize=1033%2C148\")
<\/p>\nOn the other hand, below we can see the detail of the reviews containing the score each reviewer gave to a particular game. In this case, all the reviews for FIFA 17:<\/p>\n
![\"\"](\"https:\/\/i0.wp.com\/www.geekosas.com\/wp-content\/uploads\/2017\/02\/review-fifa17-300x187.jpg?resize=967%2C603\")
<\/p>\nData Extraction<\/h3>\n
To extract the data we will use Selenium through Python. Selenium is a controller that allows you to control a browser \u2013 either Firefox, Chrome or PhantomJS \u2013 through code written in a language like Python.<\/p>\n
To keep things simple, we will extract the data to a set of .csv files, each containing the information available for 100 games.<\/p>\n
The code is as follows:<\/p>\n
import pandas as pd\nimport time\nimport sys\nfrom selenium import webdriver\nfrom selenium.webdriver.common.keys import Keys\n\ndriver = webdriver.Chrome()\n#driver = webdriver.PhantomJS()\n\nurl_generica = "http:\/\/www.metacritic.com\/browse\/games\/score\/metascore\/all\/ps4\/filtered?sort=desc&page={}"\n\ndef get_game(url):\n print("GoTo " + url)\n driver.get(url)\n game = driver.find_element_by_xpath('\/\/h1[@class="product_title"]\/a[@class="hover_none"]\/span[@itemprop="name"]').text\n print(game)\n registro = pd.DataFrame({\n "game" : [game],\n "publisher" : [driver.find_element_by_xpath('\/\/*[@itemprop="publisher"]\/span[@class="data"]').text],\n "devel" : [driver.find_element_by_xpath('\/\/*[@class="summary_detail developer"]\/span[@class="data"]').text],\n "metascore" : [driver.find_element_by_xpath('\/\/*[@itemprop="ratingValue"]').text],\n "userscrore" : [driver.find_element_by_xpath('\/\/div[@class="userscore_wrap feature_userscore"]\/a[@class="metascore_anchor"]\/div').text],\n "metascore_url" : [url + "\/critic-reviews"],\n "userscore_url" : [url + "\/user-reviews" ],\n "game_url" : [url]\n })\n print("Metascore Scrapped")\n return(registro)\n\ndef get_metascore_detail(url):\n driver.get(url)\n print("Scrapping Details")\n registro = pd.DataFrame({\n "detail_metascore_value" : [x.text for x in driver.find_elements_by_xpath('\/\/div[@class = "module reviews_module critic_reviews_module"]\/\/div[@class = "review_grade"]')],\n "detail_metascore_source" : [x.text for x in driver.find_elements_by_xpath('\/\/div[@class = "review_critic"]\/div[@class="source"]')]\n })\n registro["metascore_url"] = url\n print("Details Scrapped")\n return(registro)\n\npaginas = [x for x in range(8)]\nwhile len(paginas) > 0:\n p_num = paginas.pop()\n url = url_generica.format(p_num)\n print(url)\n driver.get(url)\n games_urls = driver.find_elements_by_xpath('\/\/*[@class="product_item product_title"]\/a')\n games_urls = [l.get_attribute("href") for l in games_urls]\n\n compiled_metacritic = pd.DataFrame()\n compiled_detail = pd.DataFrame()\n while len(games_urls) > 0:\n print("********* NEW SCRAP ********")\n url = games_urls.pop()\n compiled_metacritic = compiled_metacritic.append(get_game(url))\n compiled_detail = compiled_detail.append(get_metascore_detail(url + "\/critic-reviews"))\n print("********* END SCRAP ********")\n compiled_metacritic.to_csv("metacritic\/metacritic_" + str(p_num) + ".csv", index = False)\n compiled_detail.to_csv("detail\/metacritic_detail" + str(p_num) + ".csv", index = False)<\/code><\/pre>\nAnalysis<\/h3>\n
The first thing is to look at the data in general. What we will do first is to see at the aggregate level the userscore versus the metascore. As can be seen in the table below, on average the userscore is 4.26 higher than the metascore, which means that generally there is a tendency for users to be more demanding than reviewers.<\/p>\n
\n\n
\n \npromedio metascrore<\/th>\n promedio userscore<\/th>\n diferencia<\/th>\n<\/tr>\n<\/thead>\n \n 71.53<\/td>\n 67.27<\/td>\n 4.26<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n By performing a t\u2011test, we obtain a p\u2011value that allows us to statistically reject the null hypothesis that the means are equal. In other words, there is statistical evidence to assert that the userscore is lower than the metascore:<\/p>\n
![\"\"](\"https:\/\/i0.wp.com\/www.geekosas.com\/wp-content\/uploads\/2017\/02\/test-t-300x104.jpg?resize=1056%2C366\")
<\/p>\nNow we will perform the same test but for each publisher using R. The code is as follows:<\/p>\n
options(stringsAsFactors = FALSE)\nSys.setenv(TZ='GMT')\nlibrary(plyr)\nlibrary(reshape2)\nlibrary(xtable)\n\nread_csv_folder = function(folder){\n files = dir(folder)\n ldply(files, function(x) read.csv(paste0(folder,"\/",x)), .progress = "text")\n}\n\nmetacritic = read_csv_folder("metacritic")\nmetacritic = transform(metacritic, metascore = as.numeric(metascore), userscrore = 10*as.numeric(userscrore))\nmetacritic = metacritic[complete.cases(metacritic),]\nhead(metacritic[,c("game","publisher","metascore","userscrore")])\n\ndetails = read_csv_folder("detail")\ndetails = transform(details, detail_metascore_value = as.numeric(detail_metascore_value))\ndetails = details[complete.cases(details),]\n\n# General\ngral = data.frame(promedio_metascrore = round(mean(metacritic$metascore),2),\n promedio_userscore = round(mean(metacritic$userscrore),2),\n diferencia = round(mean(metacritic$metascore),2) - round(mean(metacritic$userscrore),2)\n)\nwrite.csv2(gral, "clipboard")\n\n# Publisher\nmetacritic_stats = ddply(metacritic, "publisher", function(x) data.frame(\n metascrore = mean(x$metascore),\n userscore = mean(x$userscrore),\n diferecia = mean(x$metascore - x$userscrore),\n p_value = tryCatch(t.test(x$metascore, x$userscrore, alternative = "two.sided")$p.value, error = function(e) NA),\n cases = nrow(x)))\nmetacritic_stats = subset(metacritic_stats, cases > 5)\nnrow(metacritic_stats)\nmetacritic_stats = subset(metacritic_stats, p_value <= 0.05)\nmetacritic_stats = metacritic_stats[order(metacritic_stats$p_value, decreasing = T),]\nprint(metacritic_stats)<\/code><\/pre>\nIn the following table we can see that when performing the same t\u2011test for publishers with at least 5 games, out of the 26 that meet the condition, 7 have a significant p\u2011value, which allows us to conclude that their games statistically have a metascore higher than the userscore, or in other words, something makes these games liked more by reviewers than by those who play them… strange, maybe beta games are better than final versions? haha.<\/p>\n
![\"\"](\"https:\/\/i0.wp.com\/www.geekosas.com\/wp-content\/uploads\/2017\/02\/pvalues-empresas1-300x71.jpg?resize=1154%2C273\")
<\/p>\nWhich publishers have more "generous" reviewers? To find out, we will compare the difference between the metascore and the userscore for each publisher\u2011reviewer, testing that the value is 0. If the null hypothesis is rejected, we can affirm that the value is different from 0. Then we will see, for each company, what percentage of reviewers are generous. The result in the table below shows that the companies from the list above have a very good relationship with reviewers. For example, 100% of the companies that review EA Sports and Konami give statistically higher scores to games than users do.<\/p>\n
comp = merge(metacritic, details, by = "metascore_url", all.x = T)\n\n# Reviewer with largest error\nreviewer_stats = ddply(comp, c("publisher", "detail_metascore_source"), function(x) data.frame(\n error = mean(x$metascore - x$userscrore),\n p_value = tryCatch(t.test(x$metascore - x$userscrore)$p.value, error = function(e) NA),\n cases = nrow(x)))\nreviewer_stats = subset(reviewer_stats, cases >= 5)\nreviewer_stats_numero_evaluado = ddply(reviewer_stats, "publisher", function(x) data.frame(total_reviewers = nrow(x)))\nreviewer_stats = subset(reviewer_stats, p_value <= 0.05 & error > 0)\nreviewer_stats_numero_afectado = ddply(reviewer_stats, "publisher", function(x) data.frame(reviewers_generosos = nrow(x)))\nreviewer_stats = merge(reviewer_stats_numero_evaluado, reviewer_stats_numero_afectado, all.x = T)\nreviewer_stats[is.na(reviewer_stats)] = 0\nreviewer_stats$porcentaje_generoso = round(100 * reviewer_stats$reviewers_generosos \/ reviewer_stats$total_reviewers)\narrange(reviewer_stats, -porcentaje_generoso)<\/code><\/pre>\n![\"\"](\"https:\/\/i0.wp.com\/www.geekosas.com\/wp-content\/uploads\/2017\/02\/reviewers-generosos-300x183.jpg?resize=987%2C602\")
<\/p>\nConclusions<\/h3>\n
We can conclude that there is indeed a relationship between reviewers and publishers that can confuse us when choosing games. Considering that games cost approximately 60 dollars, I would recommend waiting to see what other users say before buying a game, especially for large companies such as: Sierra Games, Konami, Ubisoft, Electronic Arts, EA Sports, Zen Studios, and Activision.<\/p>\n
I hope you liked it and share it! Greetings!<\/p>\n
Attachments: bd_juegos<\/a><\/h3>\n","protected":false},"excerpt":{"rendered":"