Introduction / Abstract

A data scientist is one who manages to make data speak to them; it is basically a conversation, where you ask questions and the data answers.
In this notebook I want to share my latest conversation with this dataset of scores assigned to different video games. The goal is to show how the work as a data scientist develops over time.
Initially, I had an idea that recently, despite the video game industry growing and growing, the quality of games has been declining, so I decided to measure this effect. Hence comes question number 1… Where do I get data? The answer was Metacritic, which is a website that has the average critic score (metascore) for each game on each console, and also the score given by people using the site (userscore).
To work, I performed a web scraping of Metacritic (which took much longer than doing this analysis), where I extracted all reviews available as of March 23, 2018, but I did not include PC Master Race, because it had more games than all consoles combined.

Initial Descriptives

The goal is to become familiar with the data rather than to find an answer to the initial question. Personally, I prefer to leave the initial question for the end, so I have a more complete idea before looking at the numbers.

Dataset Description

11,548 complete records were extracted from Metacritic. Only games that have both a userscore and a metascore are considered, mainly because games that do not have these values are low-quality games that nobody paid attention to; they are generally shovelware or irrelevant.
The following data were extracted from Metacritic:

• platform: console (Xbox One, PlayStation 4, PlayStation 3, Xbox 360, Dreamcast, GameCube, PlayStation, DS, Wii, Game Boy Advance, Xbox, Wii U, PlayStation 2, PlayStation Vita, 3DS, PSP, Switch, Nintendo 64).
• product_title: Name of the game.
• release_date: Date the game was released.
• metascore: Average score assigned by critics.
• userscore: Average score assigned by users.

which was enriched with the following calculated fields:

• playscore: average of userscore and metascore, a metric used in some YouTube channels.
• critic_love: extra points given by critics (metascore – userscore).
• from_first_release: difference in days between the release date and the date of the first game on that console.

Number of Games

Number of Games by System

The 11,548 evaluated games are distributed as follows:
Older consoles clearly do not have their entire history.

ggplot(dataset,aes(platform)) +
geom_histogram(stat="count") +
theme(axis.text.x = element_text(angle = 45, hjust = 1))

Time Series

Number of Games Released Monthly

Since not all games are available for old consoles, we will work with the area inside the light blue rectangle, which corresponds to the period between 2000 and 2017 (inclusive).
We can see a peak in the number of games and seasonality.

aux = mutate(dataset,release_month = release_date – day(release_date) + 1) %>%
group_by(release_month) %>%
summarise(numero_releases = n())
ggplot(aux,aes(release_month,numero_releases)) +
annotate("rect",xmin = as.Date("2000-01-01"), xmax = as.Date("2017-12-31"), ymin = 0, ymax=Inf, fill = "blue", alpha = .3,) +
geom_line() +
geom_vline(xintercept = as.Date(c("1995-01-01","2000-03-04","2006-11-11","2017-02-22") )) +
annotate("text",x=as.Date(c("1995-01-01","2000-03-04","2006-11-11","2017-02-22") ) + 180, y = 150, label = c("ps1","ps2","ps3","ps4"))

dataset = filter(dataset, release_date >= as.Date("2000-01-01") & release_date <= as.Date("2017-12-31")) %>%
mutate(playscore = (userscore+metascore)/2,
critic_love = metascore – userscore) %>%
group_by(platform) %>%
mutate(from_first_release = as.numeric(release_date – min(release_date))) %>%
ungroup()

Average Games per Month

What is the detected seasonality? Let’s measure how many more releases per month than the annual average. This is done with a linear regression with dummy variables like: releases ~ month + year -1.
There is a considerable increase in the number of releases at the end of the year, probably to capture higher Christmas sales; then December and January are the months with the fewest releases.

aux = mutate(dataset,month = str_pad(month(release_date),2,pad="0"), year=as.character(year(release_date))) %>%
group_by(month,year) %>%
summarise(releases = n())
fit = lm(releases ~ month + year -1, aux)
aux = data.frame(month = str_extract(names(coef(fit)[1:12]),"[0-9]+$"), releases = coef(fit)[1:12] – min(coef(fit)[1:12]) )
ggplot(aux, aes(month,releases)) +
geom_bar(stat ="identity")

Lifespan of Each Console

On what dates did each console have releases, and how many?
PS4 by far had the highest number of releases in 2016 and 2017. On the other hand, SWITCH (only one point) had more releases in its first year than PS4 in its second year.
colors = brewer.pal(8, "Dark2")
colors = brewer.pal(8, "Dark2")
color_generator <- colorRampPalette(colors)
aux = mutate(dataset, release_year = as.character(year(release_date))) %>%
group_by(platform,release_year) %>%
summarise(releases = n())
ggplot(aux,
aes(release_year,releases,color = platform, group = platform)) +
scale_color_manual(values = color_generator(length(unique(aux$platform)))) +
theme(axis.text.x = element_text(angle = 45, hjust = 1)) +
geom_line(size=2) +
geom_point(size = 2)

Lifespan of Consoles, Cumulative Number of Releases by Console Years

The PS4 is by far the console that achieved the most games in its early years; on the other hand, the Wii U is the console with the fewest games.
The release behavior does not follow an S-curve, probably because many games are needed at the beginning to make the console attractive.

aux = mutate(dataset, release_year = cut(from_first_release, seq(from = 0, to = 301210, by = 360), include.lowest = T, labels = as.character(1:10) )) %>%
group_by(platform,release_year) %>%
summarise(releases = n()) %>%
arrange(platform,release_year) %>%
mutate(cum_releases = cumsum(releases))
seleccionadas = filter(aux, release_year == "4") #Que tengan 4 años de juegos
ggplot(subset(aux, platform %in% seleccionadas$platform),
aes(release_year,cum_releases,color = platform, group = platform)) +
scale_color_manual(values = color_generator(length(unique(aux$platform)))) +
geom_line(size=2) +
geom_point(size=2)

Annual Releases

How many releases are there in total each year?
The peak corresponds to the year 2009, but it is not the year with the most games, only where seasonality was most concentrated, possibly corresponding to a change in strategy. If many games come out together, that channels sales.
After the peak in 2009, there is a recession that begins to recover in 2013.
The launch of the Nintendo Switch on one hand explains the peak in 2016, and also took presence away from the leader PlayStation 4. On the other hand, the Switch’s first year had more releases than the PS4’s first year, so in 4 more years it should break the PS4’s release record.

aux = mutate(dataset,release_year = as.character(year(release_date)))
ggplot(aux,aes(release_year, fill = platform)) +
geom_bar(stat="count") +
theme(axis.text.x = element_text(angle = 45, hjust = 1)) +
scale_fill_manual(values = color_generator(length(unique(dataset$platform)))) +
ylab("Number of Releases")

Market Share of Releases by System

What is the market share in number of releases by system?
When comparing the first year’s market share of the Switch, it looks larger than that of the PS4, so in 3 more years it should have as many releases as the PS4 in 2016.
Nintendo Wii, despite having high sales, did not have as many games, probably due to the shape of its controllers.

ggplot(aux,aes(release_year, fill = platform)) +
geom_bar(position="fill") +
theme(axis.text.x = element_text(angle = 45, hjust = 1)) +
scale_fill_manual(values = color_generator(length(unique(dataset$platform)))) +
ylab("Market Share")

Number of Ports per Game

How many systems does each game come out for?
Most games are exclusives.

aux = group_by(dataset,product_title) %>% summarise(sistemas = as.character(n()))
ggplot(aux,aes(sistemas)) + geom_histogram(stat="count")

Evolution of the Number of Games with Ports

Has the number of games with ports increased as consoles become more powerful and there are more APIs, thereby allowing multi-platform development?
Yes, it has increased, but since the PS2 era, especially for games released on 2 systems, probably because there are always 2 similar consoles on the market.

aux = mutate(dataset, year = year(release_date)) %>%
group_by(year,product_title) %>%
summarise(sistemas = as.character(n()))
ggplot(aux,aes(year, fill = sistemas)) +
geom_bar(position="stack") +
theme(axis.text.x = element_text(angle = 45, hjust = 1)) +
geom_vline(xintercept = c(2001,2007,2014) ) +
ylab("Number of releases") +
annotate("text",x=c(2001,2007,2014)+.5, y = .9, label = c("ps2","ps3","ps4"))

Score Analysis (metascore and userscore)

Console Analysis

Average Scores by Console

Are the games on certain consoles better rated by users or critics?
We can see that the consoles with the best games are the old ones (Dreamcast, Nintendo 64 and PlayStation 1). On Metacritic, only the best and most popular games for these systems have scores. To balance this, we will work with the top 50 games of each system. This makes sense, because when people choose among that list, the rest of the games are unknown to the average consumer.

dataset = readRDS("scores.rds")
aux = rbind(group_by(dataset,platform) %>% summarise(mean_value = mean(metascore), score = "metascore"),
group_by(dataset,platform) %>% summarise(mean_value = mean(userscore), score = "userscore"))
ggplot(aux,aes(platform,mean_value, fill=score)) +
geom_bar(stat="identity", position = "dodge") +
theme(axis.text.x = element_text(angle = 45, hjust = 1))+
scale_y_continuous(limits = c(60, 80),oob=rescale_none)

Scores of the Top 50 Games by Console

We can see that in general, metascores are higher than userscores; this only does not hold for Game Boy Advance and PSP. On the other hand, on Wii, Wii U and 3DS, the userscore is very close to the metascore.
Moreover, the latest generation consoles (PS4 and Xbox One) are the worst rated by users, while critics remain constant.

dataset50 = group_by(dataset, platform) %>% top_n(50,playscore)
aux = rbind(group_by(dataset50,platform) %>% summarise(mean_value = mean(metascore), score = "metascore"),
group_by(dataset,platform) %>% summarise(mean_value = mean(userscore), score = "userscore"))
ggplot(aux,aes(platform,mean_value, fill=score)) +
geom_bar(stat="identity", position = "dodge") +
theme(axis.text.x = element_text(angle = 45, hjust = 1)) +
scale_y_continuous(limits = c(60, 95),oob=rescale_none)

Correlation between userscore and metascore

Is there a correlation between userscore and metascore?
The correlation between the two for the top 50 games per console is 0.076. Below we present the correlation by system, where we detect that in general there is a negative correlation between userscore and metascore.

aux = group_by(dataset50,platform) %>%
summarise(correlacion_ms_us = cor(userscore,metascore))
ggplot(aux,aes(platform,correlacion_ms_us)) +
geom_bar(stat="identity", position = "dodge") +
theme(axis.text.x = element_text(angle = 45, hjust = 1))

critic_love by System

Is there a preference when evaluating games on a system?
critic_love is the number of extra points that critics give compared to users.
It is interesting to see that critic_love is similar for: Nintendo 64 with PlayStation, PlayStation 2 with Xbox and GameCube, PlayStation 3 with Xbox 360, and PlayStation 4 with Xbox One, corresponding to consoles of the same generations. This could be due to multi-platform games generating correlation or simply a seasonal trend.

aux = group_by(dataset50,platform) %>%
summarise(mean_critic_love = mean(critic_love))
ggplot(aux,aes(platform,mean_critic_love)) + geom_bar(stat="identity") + theme(axis.text.x = element_text(angle = 45, hjust = 1))

Which is the best console?

This is a difficult question; there are several ways to compare consoles: price, number of games, or quality of games.

Average userscore for the top n games

If we look at the top n games, which console has the highest scores?
If you are going to have only 1 game, the best consoles are GameCube and PS4. If you are a bit more of a gamer and will have about 5 games, a PS1 is the option. After that, the king is PS2.
Although it is worth noting that there are other factors such as how old the console in question is. For example, a PS1 can be a bit disappointing in terms of graphics.
On the contrary, the Nintendo 64 seems to be by far the worst console for intensive gamers; probably that is why the PSX managed to dethrone Nintendo.

aux = group_by(dataset, platform) %>%
arrange(-userscore) %>%
summarise("01" = mean(userscore[1]),
"02" = mean(userscore[1:2]),
"03" = mean(userscore[1:3]),
"05" = mean(userscore[1:5]),
"10" = mean(userscore[1:10]),
"15" = mean(userscore[1:15]),
"20" = mean(userscore[1:20]),
"30" = mean(userscore[1:30]),
"40" = mean(userscore[1:40]),
"50" = mean(userscore[1:50])) %>%
gather(numero_juegos,userscore,-platform)
ggplot(aux,aes(platform,userscore, fill = numero_juegos)) +
geom_bar(stat = "identity",position="nudge") +
theme(axis.text.x = element_text(angle = 45, hjust = 1)) +
scale_fill_manual(values = color_generator(10)) +
scale_y_continuous(limits = c(min(aux$userscore), 95),oob=rescale_none)

Difference in userscore for the same game

Is there any difference in userscore for the same game on different consoles? This is interesting because it could mean that it is easier to program on one console than another.
We will focus on games that are on at least 3 systems. There we will see the average difference (reward or penalty) that each console’s userscore has compared to the game average. We will filter out consoles that do not have at least 30 games to compare.
We can see that the most penalized are Nintendo’s portable consoles, DS and 3DS, probably because they tend to be reduced versions of the other games. On the other hand, the consoles with the best ports are the Wii U, then PlayStation 2 and PS Vita.

multisistema = group_by(dataset,product_title) %>% summarise(sistemas = n()) %>% filter(sistemas >= 3)
aux = filter(dataset, product_title %in% multisistema$product_title) %>%
group_by(product_title) %>%
mutate(userscore_premium = userscore-mean(userscore)) %>%
group_by(platform) %>%
summarise(userscore_premium = mean(userscore_premium), numero = n()) %>%
filter(numero > 30)
ggplot(aux,aes(platform,userscore_premium)) + geom_bar(stat="identity") + theme(axis.text.x = element_text(angle = 45, hjust = 1))

Temporal Analysis of Scores

Evolution of the average userscore for the top n games

Is there a change in the average userscore of games over time? (the initial question)
To perform this analysis, we group ports into a single game with their average score.
We see that the top games of the year maintain a score with no apparent trend, but for games from number 15 onward, we detect that the userscore has decreased year by year.

aux = filter(dataset, release_date >= as.Date("2001-01-01") & release_date <= as.Date("2017-12-31")) %>%
mutate(release_year = year(release_date)) %>%
group_by(release_year, product_title) %>%
summarise(userscore = mean(userscore)) %>%
group_by(release_year) %>%
arrange(-userscore) %>%
summarise("01" = mean(userscore[1]),
"02" = mean(userscore[1:2]),
"03" = mean(userscore[1:3]),
"05" = mean(userscore[1:5]),
"10" = mean(userscore[1:10]),
"15" = mean(userscore[1:15]),
"20" = mean(userscore[1:20]),
"30" = mean(userscore[1:30]),
"40" = mean(userscore[1:40]),
"50" = mean(userscore[1:50])) %>%
gather(nth_score,userscore,-release_year)
ggplot(aux,aes(release_year,userscore, fill = nth_score, group = nth_score)) +
geom_area(size = 1.5, position = "nudge") +
theme(axis.text.x = element_text(angle = 45, hjust = 1)) +
scale_fill_manual(values = color_generator(10)) +
scale_y_continuous(limits = c(min(aux$userscore), 95),oob=rescale_none) +
geom_vline(xintercept = c(2001,2007,2014) ) +
annotate("text",x=c(2001,2007,2014)+1, y = 95, label = c("ps2","ps3","ps4"))

Evolution of the average metascore for the top n games

Does the change in userscore extrapolate to metascore?
No, the metascore has remained relatively constant and without a trend over time.

aux = filter(dataset, release_date >= as.Date("2001-01-01") & release_date <= as.Date("2017-12-31")) %>%
mutate(release_year = year(release_date)) %>%
group_by(release_year, product_title) %>%
summarise(metascore = mean(metascore)) %>%
group_by(release_year) %>%
arrange(-metascore) %>%
summarise("01" = mean(metascore[1]),
"02" = mean(metascore[1:2]),
"03" = mean(metascore[1:3]),
"05" = mean(metascore[1:5]),
"10" = mean(metascore[1:10]),
"15" = mean(metascore[1:15]),
"20" = mean(metascore[1:20]),
"30" = mean(metascore[1:30]),
"40" = mean(metascore[1:40]),
"50" = mean(metascore[1:50])) %>%
gather(nth_score,metascore,-release_year)
ggplot(aux,aes(release_year,metascore, fill = nth_score, group = nth_score)) +
geom_area(size = 1.5, position = "nudge") +
theme(axis.text.x = element_text(angle = 45, hjust = 1)) +
scale_fill_manual(values = color_generator(10)) +
scale_y_continuous(limits = c(min(aux$metascore), 100),oob=rescale_none) +
geom_vline(xintercept = c(2001,2007,2014) ) +
annotate("text",x=c(2001,2007,2014)+1, y = 100, label = c("ps2","ps3","ps4"))

Evolution of critic_love for the top 100 games each year

So are critics becoming less critical?
We took the top 100 games and grouped multi-platform games into one, to avoid overweighting.
We can see that since the PlayStation 3 came out, critics have become less critical compared to players.

dataset100 = filter(dataset, release_date >= as.Date("2001-01-01") & release_date <= as.Date("2017-12-31")) %>%
mutate(release_year = year(release_date)) %>%
group_by(product_title,release_year) %>%
summarize(critic_love = mean(critic_love), userscore = mean(userscore), metascore = mean(metascore)) %>%
group_by(release_year) %>%
top_n(100,userscore)
aux = group_by(dataset100,release_year) %>%
summarise(mean_critic_love = mean(critic_love))
ggplot(aux,aes(release_year,mean_critic_love)) +
geom_line() +
theme(axis.text.x = element_text(angle = 45, hjust = 1)) +
geom_vline(xintercept = c(2001,2007,2014) ) +
annotate("text",x=c(2001,2007,2014)+1, y = 5, label = c("ps2","ps3","ps4"))

Evolution of the correlation between userscore and metascore for the top 100 games

Was there ever a time when critics were aligned with consumers?
No, the correlation has always been low…

aux = group_by(dataset100,release_year) %>%
summarise(correlacion_ms_us = cor(userscore,metascore))
ggplot(aux,aes(release_year,correlacion_ms_us)) +
geom_line() +
theme(axis.text.x = element_text(angle = 45, hjust = 1)) +
geom_vline(xintercept = c(2001,2007,2014) ) +
annotate("text",x=c(2001,2007,2014)+1, y = 1, label = c("ps2","ps3","ps4"))

Conclusions

Indeed, the average user evaluation has decreased on average, but for the best games of the year it has remained constant. In other words, if you are a person who plays many games, you probably feel that the past was better, while if you are a person who plays 2 or 3 games a year, you have not noticed a change.
This leaves us with several hypotheses. One is that gamers may be getting bored with game series, or that developers are spending more resources on favors to critics than on their games. It could also be that, given how expensive development is today, companies do not dare to innovate much.
Does anyone else have more hypotheses or ways to answer them?
Greetings!
And as usual, I share the dataset and the R notebook at the following link: metacritic