Advanced Retention Metrics in Free-To-Play
What is Retention?
You can think of retention as a funnel. You have all players, say N number of players at the top of the funnel. This starting point is when they install the game and launch it for the first time. They enter the game and eventually close it. They've tried the game, but will they come back and continue playing? Retention metrics are designed to measure how well a game can get its players to return to the game tomorrow, in a week, in a month, and even years later.
Now we will look at the most important aspects of retention metrics that game developers need to understand.
Benchmarks
To understand the meaning of retention metrics, you need some retention benchmarks from other games. With these metrics, you can understand whether your new game is doing well or badly and how far you are from other games that live in the same game genre.
A few resources for obtaining retention metrics are GameAnalytics, SensorTower, and AppAnnie reports, which they usually publish annually.
We will look at benchmarks for various retention metrics within each of the metrics listed below.
Statistical Significance
For retention metrics to be accurate, your game needs to be played by hundreds or even thousands of players. To get new retention and playtime metrics, you can count on a few hundred players. However, ensuring data accuracy and reliable storage requires professional software development. When you start looking at players returning a week after the first day or even a month later, you will need over a thousand players in your cohorts.
The reason long-term retention metrics require larger cohorts is that more and more people will leave the game every day. If you have a cohort of 100 players on day 0, after 30 days you may only have a few players still playing. Statistical significance will not provide sufficient accuracy with only a few players, as it may turn out that these players do not play every day, and the numbers will start to look strange.
But if you have 1000 players on day 0, then after a month you should have dozens still playing, and accuracy will be much higher.
Cohorts
When you start studying retention metrics, it is very important to consider players as cohorts. Cohorts are groups of people with certain similar characteristics. Here we will break down some of them.
Let's say 1,000 players play your game. To look at their retention metrics, you want to look at players who started the game with the same version of the game and came from the same country. Why is this important?
The game version must be the same, because if you made changes to the game and released an update, then the changes in the game will affect player behavior. They may become better or worse. To ensure data accuracy, look at players who started the game with the same version.
Grouping by country is very important, as game preferences may differ in some countries. Here are a few reasons why this may happen: A) Menus and texts in English will not hinder a player, unlike the desire to play localized content. B) Public holidays and days off work may differ between countries. When people have more free time, they will play more games.
What can go wrong if you neglect cohort-based measurement? You may confuse players who are further along in the game with players who just started. If we take the next metric we will discuss - Day-0 playtime - developers might measure the playtime of all their players and use it as a metric.
Another area where developers often struggle is worrying about small cohorts when the game gets fifty to a hundred new players daily. However, by summing up the number of players over a week, they can start looking at a weighted average for a cohort of players. It is necessary to ensure that players entered the game under the same conditions, such as by game version and country. To build such an analytics system with support for large data volumes, we recommend using scalable web service development.
Day-0 Playtime
Before starting to define retention, you need to look at Day-0 session time. This is the first indicator of how likely people are to return to the game. Day-0 is the day when a player first enters the game. We measure player engagement at the early stages of the game.
In 2018, Google Play released a report on player retention. The report showed that Day-0 playtime is the highest indicator of player return. The more time a player spends in the game on the first day, the more likely they are to come back. The trend was similar, from games with high metrics to games with low metrics.
As seen in the figure above, an excellent benchmark for Day-0 playtime is achieving 10 minutes of average Day-0 playtime.
To calculate Day-0 playtime, take a cohort of players of the same version of the game and look at the total average time that a player plays on Day-0.
Here is a simplified example with four players. In real life, you will need hundreds of players to achieve statistical significance. Automating the collection and processing of such data often requires custom software development. Note: For statistical significance of Day-0 metrics, you will need fewer players because on Day-0 there will still be 100% of players.
Day-1 Retention
Day-1 retention is a cornerstone metric for free-to-play games. A player starts playing a new game and decides to return to it the next day. When you have a large cohort of players, at least over 500 people, you can measure their Day-1 retention with statistical significance.
To get the Day-1 retention metric, you take a cohort and look at how they return to the game one day after they started playing.
The cohort must return exactly the day after Day-0. The reason for this strict rule is simple. If you calculate so-called "rolling retention," where you look at Day-1 and include all subsequent days in the metric, you will get a metric that gets better each subsequent day as more people return later. The strict rule: if you look at the day after Day-0, i.e., Day-1, you will have an exact and final number for decision-making.
These metrics are critically important, so it is highly recommended to rely on accurate analytical tools, which require reliable software development.
Here are some benchmarks for the first day in mobile games, which are almost the same for all game genres.
- Less than 30% return on the first day is bad. You are losing a huge number of players who do not want to continue playing.
- Between 30% and 39% return on the first day is not the end of the world. If you make significant improvements to the game, you can push the metrics to improve.
- Between 40% and 45% is the industry standard for a decent Day 1 retention rate. If you achieve this, you have something interesting in your hands that players will want to play.
- A rate above 45% is excellent, and you can move on to optimizing retention for subsequent days such as Day-3 and Day-7.
Day-3 to Day-1 Retention Ratio
Once you have convincing Day-1 metrics, at least over 40%, you can start paying attention to Day-3. Once you are sure that players have received enough content to enjoy Day-3, pull out the Day-3 retention numbers.
When you have Day-3 metrics, compare the ratio of Day-3 metrics to Day-1 metrics for the same cohort. This ratio is important because it gives you an idea of how well players are staying in the game. This ratio works similarly to the Day-0 playtime ratio, as it is an indicator of future retention metrics.