Introduction to analytics

You know the situation when you go to a forum or the comments to a post and there is "The developers don't play their game! The whole balance is reduced to 0.5 here, and 0.5 there! There are only the event kids!" And immediately you realize... I'm home. There's some fun in that. But still, we wish we understood each other better because not everything is as clear as it seems at first glance. So let's bust these and some other myths today and maybe we get closer to each other. © Tolmach, The Bar Negotiator.

To understand all aspects of the game, we can use the data collected daily from everyone who logs into the game. How much money do players spend on purchasing consumables? How much do they bring back from raids? Where do they have the worst lag?

Daniel "Highnoon" Kornev, the head of the Analytics Department, shared some information about the work and interesting facts about the players. You could see him in the First Developer Diaries.

This article will let you look at STALCRAFT: X as a developer and help you understand how developers think, what guides their decisions, and how they track reactions to change.

In this article you will learn:

• The difference between statistics and subjective game experience
• The emergence of analytics in the game
• Player Feedback
• Gear
• Economy
• Analytics and leveling design
• Conclusion

The difference between statistics and subjective game experience

Have you ever felt that the decisions made by the developers defy all logic? In other words, why do they nerf an already weak weapon (which I can't kill anyone with) and upgrade an already strong one (which I get killed with all the time)?

The statistically checked majority usually does not take 30 seconds to think and immediately gives answers about the professional suitability of balancers and swears at them. Let's figure out briefly the real reason for the prevalence of such misconceptions and justify our favorite balancers.

The frequency illusion

It most often shows values that are opposite to the subjective perception of individual players. Why does this happen? Because people by nature tend not only to exalt their own experience over all others but also often fall victim to their cognitive distortions. In other words, we like to deceive ourselves.

One of these tricks is often found in everyday life and is called the frequency illusion or the Baader-Meinhof phenomenon. If you begin to attach meaning to any information, your brain will happily cling to any confirmation of it.

For example, if you imagine the number 47 as something magical and significant, you will start noticing it everywhere. The root of omens, horoscopes, and superstitions is connected with this peculiarity of the human brain. The stronger a person believes in his hypothesis, the more actively he or she pays attention to any coincidences confirming it.

For instance, if I get kicked all day by players with an AMB-17, it doesn't mean it happens to most people at my level of matchmaking. Then I die and feel sad. But when I get killed with other weapons, I don't pay attention to it.

AMB-17 is a popular weapon, but not all of its owners are always winners. And if you look at a selection of at least a few thousand masters, it will turn out that the owners of other assault rifles kill not less often.

The experience of each particular player is important, but it is always subjective and may not coincide with the real state of affairs. Analytics, on the other hand, summarises many subjective experiences together in an attempt to be objective, as each unit in the statistics is the experience of a particular player. Since one player's impressions may differ from another player's impressions sometimes dramatically, these impressions put together form a picture that is as close to the real state of affairs as possible.

The emergence of analytics in the game

In any online game, analytics starts with the collection of data about the state of servers and online on them. STALCRAFT is not different. Because data collection was not as deep and comprehensive as it is now, many processes were carried out quite differently. For example, our much-loved weapon balance.

The balancer had his vision of each weapon: he wanted each weapon to be unique in its own way, but it was hard to achieve that. The audience was much smaller, and the balancer mainly focused on his feelings about the weapons (he played a lot with different models himself) and players' feedback. Then he looked at the metrics of the popularity of this or that weapon and based on that he made fixes.

Since then, the metrics have been analyzed more carefully, taking into account popularity at different levels of distribution, the effectiveness of players with weapons (KD ratio), the number of kills from weapons, and in which game situations their wielders win and in which ones they lose battles.

The next turning point for analytics was Perestroika. At that time, it was important to create all conditions to track the viability of the newly created barter model.

To do this, they began to track not only who was farming what, but also at what rate. This was part of a larger project. To add factions and then maintain them. At that time, they also began measuring the effectiveness of the newly released ‘parcel delivery’.

As the department evolved, tasks began to move away from manual data collection to creating automated systems to track data regularly. Analysts began to work more closely with game designers.

Since then, every new feature has been reflected in the data to track it comfortably. Even though there was already a lot of data, developers were always short of it.

Gradually, the system grew to its current, gigantic scale and continues to evolve. Now analysts collect a huge amount of data: players' routes, used gear, and who farms, sells, and buys something.

Of course, the stability of the game is also monitored for every aspect: ping, lags, crashes, as well as on which locations it happens, how often, and for what reasons. In general, Big Brother is watching.

Keeping track of everyone individually is not very interesting, it gives you almost nothing. We use large data sets, analysing them as a percentage of the total number of players or other user groups. The more people play, the more reliable the data is, because the law of large numbers starts to work. © Highnoon

We will not go into the definition of this law. In simple words, it is explained as follows: the more data and the more often it is measured, the more reliable the calculations are.

This is how the whole game design works. The developer builds expected behavior and expected performance into each of his ideas.

After the tests, the analyst compares the planned results with the actual results and adjusts the mechanics if necessary. It is important to note that it is the data from analytics that he gets the most representative data. Which better shows players' attitudes towards mechanics than any words on resources. That's why the data collected by analytics is so important for developers.

Player Feedback

It is possible to think that it's all about pure statistics and tonnes of numbers, but it's not.

Perfect balance in terms of maths, although it is an important part of any game, does not make the game interesting and fun by itself. Or it does, but not for long because it doesn't change. A very important part of the game is to enjoy the process: healthy interest, positive emotions, and fun. All of these are maintained when we overcome and transcend, in other words, when we are in the gap between too difficult and too easy.

Mathematically correct balance cannot always keep us on this fun road, because we are held by emotions. And to influence them, and to overcome them more qualitatively, the balance must be, firstly, shifted in the direction of complication, and secondly, it must be changeable. And just to control the level of difficulty and emotions of the game, the developers need your help and feedback.

Some problems cannot be seen in statistics, but only calculated in real conditions on the battlefield. When such precedents happen and players on media sources talk about it, the developers check the information and look for a solution. So thank you for writing, gentlemen. We appreciate your comments and feedback on the game we all love.

As we discussed above, emotions in a game are as important as mathematical balance. Developers also build them into the expected behavior as well as the players' actions directly in the game. Naturally, they can't rely on a single or few opinions, a large sampling is also applied here. But we should not forget that this sample is made up of many individual opinions, each of which has a place and is important in its own way.

Players often touch upon the topic of the superiority of one gear over another. Well, let's see what it looks like from the developers' side.

Gear

One of the beauties of analytics is that it can be automated. The necessary data is regularly stitched into Dashboards. This is an interactive interface that allows you to track indicators in the form of simple charts and graphs, with specified conditions and parameters, which can be infinite (or almost infinite) in number.

The dashboards are an example of the eternal confrontation between the A-545 and the FN F2000 Tactical.

But the functionality allows you to choose any location, any level of distribution, and sharpening in the killer and the killed. So don't hesitate, the team is vigilant.

It turns out that everyone who thinks that one weapon is better than another is right and wrong at the same time. Each weapon is suitable for its purpose. And you should choose them based on your play style.

In a couple of clicks, balancers get information about how many people use these guns, with what sharpening, in what armor they run, who and with what equipment they kill or do not kill, and a lot of other indicators.

At the time the spring season ended, it seemed like absolutely every opponent was carrying the FN SCAR SSR. In reality, it's nothing short of peak popularity. All those who haven't had a chance to run with it before are testing it, modifying it, and trying it out. Eventually, most of them will change it for something else, and there will be only connoisseurs who have this gun in their hearts, as was already the case with SA-58 CTC.

SA-58 CTC in general is a great master weapon that has carved out a place for itself. The graph shows the number of kills at high distribution levels with SA-58 CTC in the open world. You can see that during the Dead Man's Marathon, the audience was actively recruited with this gun. Everyone wanted to play with the new weapon and one day almost 20% of all kills were with SA-58 CTC. However, over time, most players got used to it, and the gun took its place with 5-7% of all kills in the last days. In general, a similar fate befell the New Year's FAMAS G2 (in the following screenshot).

That's partly the answer to the question about ‘event kids’ as well. It makes sense that at the peak of players having the item, it is very popular and seems to be found in every second player, which is not that far from the truth. But over time and with the arrival of new weapons or meta changes, they become gear of their niche and their purpose. All in all, it's not at odds with our plans.

For example, SBA TANK and FN SCAR SSR peaked in popularity at the end of the spring season, but it should be realized that this equipment is even more niche than SA-58 CTC and FAMAS G2. They are pretty much stand-alone weapons, while the SBA TANK and FN SCAR SSR are situational, after all. So, no matter how much crate weapons hurt, we can get over it, and in the next seasons, we will ‘admire’ already other guns.

Such tools allow you to track the fading and emergence of new meta, the impact of a particular weapon on the player's game experience, and the chances of meeting an opponent with a certain weapon. We've already looked at it from different approaches and with different settings and concluded that May began the era of the greatest variety of weapons in the open world in terms of the contribution of each weapon to the game experience, which can't help but make us happy.

We hope that in the future we will only be able to increase this diversity and support different play styles, where the known meta is not "FN SCAR-SSR + FN-FAL, which makes 50% of kills", but "every weapon has its place, situation, and applicability".

Analysts are even picking up on interface problems.

A lot of new things came with the updated gear lines. And the community was mostly happy about it... But you can't see M16A1 in the lines of assault rifles because of the lack of interface scaling. Even metrics show that a huge number of new players are crafting AKS-74U, not M16A1.

Colleagues said that it was just because of the popularity of AKS-74U. However, it still seemed like the fact that you can't see some gear at the beginning of your gaming journey might influence your choice. They insisted on "lifting" M16A1 into the visible part, and the result is obvious (on the graph you can see the moment of lifting M16A1 into the visible part of the screen):

So we try to follow such user experience imperfections too, although there is still a lot of work to be done in this area. © Highnoon

Economy

In STALCRAFT: X, the economy works almost the same way as in real life. There is demand and there is supply. If demand grows, the price grows, if supply exceeds demand, the price falls. That is, the system balances itself. This is possible thanks to the large number of players who use it.

By the way, this is why EU servers are having a problem with the price of spare parts right now. The system lacks players, and especially fully upgraded players, to balance itself out.

A smaller part of experienced users in EU and NA leads to a smaller number of players interested in the market and hardware modernization. This difference is natural because EU and NA servers are much younger than RU. Therefore, this picture is expected, and the trend will level out over time.

Until the economy on NA, EU, and SEA servers gains enough experience and becomes stable, the developers are keeping it alive artificially by adding more spare parts to the seasonal battle pass and increasing the chance of them dropping from events.

Of course, the developers interfere directly in the economy itself, as in the previous example with spare parts on NA, but not often. But cases, when rebalancing or changes in the properties of some items affect their value for players, are inevitable. The game develops, new gear, new mechanics, and new possibilities for interaction with them appear. Everything new must be harmonized with what is already there. To achieve this, you have to change the characteristics of not only new items but also old ones. The price of these items reacts sensitively to such changes.

For example, before the speed rebalances, most players used speed artifacts. Speed artifacts were more expensive and other stats were cheaper.

After the rebalance, some players became uncomfortable using speed and used alternative builds, for example, vitality. Accordingly, artifacts for vitality became more valuable than before, while artifacts for speed decreased in price.

Yes, the developers' actions affect the economy, but the value of each item is determined by the players themselves. If the price of an item does not correspond to its value in the eyes of the player, he will not buy it. You can put out a regular Veiner for 100 million, some people even do that, but will anyone buy it?

In addition, before any rebalance, the analytics department determines the potential economic fluctuations from the changes, and they have been particularly careful in the last year. On the one hand, they do not want to cause losses to more people, but on the other hand, they want to leave room to make money on the items that have been changed.

The amount of resources received by players is also recorded. So each activity was interesting and profitable for farming but also was not too much. When the ferns from the spots drop tonnes, mutants will very quickly cease to be of value to you, and flowering fern at the auction will be no more expensive than a dog's tail. So when you have too much, it's bad too.

It may seem like it would be great if there was more barter in spots right now. But that's only because you don't have a glut of it now, and it's valuable to you. If you upset the balance here, it will affect both player interest and the economy in the form of prices for transferable barter.

In general, scarcity is always better than glut for any game. If a resource is very rare, you are still motivated to get it, even if getting it is incredibly annoying. In the case of a glut, the motivation to obtain certain items is completely lost, as well as the interest in the game itself.

Now let's see how it looks on the charts. This is the rate at which the Substance 07270 and Red Fern were mined in the middle of May in the Red Forest + Yanov, by players who are at high distribution levels. The statistics are given in the amount of resources per hour.

Qa95 (quantile) shows at what value we cut off 95% of players, or from what value the top 5% of players farming ferns starts. This means that only the top 5% of players are counted, qa75 counts 25%, and so on.

Developers can choose any location, distribution level, and time (farming per hour, day, month, year), select only successful/unsuccessful forays, and a ton of other parameters to get the desired sample and analyze it. That's how we'll know if everything is fine or if something is going wrong.

Again, due to the difference in subjective perception and analytical data, the developers see the location maps not exactly as the players see them. This difference gives us different conclusions. Both may be true, but personal experience includes only the player's impressions, which are influenced by a lot of factors: from gear to game skills. While statistics collect the impressions of hundreds of other players with different gear, skills, and perceptions.

Analytics and leveling design

However, how can numbers and graphics affect the design of locations, the location of anomalies, and the paths that players take? They can, and quite a lot.

The Analytics Department compiles user routes throughout the Zone for the level designers, and the level designers compare these routes with the ones they had in mind. This way, they understand how well their designs compare to the experience of the players. If players often die where the level designers intended them not to, at some point, the developers change the terrain, the location of spots, or infected areas.

For level design, it matters not only where players walk, but also where they kill and die, not only from each other but also from NPS and mutants. To track this metric, death maps are used, which are built from player deaths tied to coordinates. Let's compare the play style of veterans and masters on Yanov using such a map, collected using data as of May 2024.

Does that mean there were more tree-climbing enthusiasts among veterans? Yes, but that's not the only reason. Less experienced players with gear inferior to the master one were much easier to farm the initial areas of the map than the more hardcore, northern areas. In addition, more experienced players were more likely to run near captured bases.

Death maps show us the points of interest of the players in each location. In particular, thanks to them, the developers balance the locations so that each faction can participate in all activities. If the team discovers that on the conditional Dump, it is more difficult for bandits to beat mutant spots than stalkers, they find out what the reason is: either there are not enough spots, or running too far away. They change the location or add a spot so that the location for all was, if not the same, then more fair.

Conclusion

Now you know how analytics works in STALCRAFT: X, why it is important, and where it all started. And also, what balancers and other developers rely on when introducing new mechanics or adjusting existing ones.

You've seen how level designers look at locations, how gear effectiveness is tracked, and why the individual game experience can differ from the real world.

We hope this article has brought us a little closer to each other. If you lack details on any of the blocks, write about it in the comments.

Thank you for reading!