Welcome to our new Age of Sigmar Metawatch series, in which the Warhammer Age of Sigmar design team shares insights and analysis on the current state of competitive play.
Here on the Warhammer Age of Sigmar team, we’re huge fans of Matched Play. Each Monday, various team members will arrive at the studio ready to share their tales of conquest (or defeat!) from the weekend tournaments they attended, before we all examine the updated data from events around the world. It is fascinating watching factions rise and fall with shifts in the meta, as players learn how best to wield their favourite faction, discover combos, and build counters to new threats.
One of our most important tenets is Mastery Through Defeat. Players should always feel like they can learn and improve from each of their games, and should come away with new strategies, adjustments to their army roster or tactical manoeuvres that could have turned the tide of battle in their favour. This is why game balance is so important to us, as in an asymmetric game like this, any perception of inherent advantage for one side can interfere with your ability to learn and grow from the experience.
In this article we’ll touch on the following:
How we gather data to inform our design decisions
The core terminology that we’ll refer to in this series
Our current balance goals for the game
Snapshot the current faction standings to see where we stand relative to those goals.
The Data
Our analytics pipeline currently brings in all data from official Games Workshop events that are run around the world, as well as all the Warhammer Age of Sigmar games recorded in two popular community-run events tools – Best Coast Pairings and Tabletop.to. We have more than 35,000 individual games recorded in our database (captured from more than 1,700 events), and more than 4,000 games and 200 events are added every month.
We capture results, use of factions and subfactions, and we fully parse any available army rosters. This lets us see how often a particular warscroll or enhancement is used at events, and how often players who use that warscroll or enhancement go on to win their game.
It is important to note that this data is imperfect, as we are relying on community-generated data, while certain events track or present data in unusual ways. Fortunately, given the low rate of errors relative to the size of the broader dataset, these generally have a minimal impact on the broader trends.
Key Terminology
When we’re discussing game balance, things tend to boil down into two main categories. External balance measures the relative performance of one faction against others – an army is well balanced externally if it is on a similar power level as the majority of other factions.
Internal balance measures the relative performance of options within a faction. We measure this across several axes – warscrolls, enhancements, core battalions, grand strategies, and battle tactics.
There are a number of metrics we monitor for both internal and external balance, but the most common ones are Win Rate and Inclusion Rate.
Win rate is one of the key metrics for both internal and external balance. Our win rate calculation is simple – the percentage of games that result in a win. So for instance if a particular army played 10 games and won six, it would have a 60% win rate. We don’t include draws in our calculations, as a balanced result (e.g. a mirror match) results in a win rate lower than 50%.*
Inclusion rate is the percentage of army rosters which include a particular warscroll or enhancement. For instance if every Legion of the First Prince army includes Be’lakor, that warscroll has a 100% inclusion rate within that subfaction.
Finally, Margin of error is a statistic expressing the amount of random sampling error in the results of our dataset. In simplest terms, the more we split up a particular data set, the noisier and less reliable the data will get.** We focus on a 60-day rolling average, which provides enough data to reduce our margin of error to <5% across all factions. This is still a significant margin – it is entirely possible that a handful of skilled generals perform particularly well at an event, nudging the average win rate for a faction up a couple of percentage points. This makes it especially important not to panic or overreact, but rather to be patient, monitor the situation and consult players and organisers to validate trends.
Our goals
Now that we’ve introduced the framework, we can discuss our current goals for improving balance on Age of Sigmar.
1. External balance: The average win rate for each faction should be between 45-55% – if every faction has a 50% win rate, a 5% margin of error produces results in this range. We monitor external balance at various levels, by battletome (i.e. Orruk Warclans), army type (i.e. Kruleboyz) or subfaction (i.e. Grinnin’ Blades).
2. Internal balance: We have different internal usage targets set per faction, but in general we want 60% of warscrolls to be used in more than 5% of competitive lists, along with 50% of faction enhancements.
3. Universal options: Universal warscrolls (such as endless spells) and universal enhancements (such as Arcane Tome) should be used in less than 10% of army rosters. These are useful tools, and help level the playing field, but we want a stronger focus on what makes each faction special and unique.
This list comes in priority order. It’s dangerous to try to tune too much at once, which has unintended consequences and can make it difficult to isolate which change contributed most to a shift in balance. Our most recent Battlescroll focused purely on trying to improve external balance. It made a marked improvement, and 83% of our battletomes currently sit within the target 45-55% bracket. As we work on the outliers and external balance improves, we will be able to move onto improving internal balance, and then to universal warscrolls and enhancements.
These goals are not final – once achieved, we set more ambitious targets for ourselves.
Current standings
Here is our current snapshot of external balance, based on 4,552 games played at 446 events held between the 27th of July and the 25th of September 2022.
Beasts of Chaos have continued to climb in the rankings thanks to a major power boost from their White Dwarf Battletome Update. As a faction that had been at the very bottom of the standings for much of this edition, they had a long way to climb, but the strength of the Herdstone combined with a handful of particularly efficient warscrolls have pushed them too far above our target bracket.
Similarly, certain dominant warscrolls (Pusgoyle Blightlords) and subfactions (Drowned Men) have pushed Maggotkin of Nurgle over the top. Sons of Behemat are a different story; while the Krondspine Incarnate has increased in popularity, the biggest factor appears to be a narrower spectrum of results. While they have a lower percentage of podium finishes relative to their win rate, they also have a much lower percentage of events where they win fewer than two of their games played, resulting in a net boost to their overall win rate.
At the bottom end, Kruleboyz and Gloomspite Gitz both continue to struggle, and have suffered at the hands of Bounty Hunters. Both factions should expect significant boosts in the next Battlescroll.
Where next for Metawatch?
We’ll use these articles to zoom in on the competitive meta for Age of Sigmar, touching upon internal balance, external balance between subfactions, and universal enhancements and core battalions.
We’re planning on supplementing this series with a new Metawatch podcast, which will allow us to delve deeper into these topics. We’re already planning an episode to your questions, so please email in any queries you’d like answered to aosfaq@gwplc.com. Topics can include specific data you’re curious about (“which faction is most likely to field a Krondspine Incarnate?”), or questions about our process (“how do we handle team or doubles events?”).
Game balance is an iterative process, and we are constantly working to improve our tools and methodologies. We feel like we’re going in the right direction, but recognise we still have a ways to go. The Warhammer Age of Sigmar team would like to extend a huge thank-you to everyone who has contributed to running or playing in events, or who has submitted feedback. We’re excited to hear what you all think of our next Battlescroll (coming soon!) and even more excited to measure the impact that it has!
* If a faction plays 10 games, wins four, loses four and draws two, it will only have a win rate of 40%.
** Imagine you roll a D6 three times and happen to roll 1,1, and 2. This results in an average value of ~1.3. If you rolled that same dice 1,000 times, you would get much closer to the true average value of 3.5.