Hello everyone, and welcome back to the Warhammer Age of Sigmar Metawatch from the Warhammer Studio!
This week, as we eagerly await enough data to properly analyse the impact of the latest Battlescroll (early results look promising), we’ve sent the grots to grab a random selection of questions from the mail bag.
Thanks again to those of you who emailed us – it was really great hearing such a positive response from the community. Please continue to email us any questions about our data, process, or analysis to aosfaq@gwplc.com, and we’ll do our best to address your questions in a future installment.
Q: Do you use any qualitative methods to complement the quantitative data that you gather?
A: Yes! Quantitative data refers to numbers and statistics that we can measure, while qualitative data tends to deal with the more human element of words and meaning.
To give a basic example, we might measure that a particular warscroll is overperforming using quantitative data – we can see how many rosters that warscroll is included in, and how many games were won and lost when it was included. We then build a hypothesis for why this might be happening. Is the unit simply too efficient for its points cost, benefitting from a set of synergies, or does it simply serve a particularly useful role?
This is where qualitative data comes in. We have two different external feedback groups – the external playtesting team is mostly made up of talented tournament players who focus their efforts on new rules, while the metawatch advisors group consists of tournament organisers who have a strong grasp on the overall health of the game. These teams help provide valuable insight to ensure that the changes we make are as effective as possible while minimising knock-on impacts.
Q: What are the unit-type goals for army lists? Are there goals to have a 20% split between Hero, Battleline, Artillery, Behemoth, and others?
A: One of our favourite things about Warhammer Age of Sigmar is the incredible variety of factions, and we really enjoy seeing the creativity and skill involved in building compelling army rosters. We see rosters which feature enormously powerful god-like characters and their retinues, subfactions like Gristlegore or Lofnir that focus on monsters, and hordes of zombies or ratkin overwhelming the enemy.
Our goals are focused on making as many warscrolls as possible viable and interesting to use, so we’re not currently looking to push rosters towards a more prescribed structure.
Q: How do you use competitive data when writing a new battletome?
A: Since we’ve built our data gathering and analysis pipeline, we’ve been making heavy use of it when composing new battletomes.
We use the internal balance data to identify units that are competing with each other in an efficiency arms race. This happens when two different units have very similar roles – for instance, long-range damage output. We often see one unit being slightly more efficient, and becoming the dominant choice. If we try to fix this by simply increasing the points cost of the dominant unit, it can tip the efficiency scales, and everyone will switch to the newly-more-efficient choice.
In recent battletomes we’ve tried to avoid these arms races by having clearer roles for different units. One recent example are the reworked infantry heroes in the new Battletome: Lumineth Realm-lords, where we’ve seen a significant uptick in the number of different warscrolls that are seeing competitive play.
Q: Since your rolling window is 60 days, any given window is very likely to feature one new book, meaning external balance has a major kink thrown into every rolling window. Since external balance is a perpetually moving target, how do you adjust internal balance and universal options?
A: New battletomes do indeed introduce a significant challenge to balance. As mentioned above, we reference balance data when authoring a new battletome, and we try to adjust a faction’s power level to hit that 45-55% win rate from the off. I’m happy to report that this has been the case with most recent battletomes. Combined with external playtesting, this generally minimises the impact of a new battletome, and doesn’t disrupt our ability to move on to internal balance changes for factions that have a stable win rate.
Q: Regarding internal balance – where you want 60% of warscrolls to be used in more than 5% of competitive lists – which factions currently do the best and worst by this metric?
A: Here’s a recent snapshot of internal balance data from 459 events held between the 6th of September and the 6th of November, showing the percentage of warscrolls that are used in at least 5% of competitive lists.
There’s a significant correlation between the number of warscrolls in a battletome, and how many warscrolls are used competitively. Some units are more popular than others, so in a faction with a lot of choices, some will struggle to find a place in competitive rosters.
Around a third of our factions aren’t yet hitting our internal balance targets. Some of these factions saw internal balance adjustments in Battlescroll: Galletian Reinforcements, and we’re keen to see the impact of those changes. We will continue to roll out internal balance changes in future Battlescrolls and publications to help address the outliers.
Thanks again to everyone who sent questions in! We still have plenty more in the mailbag, so stay tuned for more answers next time, alongside updated data and analysis of the impact of the Battlescroll.