The Persistent Problem of PvP Rating Exploits

Velidra sent me the link to the above video of a Destruction Warlock tearing apart battlegrounds with his bare hands. The guy takes on 3 Rogues at once and walks away the sole survivor. If I walk away from an encounter with a single Rogue, I usually count myself lucky. Three of them at once? Game over, man. Game over.

But not for Adouken.

I enjoy PvP videos. They usually make me feel bad about my own UI (how do they do that with so few addons?) but they make me feel great about the potential of my class, and I try to learn from them as best I can.

Videos naturally show a skewed version of a player’s skill, but that’s doesn’t mean that players who show off their skills in them are somehow faking it. They might not be that good all the time, but damn if they weren’t that good at some point, when the cameras were rolling. Odds are pretty high that they are that good, and that they operate at a high level of play all the time.

I don’t play anywhere near as well as you see Adouken play in that video – far from it. Watch that first segment and realize that he’s casting Nether Ward inbetween the time a spell is cast at him and the time it reaches him. You notice how it looks like he reflects Death Coils back at their caster? He’s casting his own Death Coil while his opponent’s spell is in the air. That’s awesome.

There is an objective difference in skill between Adouken’s player and me. While it may not be easy, surely, we can measure it somehow, right?

That’s where PvP ratings are supposed to come in and help us know the great from the good, the poor from the mediocre. They way they work is simple, at least in concept.

• There are two numbers used in the rating system: Matchmaking Rating (also called MMR) and your PvP Rating. You have different values for each bracket.
• Your Matchmaking Rating changes with every win and loss, and is used by the system to try to find a level of skill where you’ll win about 50% of the time. You can think of the MMR as measuring your aptitude, your potential rating.
• Your PvP Rating is based upon your performance over time, changes slowly, and is what PvP achievements, gear rewards, and titles are based upon. PvP Rating, in theory, measures your performance over time.

The goal of the PvP rating system is to match you up with people of equal ability, not to allow you to win all the time.

That’s kinda weird, isn’t it? From a sport perspective, it would be really strange to have a system that wasn’t based on win/loss records (performance). But you also have different leagues and ways of stratifying talent that don’t exist in computer games – local, regional, and national competitions, playoffs, major, minor, and little leagues. So instead, the goal is to put a number on you and say, this is an arbitrary level you’re performing at.

All other things being equal, you should win about 50% of your matches against teams and people of similar PvP rank.

But that’s not how it works.

THE CURIOUS CASE OF THE MISSING MMRS

So a funny thing happened in the 4.2 Patch notes.

• The individual Matchmaking Rating column has been removed from the Arena scoreboard.
• The individual Matchmaking Rating column has been removed from the Rated Battleground scoreboard and replaced with a team Matchmaking Rating.

This is kind of curious, isn’t it? What’s going on here?

I’ve said several times that Blizzard is trying to encourage people to get into Rated Battlegrounds in patch 4.2, and that many of the changes are with this in mind. You might think that a change like this is just to make it so that people who join rBGs don’t see how outmatched they are and throwing the match immediately.

While this makes a limited amount of sense, it’s not what’s going on. Yes, this is to try to make Rated Battlegrounds more fair, and therefore more attractive. But hiding the personal MMR is aimed at stopping a series of exploits people are using to get titles in both Arena and Rated Battlegrounds, exploits which are running rampant right now. The most common exploit involves using alts to boost the main characters’s MMR, then winning enough games at the various high levels to get the desired titles.

If you’ve been comfortably playing in a lower Arena bracket, you may have noticed that the last 2 weeks have been rather… more painful than before.

You’re not imagining things.

THE PROBLEM WITH MMR, OR WHY THAT TEAM JUST STOMPED US

[Player]: How about a joke before you go?
[GM]: Your Arena rating.
[Player]: /facepalm

Consider the following facts about how MMR works.

• Your team MMR (different from your rating, mind you) is equal to the average individual MMR of all the players on the team.
• In the event of a win, individual MMR should go up, thereby raising the team MMR. Losses reduce MMR, but not as much as wins do.
• Players on new teams start out with 1500 MMR.

Let’s look at how this works out.

You start the season out with your two friends and start playing 3v3 on your mains. You win some, you lose some, but your individual MMR rises and falls together. If your MMR hits 1800, your teammates are also at 1800. Your MMR, and eventually your PvP rating, accurately reflect your team’s performance to date. Everything is rosy.

Now let’s say one of you has an alt you want to bring in. Maybe it’s because it’s a better comp, maybe it’s just time for a change. Now you’ve got two people at 1800 MMR and one person at 1500 MMR, so your team has a MMR of 1700. You’re facing teams which are a little worse than you were doing before, but maybe the alt is undergeared, so it balances out.

What’s interesting is that it might balance out to fair matches in the 1700-1800 bracket, your individual MMRs are now going to be out of sync. The alt will always have a lower MMR than the other two main characters, and can never catch up.

Now let’s take a step back and change the conditions a little bit. You have a 3v3 team, starts fresh at 1500 and goes to 1800. Two of you drop your mains and swap to alts. Your team’s MMR is now 1600. You’re facing easier teams than you did at the 1800 bracket, so you win, even though the alts might be a little undergeared. They gain +200 MMR, you gain +200 MMR, you’re now at 2000 MMR, they’re at 1700 MMR – and your team is back at 1800 MMR.

With me so far? You’re still playing at 1800 MMR teams, but your personal MMR is 2000, your team’s alts are at 1700.

Now, you swap to to one of your alts, and one of your teammates swaps to their mains. 1500 alt, 1700 alt, and 1800 main are now on a 1666 team. You play until your teammate’s main is at 2000. (You’d be at 1700, the second alt would be at 1900.)

You see where this is going, right?

By cycling through alts, teams are able to artificially boost the individual MMR of their main characters.

Now let’s take this a step further. The team cycles through once or twice, everyone’s mains are sitting around 2000 MMR. The alts are all around 1800, which is really where people’s skills are at.

So the team hops on their alts and loses every single match. Their MMR tanks. They go from an 1800 MMR team to a 500 MMR team in a night. Those characters have terrible MMR now, which is exactly what they want.

Because now, you have a crop of alts at 500 MMR to swap into a team with a 2000 MMR main. The team’s matchmaking rating is 1000, so they’re going to be facing significantly easier opponents. But they’re capable of playing at 1800 MMR, so they dominate. The main’s MMR shoots up to 3000+ while the alts are climbing back up to 1800.

And then once everyone’s mains have an MMR of 4000+, they all rejoin the team and play enough matches to bring their team rating – and therefore their PvP rating, which gives the Gladiator titles – up to the desired level. Yes, their MMR will fall from the heights it reached, but the PvP Rating will rise to meet it somewhere in the middle.

When that team comes and stomps your 1000-rated group you and your friends put together to screw around on with perfect CC chains, huge burst damage and flawless target switching… they should never have been playing you in the first place.

BUT WAIT, RATED BATTLEGROUNDS ARE EVEN WORSE

You know why MMR boosting is an even bigger problem in Rated Battlegrounds? It’s not because they’re BGs, and it’s not because I am trying to pick a fight with rBGs this week.

No, it’s because:

1. There are 10 people on your team, and
2. Rewards are based on your individual MMR, not your team MMR.

Nice, huh?

Swapping alts (or even players who don’t care) in and out of BGs can be done like in Arenas, but it’s a little easier to boost MMR due to the number of low rated alts you can bring to the team. If you have 2 players at 1800 and 8 players at 1000, your team will be at 1160 MMR and (hopefully) get matched accordingly.

The coordination required to alt swap and lose MMRs is harder to do with 10 people than with 3. There’s a lot more time involved with Rated Battlegrounds, and the effort put forth by a low-rated character is often the same (or more) than a high-rated one, but the high rated one will get rewarded disproportionately to their efforts. While there is some alt-swapping going on, it’s not as easy as some other methods of boosting your MMR.

No, the best thing to do is to work with a strong group until you’re all up to a decent level – say 1800-2000 – and then PuG like crazy. Get into the worst groups you can find who still have a chance of winning, and play with them. This has the same effect as the alt-swapping MMR boost – when you win, you win big, when you lose, you don’t lose that much – with none of the headaches of having to swap alts yourself. You can go from PuG to PuG, increasing your MMR with each win. You may not win as consistently as you do with your set group, but you will get a great rating, which in turn gives you access to the PvP titles. You don’t even have to win any matches at your new MMR to get the titles, because nothing is based on your team’s MMR or rating – just your individual rating.

Remember back when you thought people’s rating really measured their skill?

/AFK FTW

At some point above, you probably wondered how people can preserve their ratings while losing.

Well, if you leave a match before it finishes, it doesn’t count. This is how win-trading works – people queue in off-hours, trying to get specific teams to match up against, and leave the match if it’s not them. When people leave the match as soon as things start going a little wrong? They’re leaving to preserve their MMR, which gets modified at the end of the match.

You didn’t think people were /afking because they were scared of you, right? 🙂

WIN TRADING

Another reason why people /afk out of an Arena (or Rated Battlegrounds, though I think this is less common) match is because they’re trying to trade wins with another team.

This often happens late at night, when there aren’t a lot of teams playing in the different brackets, and it’s been a problem since Arenas started, but obviously if you can find a team who will throw the match for you, it’s a great way to get your PvP Rating to match your possibly inflated MMR.

I don’t have a lot to say about win trading. Don’t think it doesn’t happen, because it does.

WHY BLIZZARD IS HIDING INDIVIDUAL MMRS

Given that there are two different types of MMR inflation going on in both types of Rated PvP, you can start to see why Blizzard is trying to hide that value. It’s not going to prevent the problem from happening, especially not in Rated Battlegrounds, but it can reduce the precision with which people are doing it now. There will be more guesswork when exploiting, both in boosting and tanking individual MMRs.

There’s a concept in security circles called “Security through Obscurity,” which is a way of describing any security system that relies upon something being hidden for it to be secure. It’s usually treated as a bad thing, because once something is found that relies on it, it’s completely insecure. In cryptography, if your sophisticated code algorithm uses a single seed to generate codes, once the seed is known your code is useless. In piracy, if you bury your gold but don’t put a lock on it, anyone who finds the gold can take it.

In other words, security through obscurity is generally not very secure.

There’s a temptation to say that hiding the MMRs is just that – not making the system any less susceptible to exploitation, just hiding the problem. People can still do the things they’re doing now. You are going to face teams who are boosting themselves, who have great gear and skilled players but are playing with an MMR well below their real skill, and you won’t be able to tell anymore.

But, removing the data points does make it more difficult on the exploiters. Not a lot – not like a complete revamp of the MMR system would – but a bit. It’s a relatively simple change in terms of development time which will have some impact. That’s why it’s happening now.

I don’t really like this change, but I see that Blizzard has to do something.

Will teams still be able to boost their MMR into the stratosphere? You bet. As far as I can see, as long as the three conditions I laid out about the MMR system hold true, boosting is possible. You can’t have flexible teams and not have this kind of potential abuse. Will it be harder for other players to find out who is boosting? Yes, it will.

It’s not great. But it’s a start.

IT’S ALL RELATIVE

Man is the measure of all things.

-Protagoras

The interesting thing about the PvP Rating system, at least the Platonic ideal of the PvP rating system, is that it provides a way to compare people with very different character types. No matter what you play, or what your team is like, it should provide a relative measure against other players. The values are arbitrary and entirely dependent upon the actions of other players, as well as your own.

I think about other rating systems that assign a numeric value to your ability – college aptitude tests like the SAT/ACT, IQ tests, even professional placement exams – and they all measure ability based upon fixed criteria. Here is a test, there are right and wrong answers, how did you do? (Please note, I am an old fart, and I still think of the SAT as having all multiple-choice questions, none of this fancy writing stuff.)

Both types of test assign numeric values, which of course makes them more scientific.

But more than that, both purport to measure aptitude, but one is easy to game for your advantage, while the other is not. Why is that?

Take a look at the exploits again. Each one of them involves using other people. The system isn’t the problem, the people are. The system relies upon measuring you and your teammates, and your performance against other teams, which provides two places where it can be exploited.

Your opponents can really only modify your rating through throwing a match and win-trading, which is one kind of problem. You and your teammates can modify it through careful manipulation, boosting some characters, tanking the ratings of others, and preserving gains through /afking.

If these ratings were static and based upon some kind of objective performance, this kind of exploitation would not be possible. You can’t cheat an aptitude test by trying to throw off the bell curve and flooding the test pool with people who are going to score 0. You can’t get a 1600 on the SATs by being better than everyone else in your testing pool – you have to get every question right.

There are objective measurements of player skill, even in an environment soaked in relativity like PvP. Go back to the video at the top of the page. The player’s reaction time is faster than many others. They choose the right spells and abilities to succeed. They position themselves well, they use their abilities in the correct order. There is a measurable difference between that kind of play and my own, and that means we could construct a static test to measure it.

But static tests are hard. They have to be randomized, administered sparingly, maintained and updated. I don’t know how it would capture performance in the field fairly. I have only the vaguest ideas how a static PvP test would work. Perhaps like kata in martial arts, where mastery of a ritualized set of moves – perhaps a scripted PvP encounter for each class – is required to move to the next level?

That doesn’t feel much like PvP to me. PvP requires other players, living, breathing, thinking teammates and opponents.

And yet, as soon as we bring other people into our measure, we open the door for manipulating that rating.

SKILL > RATING

PvP Rating is not equal to skill. As much as we would like to have a system that really represents skill, the PvP Rating system is not it.

The more I look at how the PvP Rating system is being manipulated, the less I respect it. There are a lot of highly skilled players with high ratings, where ability and performance are in sync. But there are plenty of other teams that are taking shortcuts, who are going for the quickest way to their desired goal. They’ll stomp through the lower brackets while boosting a friend’s toon. The only incentives that aren’t about gaining the coveted rating are designed to get people into Rated Battlegrounds – everything else is about getting your numbers up.

Players who deliberately game the rating system sadly affect other players. A 2500 player playing in the 1250 range artificially depresses the ratings of people who would naturally be in the lower brackets. The upper brackets, in turn, get filled with people who have artificially inflated their ratings, giving the people who actually perform at that level easy opponents, inflating their ratings in turn.

The more players who game the system, the more imbalanced the brackets get.

And none of this is a reason to not play Arenas or Rated Battlegrounds.

• Arenas remains the best place to learn how to win fights in PvP, period. (The only other activity that even comes close is dueling, which is really 1:1 Arena.) Yes, it’s a death match. Yes, there are strict limits about what you can and can’t use. Yes, you’re going to have unbalanced matches. Try to win them anyway. Learn from your losses.
• Rated Battlegrounds delivered on their promise – they let you play BGs with the team composition you want against really good opponents. You have to win the individual fights, you have to execute a strategy, you have to do it against an organized opponent. Yes, you’re going to have unbalanced matches. So what? Get stronger.

As long as PvP Ratings are a relative measure, players will work together to game the system and artificially inflate their ratings. The exploits I’ve discussed are just some of the ways that players are trying to get around the system.

Is this cheating? Yep, you better believe it. Creative use of game mechanics, my foot.

But while it unbalances PvP, it’s not a reason to abandon Arenas and Rated Battlegrounds.

Skill is not equal to rating. Skill can’t be gamed, it can only be acquired through work and talent.

Screw your PvP Rating. Focus on improving your skill instead.

If you do that, all the exploits in the world won’t matter one bit.