League of Legends Tribunal

8 months ago League of Legends brought in their tribunal system as a method to try to foster good behaviour amongst the player base. The basic idea is players can report people in their games and then a jury of their peers would vote on in the player behaved in a way that broke the rules and should be punished or not. I'd heard about it when it first came out but didn't pay much attention to it. I report people all the time but I never really cared about what went on after submitting a report.

A couple weeks ago I finally got around to signing in as a jurist to punish some bad players. It was interesting to see what actually gets submitted when a player gets reported. The jury gets to see the entire chat log from the game (but not the champion selection or game recap screens) with the offender's chat in a different colour to make it easier to find what they said. It also lists their final items, their kills/deaths/assists in the game, their incoming and outgoing DPS, and what level they finished the game at. It also lists every reason they were reported after the game and differentiates if it was a teammate or an opponent who reported them. It doesn't say which player specifically made which complaint.

The great part is it lists all the games they played where they got a complaint. You can browse all complaints to see if there's a pattern of bad behaviour or if people are just being really oversensitive or bitter when reporting.

I did a few and got an email on Friday saying that I got 8 cases right and they were giving me 40 IP for my help. I don't remember how many I did. It doesn't seem there's any way to see how many I did, so I don't know what my winning percentage is. I do know I pardoned more people than I was expecting...


I did some poking around today to see if I can work out what's exactly going on behind the scenes here. They posted some interesting stats last month... Apparently only 1.4% of players have been punished by the tribunal. But of people who get presented at the tribunal 94% get a punish vote. Half of those people get sent to the tribunal a second time and get another punish vote. Apparently most people who get reported are on the losing team and get reported by their teammates. (That makes sense since you're more likely to get angry and start raging at someone who you think is making you lose a game.)

I also dug into the forums and found some facts about how the system works. They refuse to give out specifics since they think offenders will be able to abuse the system if they know details. I guess that could be true but as the saying goes... Security through obfuscation isn't security at all. I'd rather they have a system that couldn't be abused but I guess they don't feel like they got that far. What they have said, however...

• One report won't get your case presented at the tribunal. It takes several games with reports to get presented for voting. The percentage of games played compared to games with reports matters.
• Long term or permanent bans have to get approved by a Riot employee but everything else is automated.
• Your first time found guilty you get a warning email that doesn't tell you what you did wrong, just links to the code of conduct.
• The second time you get a short term ban (1-3 days or so) but still don't get told specifically what you did wrong.
• Even if you get permanently banned you don't get told what you did wrong.
• Riot employees who spot check random cases think the voters are too lenient. With a 94% conviction rate. Sounds to me like they need to start sending more borderline cases to the tribunal since if the conviction rate should be more than 94% I can't imagine it makes sense to actually review the cases. Punish everyone and move on!
• You can get reported for unskilled play but those reports don't get added to tribunal cases. The idea is to give people a way to vent without actually hurting anyone since people being bad isn't an offense. (With the rating system they should eventually fall out of your play range if they're actually worse than you!)
• The system had a lot more oversight built in at launch but they're pretty happy with how things are working so it's almost all automated now. The computer picks who gets put on trial. The computer tabulates the votes and decides when someone should be punished. The computer punishes them.

So it is entirely possible that, through no fault of my own, I could get banned. If enough of the people I play against randomly report me and enough of the jurists blindly vote to punish I'll get hit without any oversight. On the other hand it would have to happen in a lot of games. And since I really don't do anything bad and the jury does pardon 6% of people... I really have nothing to worry about. It's theoretically possible for the universe to align and screw me but it's really not plausible. 

It seems sketchy that they don't tell people what they did wrong, but I've been a moderator for an online game. People who are apt to misbehave are also apt to whine and argue and try to rules-lawyer their way out of their punishment. I didn't really have all that much going on so I'd typically indulge them but it almost never went anywhere productive. They'd want to be able to swear. Or trade in the play areas. I'd always tell them specifically what they did wrong but the vast majority of the time they already knew what they did. They knew we didn't allow it. But they thought we should allow it so they were going to do it anyway. 

The real key is they would gladly debate the facts for hours on end. They'd try to convince me I was wrong. And from their point of view it was just them wasting a couple hours. But from my point of view, when you get several of these people every day... Suddenly my whole day is wasted. (In reality I'm very good at multi-tasking chat and my side of the 'conversation' was pretty limited so I'd mostly just let them chat themselves out.) Even then, we had more chat moderators than Riot does. And fewer players. We provided better customer support for sure, but I don't know that Riot is doing things wrong. It doesn't make financial sense for them to hire a bunch of people to debate with people who are almost certainly going to get banned soon.

I do think it would make sense for them to provide a list of the reasons on their tribunal case. Add in a disclaimer about how they were reported for some of these reasons and be done with it. At least then people can rest assured that they weren't banned for unskilled play, for example. Getting an email warning saying you've been bad and to read the code of conduct is great and all but getting an email warning saying you were reported for 'AFK, AFK, AFK, AFK, AFK, Negative Attitude, AFK' gives you a pretty good idea what your problem is!