Pain Newton Comments:
I think I understand the algorithm, but I wish it took into account recent head to head wins. The current algorithm undermines itself by having a lot of top 10 teams that have beat teams recently ranked above them.  In the top 10 alone in the past 6 months, in head to head games Rocky beat Oly twice, Rocky beat Gotham, Denver beat Rose most recently, BAD beat Rose & Denver...; yet all of them are ranked below the teams they beat. I understand that no matter what you do you will have some inconsistencies but the top 10 seem to have a whole bunch so I can only imagine if you go through the rest of the teams how many you would have

David Pontificates:
I’ve received many comments that are, at their core, similar to this one. The central assumption is that any team that wins is, by definition, better. However, is it not the very nature of sports that on any given day, “anything can happen”? Any single result is always the combination of many factors: skill, tactics, officiating, local conditions, health, and certainly luck. We know, for example, that in sports like baseball and basketball, where two teams play a series of games, the same team does not always win. Just a quick look at last year’s baseball results supports this idea. The team that ended 2010 with the best record, and almost went to the world series, was Philadelphia with 97 wins and 65 losses. The team with the worst record in MLB was Pittsburgh with 57 wins and 105 losses. I don’t think anyone would argue that Philly was a better team than Pittsburgh last year. However, in the 6 times those two teams met during the regular season, Pittsburgh won 4 games and Philly won only 2.  There are many similar examples and the general sporting world shows us every day that the better team does not always win, and the winning team is not necessarily the better team.  Better teams just win more often and more consistently. It’s that consistency that we try to capture with our algorithm.

Now the interesting thing to me, is how often does the “better” team actually win in roller derby. If we look at all WFTDA bouts since 2005 in which the two teams are pretty comparable (defined as being less than 20 power rating points apart), the higher rated team actually won only 135 out of 227 bouts or just under 60% of the time. Unfortunately, lacking the decades of development, derby has not yet achieved the parity of professional sports like football, where all but 4 teams have won the Superbowl at least once. As a result, the difference in skills can actually be quite large. If we go to slightly more lopsided contests in which the teams’ ratings are separated by somewhere between 50 and 100 points (for example: Philly vs. Jet City or Detroit vs. Nashville) we still find that the higher rated team is not always victorious, with 305 victories out of 359 bouts or about 85% of the time. It’s this idea of relative strengths that is the focus of our algorithm. The important thing to look at is not the specific ranking (ie. who’s number 22 and who’s number 23), but rather how different are the ratings. Anything less than 10 or 15 points means that the teams are very similar and picking the winner is a virtual toss-up. We’ve tried to capture this idea in the predictions that we now display for upcoming bouts. By showing the actual probabilities for each team to win, we hope to reinforce the idea that regardless of the rating differences, we recognize that both teams have a very real possibility of winning.


I agree with what you've said here but I have a follow up question and this may be more of what Pain Newton was getting at.

In the championship game last year, you have Rocky Mountain entering rated at 973.7 and Oly entering rated 953.9. So, FTS correctly predicts that Rocky Mountain wins as it is played on a neutral floor. But, Rocky Mountain actually moves below Oly after this game. Rocky was rated 960.8 and Oly 966.7. I can see how if Rocky entered the game ranked below Oly, a 1 point win wouldn't be enough to rank them higher. I could also see if the game was a home game for Rocky, how a 1 point game might move them below Oly. But, I don't get how a team that is ranked higher entering a game on a neutral floor could possibly move below the team they just beat based on your system. Isn't that a logical error?

The key component here is that, as far as the ranking system goes, Rocky was the effective home team and was expected to win by much more. I realize that assigning a home-team bias when both teams are on the road may seem a little counter-intuitive, but in tournament play, the "home" assignment is given to the higher-seeded team. In looking at all past data from regional and championship tournaments, we found that there was a definite bias in favor of the higher-seeded team. This bias is over and above the natural expectations of the relative strengths of the two teams. The correction for this tournament bias is a little larger than the standard home/away bias. As a result, in the specific case you mentioned, Oly effectively beat the bias. So, from the algorithm's perspective, Oly won that bout.

As a side note, the whole topic of biases in sports is really very interesting. There's a fascinating new book out called Scorecasting that does an impressive analysis of some of the underlying biases found in professional sports.  Some of the findings are very applicable to derby.  I highly recommend it to anyone who's interested in understanding more.