March 27th, 2011 by rzr
Flat Track Stats aims to help skaters and fans interpret the most recent WFTDA sanctioned inter-league bout results. In our feature, “Behind the Scenes”, we let you in on what goes on at FTS to bring you these rankings.
Montreal blows-out Tri-City but goes down in rating?
Aaron asks: David --
Seems like a tough break for Montreal to lose 15 rating points. Here's how I interpret what the algorithm did here. Humor me for the sake of my understanding?
Montreal destroyed Tri-City 256 - 52, but given their disparate incoming ratings, the algorithm wanted even more. ~30% more if I'm reading it right, so Montreal should have hit ~350 points to not lose any ground.
That's an interesting scenario because it potentially pushes up against the physical limits of roller derby. I don't remember off hand, but FTS 1.0 had some records for biggest blowouts and I think they were around 300 points. And this match-up isn't even as extreme (in rating differential) as it could be.
As a thought experiment:
In the (unlikely) scenario of the best team playing the worst team, wouldn't the best team have to rake in a physically impossible spread to maintain their position?
Tom adds: I think Texas had the largest score differential, which was 443 points. There have been a few score differentials in the mid to high 300s as well. I think that is what happens when an established team plays a fairly new team. Along these same lines, Montreal, as an established team, played Tri-City who only had 1 game under their belt.
David replies: Ok, I guess I had the same initial reaction (re: Montreal going down 15 pts), but...don't forget, 15 points isn't really very much.... What makes it look larger is that there are so many teams clustered within something like 10 points of each other, so a little shift ends up in losing 5 places [on the current overall rankings].
Now … about how big a win they have to have....don't forget, Difference-Over-Sum [The scoring metric used in FTS’ Algorithm] is all about controlling the game, not just scoring points. So, really, Montreal needed to score a little more than 17 points for every point that Tri-City scored. So yes, if we let Tri-City have 50-something points, then Montreal needs a bunch more....but, if they kept Tri-City to 15 points, it would have all been right on target.
Aaron counters: I think the results are reasonable -- I'm not reacting to the 15 point loss or the number of teams Montreal drops below.
My point is more abstract, that is if you are a highly ranked team you're really going to have to bust your ass playing a significantly lower ranked team (and not lose rating points as a result). More so (proportionally) than against a mid rank team, because you need to cram in so many more points to break even. Yes it's supposed to be easier... But I assume there's diminishing returns on that when you're being asked for a 300+ blowout to break even [for your rating]. At some point, it would be hard to bring in these high scores skating against no opponent -- of course the algorithm is not sympathizing with the physical limits of roller derby.
… And Tri's 50 points is probably more impressive than I originally considered.
Donna quips: I think this also prevents "Rating Inflation" by beating up on a bunch of much lower ranked teams.
David adds: Yes to all the observations [above]...As Donna says, it's exactly this tension that keeps the ratings from inflating. Another interesting thing to note... the wider the differential [in power rating], the more burden it puts on the defense.
Tom comments: A similar thing happened with the Derby City vs. Southern Indiana (ROSI) bout. Derby City beat ROSI by 100 points, but Derby City was ranked low enough previously that the 67 points gained put them just below ROSI. Southern Indiana and Derby City are both fairly new teams so I'm seeing this as new team thing that self-corrects as the teams play a few more bouts.
Last week, ROSI had a Rating in the mid-700s after their first bout and Derby City in the high 500s after five bouts. With Derby City’s win over ROSI, both teams are now in the mid-600s.
David replies: I think [it] points to a different effect. Clearly Derby City has turned their program around [for the 2011 season]. That was a really big difference from expectation and so they are going to be re-calibrating for a few bouts. I get that seeing them ranked "below" ROSI makes a "logical error" to use the FTS 1.0 terminology. But, within the context of assuming that teams can have a distribution of performances, it all kind of works out...
That's an interesting scenario because it potentially pushes up against the physical limits of roller derby. I don't remember off hand, but FTS 1.0 had some records for biggest blowouts and I think they were around 300 points. And this match-up isn't even as extreme (in rating differential) as it could be.
As a thought experiment:
In the (unlikely) scenario of the best team playing the worst team, wouldn't the best team have to rake in a physically impossible spread to maintain their position?
Tom adds: I think Texas had the largest score differential, which was 443 points. There have been a few score differentials in the mid to high 300s as well. I think that is what happens when an established team plays a fairly new team. Along these same lines, Montreal, as an established team, played Tri-City who only had 1 game under their belt.
David replies: Ok, I guess I had the same initial reaction (re: Montreal going down 15 pts), but...don't forget, 15 points isn't really very much.... What makes it look larger is that there are so many teams clustered within something like 10 points of each other, so a little shift ends up in losing 5 places [on the current overall rankings].
Now … about how big a win they have to have....don't forget, Difference-Over-Sum [The scoring metric used in FTS’ Algorithm] is all about controlling the game, not just scoring points. So, really, Montreal needed to score a little more than 17 points for every point that Tri-City scored. So yes, if we let Tri-City have 50-something points, then Montreal needs a bunch more....but, if they kept Tri-City to 15 points, it would have all been right on target.
Aaron counters: I think the results are reasonable -- I'm not reacting to the 15 point loss or the number of teams Montreal drops below.
My point is more abstract, that is if you are a highly ranked team you're really going to have to bust your ass playing a significantly lower ranked team (and not lose rating points as a result). More so (proportionally) than against a mid rank team, because you need to cram in so many more points to break even. Yes it's supposed to be easier... But I assume there's diminishing returns on that when you're being asked for a 300+ blowout to break even [for your rating]. At some point, it would be hard to bring in these high scores skating against no opponent -- of course the algorithm is not sympathizing with the physical limits of roller derby.
… And Tri's 50 points is probably more impressive than I originally considered.
Donna quips: I think this also prevents "Rating Inflation" by beating up on a bunch of much lower ranked teams.
David adds: Yes to all the observations [above]...As Donna says, it's exactly this tension that keeps the ratings from inflating. Another interesting thing to note... the wider the differential [in power rating], the more burden it puts on the defense.
Tom comments: A similar thing happened with the Derby City vs. Southern Indiana (ROSI) bout. Derby City beat ROSI by 100 points, but Derby City was ranked low enough previously that the 67 points gained put them just below ROSI. Southern Indiana and Derby City are both fairly new teams so I'm seeing this as new team thing that self-corrects as the teams play a few more bouts.
Last week, ROSI had a Rating in the mid-700s after their first bout and Derby City in the high 500s after five bouts. With Derby City’s win over ROSI, both teams are now in the mid-600s.
David replies: I think [it] points to a different effect. Clearly Derby City has turned their program around [for the 2011 season]. That was a really big difference from expectation and so they are going to be re-calibrating for a few bouts. I get that seeing them ranked "below" ROSI makes a "logical error" to use the FTS 1.0 terminology. But, within the context of assuming that teams can have a distribution of performances, it all kind of works out...
Comments
Thank you for the clarification. It makes sense, within the statistical model, that Montreal should have kept Tri-City to a lower score. It would be interesting to see when those points were scored during the game to see if Montreal dialed back the defense for the benefit of the spectators. If you look at their last home game against Queen City, they beat your expectations by .08 with a ~12:1 margin in score. That had to be about as interesting to watch as the invasion of Grenada. Past experience has shown that a home team who dismantles a visitor runs the risk of alienating fans.
It's also a reflection on team management. You don't see a new boxer jump in the rink against Mike Tyson in his first fight. They have to work their way up the rankings to face a top tier opponent. Montreal did Tri-City a great service by playing them so early. It should be interesting to see how this match up plays out over the year as Tri-City gets more bouts in. You have to figure Tri-City must have learned a lot of new tricks/strategy while playing Montreal. They have 2 months before their next bout. Assuming they dissect the Montreal bout and work on their weaknesses, the teams on their schedule have something big to worry about now.