We’ve gotten a few emails from readers who have asked us, “Why doesn’t USCHO endorse KRACH?”
USCHO doesn’t endorse KRACH or, for that matter, any ranking algorithm. (We don’t even endorse the PairWise Rankings, even though we were the first to publish them 15 seasons ago as a model that mimics the process the NCAA uses to compare teams for playoff at-large bids and seeding. The PairWise is not the process, but it replicates the process.)
Readers also ask, “But isn’t KRACH completely objective?”
Yes, the calculation of KRACH is objective. But for that matter, the calculation of the PairWise Rankings and the Ratings Percentage Index are also objective, as are the data from which each is calculated. Each, including KRACH, is also arbitrary to some extent because the algorithm chooses which criteria it is going to measure. It has been recognized, for example, that KRACH does an inadequate job of measuring the effect of home ice advantage, so there is another algorithm that introduces that as a factor. There are also other objective computer ranking systems, such as CHODR or HEAL, which have their own advocates and which take other factors into consideration in their algorithms.
Granted, the KRACH algorithm would be an improvement over the current RPI. The NCAA over the past few years has pulled and twisted and tugged on the RPI formula to provide better results. It has altered the ratio of a team’s winning percentage, its opponents’ winning percentage, and the winning percentage of its opponents’ opponents a few times. (Currently, that ratio is .25, .21, and .54 for D-I men, meaning that 54% of the RPI is the result of the schedule that your opponents have against other teams — something completely out of your hands.)
In addition, the NCAA has tinkered with the RPI by adding — and then in later seasons removing — bonuses for wins on the road or neutral ice. The RPI also was altered in recent years to remove the effects of beating weaker teams: first, during playoffs, and now, for the entire season.
So the RPI is flawed. If a computer ranking is to continue to be part of the NCAA selection process and part of the seeding of teams, then the KRACH algorithm would be an improvement. Most certainly.
However, what KRACH absolutely cannot be is a replacement for the PairWise method of selecting and seeding teams, and we would never endorse that.
Why?
Because KRACH treats all games as equal. And as that storied hockey writer George Orwell once wrote, “All games are equal, but some games are more equal than others.”
Any system that treats all games as equal simply ignores the fact that not all games are of equal impact. To treat all games as equally important is to introduce inequity into the process.
I think any fan or any coach will tell you that some games are more important than others. I’ll give you just one example from this season. Rochester Institute of Technology has a win over Cornell this season. At the time, it seemed like a pretty big upset and one with post-season ramifications. But it pales in importance now for RIT in comparison to a 6-2 thumping the Tigers took at American International a couple weeks ago. Those two points in conference could be the difference between a first-round playoff bye or a first-round playoff game. You can repeat that scenario with dozens of teams this season.
(Don’t get me started on Atlantic Hockey’s playoff formula this year. My colleague Chris Lerch has been covering that well in his column. But it’s possible that a team could finish in a three-way tie for first place and have to play a first-round playoff game while the team that finishes in eighth place — EIGHTH! — would have a first-round bye.)
Acknowledging that some games are more important than others is exactly where the PairWise approach finds its strength. The NCAA rightly understands that certain games mean much more than others when comparing two teams. You as a fan, coach or player already know that.
The most important of the criteria in the PairWise is the one weighted most heavily: head-to-head competition.
Let’s look at a hypothetical. If, after conference playoffs are completed, North Dakota and Maine are otherwise equal in criteria, doesn’t it make sense that its two wins at Orono over the Fighting Sioux would give the Black Bears the upper hand? You can see the impact of those games in the PairWise Rankings comparison grid: the Maine comparison against North Dakota sticks out like a sore thumb.
The record against common opponents is another comparison used, as is a team’s record against the top 25 teams in the RPI. Doesn’t it make sense that how two teams do against teams they both play or how they fared against the top teams in the country would be a consideration in ranking or choosing one over another? Shouldn’t one team’s success against teams they both play or against the best competition they each face mean more than games against cellar dwellers? I think so. The NCAA has made it so.
The NCAA has a system that may not pick what a computer algorithm would select as the eleven best at-large teams, but it has a system that selects what it believes are the teams most deserving of a playoff bid. Comparing each pair of teams in games that mean more than other games does exactly that.
Would USCHO endorse replacing the RPI with a better computer ranking algorithm? I think we would. Enthusiastically. But I don’t think we’d ever support replacing the current method with a system that fails to look at the key games between teams being considered for an at-large bid.
Some games are indeed more equal than others.
