KRACH Ratings

[TonyTheTiger20]

I've been complaining all year that USCHO's KRACH ratings are being calculated incorrectly, so I finally did something about it.

Here's a link to the correct KRACH ratings, which do not omit games against Merrimack like USCHO's calculation is doing. I put together an Excel spreadsheet using a couple clever (if I do say so myself) little tricks that do the calculations, and like the Pairwise Predictor, also allows you to change results if you want to see how it affects the rankings.

Let me know if it does anything wonky.

 

[Call It]

Re: KRACH Ratings

I've been complaining all year that USCHO's KRACH ratings are being calculated incorrectly, so I finally did something about it.

Here's a link to the correct KRACH ratings, which do not omit games against Merrimack like USCHO's calculation is doing. I put together an Excel spreadsheet using a couple clever (if I do say so myself) little tricks that do the calculations, and like the Pairwise Predictor, also allows you to change results if you want to see how it affects the rankings.

Let me know if it does anything wonky.

It doesn't look right to me, BC is only ranked #1. I would think they would have the top 4 spots locked up? They are currently 20-0 playing a very weak schedule!

 

[TonyTheTiger20]

Re: KRACH Ratings

It doesn't look right to me, BC is only ranked #1. I would think they would have the top 4 spots locked up? They are currently 20-0 playing a very weak schedule!

I get the joke but honestly the uproar over BC's schedule this year is kind of misplaced. It's been average at worst as pretty good at best.

KRACH has them with the 11th best SOS in the country and RPI (admittedly a crap way of ranking teams but mathematically crap nonetheless) has them 6th, ahead of Wisconsin in 11th and fractionally behind Minnesota in 5th.

 

[D2D]

Re: KRACH Ratings

...RPI (admittedly a crap way of ranking teams but mathematically crap nonetheless) has them 6th, ahead of Wisconsin in 11th and fractionally behind Minnesota in 5th.

Grant, when it comes to math you're way more knowledgeable than I'll ever pretend to be, but if RPI is mathematically crap then why even bring it up?

 

[5mn_Major]

Re: KRACH Ratings

I get the joke but honestly the uproar over BC's schedule this year is kind of misplaced. It's been average at worst as pretty good at best.

KRACH has them with the 11th best SOS in the country and RPI (admittedly a crap way of ranking teams but mathematically crap nonetheless) has them 6th, ahead of Wisconsin in 11th and fractionally behind Minnesota in 5th.

The RPI is crazy crap. Its got Q above Minnesota...just because of a single tie vs. a loss. Amazingly the SOS difference between the two doesn't make up for that.

 

[ARM]

Re: KRACH Ratings

January is when BC's schedule really dips, with five of the seven games against two of the teams having the most losses in the country. At one point we may have thought that games versus BU and Harvard would mitigate that, but now I'm not sure how good either of those teams are.

 

[TonyTheTiger20]

Re: KRACH Ratings

Grant, when it comes to math you're way more knowledgeable than I'll ever pretend to be, but if RPI is mathematically crap then why even bring it up?

My point was mostly that even if it's not the best way to rank teams, it's still objective. That's what I meant by italicizing "mathematically" haha

We're limited in the number of objective ways to measure strength of schedule and even if RPI is crap compared to KRACH it's still better than someone saying "WHAT A WEAK SCHEDULE!!!" based on feeling and gut.

 

[TonyTheTiger20]

Re: KRACH Ratings

January is when BC's schedule really dips, with five of the seven games against two of the teams having the most losses in the country. At one point we may have thought that games versus BU and Harvard would mitigate that, but now I'm not sure how good either of those teams are.

You're not kidding -- I was looking through BC's schedule earlier this morning actually and the soft part gets really soft in the 2nd half. Mack, PC, and Vermont are 8 of our 14 remaining games.

In the first half 10 of our 20 games were against Cornell, SLU, UMD, BU, and NU.
In the second half 5 of our 14 games are against BU, NU, and HU.

That's a measurable difference in competitive games. But moreover even the bad teams of the first half (UConn, UNH, Maine) are better, IMO, than the bad teams of the 2nd half (Merrimack, PC, Vermont).

I'm a little (a lot) worried we're going to be in for a reality check come March because we might hang 10 goals on a couple of those teams and start feeling too good about ourselves again.

 

[Fishman'81]

Re: KRACH Ratings

My point was mostly that even if it's not the best way to rank teams, it's still objective. That's what I meant by italicizing "mathematically" haha

We're limited in the number of objective ways to measure strength of schedule and even if RPI is crap compared to KRACH it's still better than someone saying "WHAT A WEAK SCHEDULE!!!" based on feeling and gut.

Just dropping-in to this thread as a very casual fan of the women's game, and without an axe to grind... But yes.

Any credible statistical model must recognize perfection if perfection exists, especially with a 20-game body of work to evaluate. The "eye-test" speculation can't reasonably be factored-in... It's all about the numbers, period.

(And so it follows that even one loss to a statistically bad team by BC might drop them like a stone. It's fair, either way you care to slice it.)

 

[5mn_Major]

Re: KRACH Ratings

(And so it follows that even one loss to a statistically bad team by BC might drop them like a stone. It's fair, either way you care to slice it.)

If that's true, then a loss or numerous ties with a much weaker schedule yet would have dropped Q significantly. Its still on knocking on the door at #4.

 

[TonyTheTiger20]

Re: KRACH Ratings

If that's true, then a loss or numerous ties with a much weaker schedule yet would have dropped Q significantly. Its still on knocking on the door at #4.

Man some of you are just so unwilling to accept eastern teams being good, huh...

 

[ARM]

Re: KRACH Ratings

Man some of you are just so unwilling to accept eastern teams being good, huh...

The specific issue here, though, is Q's schedule. In nonconference, the highest KRACH-rated opponent was Mercyhurst, who the Bobcats might have expected to be a strong opponent, but so far the Lakers haven't been. Q still has BU coming up, and maybe by the end of the season, playing the Terriers will prove to be more formidable than it looks now. The ECAC itself, outside of Quinnipiac, is loaded with teams that are good, but not really good. So while the Bobcats schedule isn't out and out weak, more like middle of the road, it definitely lacks sizzle. No BC or Northeastern, the two most consistent teams from Hockey East. Clarkson has been the highest-ranked opponent, and the Golden Knights are hard to rank, because their schedule is even weaker than that of Quinnipiac.

 

[TonyTheTiger20]

Re: KRACH Ratings

That's the thing though, we have very good ways to rank teams with different schedules using math and objectivity, and now people are poo-pooing KRACH because it has/had Quinnipiac near Minnesota.

 

[ARM]

Re: KRACH Ratings

That's the thing though, we have very good ways to rank teams with different schedules using math and objectivity...

These systems leave the door open to second guessing because they do such a poor job of evaluating strength of schedule. There is no way that Minnesota has played the second-toughest schedule. Compare it to OSU at 11th. They both played Penn State twice, so that should cancel. They've played the same WCHA schedule, except for each other. So the differences are the Gophers played OSU (x 3), and Yale (x 2), while the Buckeyes played UM (x 3), Lindenwood (x 2), BU, and Vermont. It isn't obvious to me why the first set would work out as being tougher than the second.

 

[Eeyore]

Re: KRACH Ratings

The single biggest problem with KRACH is that it's opaque. In the end, I don't think it's possible to build a good, objective ratings system that is also transparent in how it arrives at its conclusions; those objectives are mutually exclusive. That's a flaw in them, because the vast majority of people can't look at them and figure out exactly how the system arrived at the conclusions it did, and the few people who do understand the math and can pull it apart to examine how it works can't really explain all of that in a way that everyone else can grasp it.

 

[TonyTheTiger20]

Re: KRACH Ratings

The single biggest problem with KRACH is that it's opaque. In the end, I don't think it's possible to build a good, objective ratings system that is also transparent in how it arrives at its conclusions; those objectives are mutually exclusive. That's a flaw in them, because the vast majority of people can't look at them and figure out exactly how the system arrived at the conclusions it did, and the few people who do understand the math and can pull it apart to examine how it works can't really explain all of that in a way that everyone else can grasp it.

I used to think KRACH was opaque too until I had to dive deep into it to build the spreadsheet.

KRACH is seriously brilliant. You take the percent chance each team has of winning each of their games (calculated using the KRACH ratings) and add them up, and make it so all the KRACH ratings are such that those winning percentages add up to the actual number of wins that team has.

So, for example, let's say Minnesota played Duluth (let's say they have a rating of 150), Mankato (rating of 50), and Wisconsin (rating of 500), beating the first two and losing to Wisconsin. Minnesota's KRACH rating is the value such that the sum of Minnesota's chances of winning those games equals 2 (the number of wins Minnesota actually got in those three games).

The chance Minnesota has of winning a given game is (Minnesota's KRACH rating) divided by (Minnesota's KRACH rating + opponent's KRACH rating).

SO if Minnesota's rating is 359:

[359/(359+150)] + [359/(359+50)] + [359/(359+500)]
= 0.705 + 0.877 + 0.418
= 2.000

Which is exactly equal to the number of wins Minnesota got in those three games. Therefore, 359 is Minnesota's rating.

KRACH is not opaque. It's the most logically constructed ranking formula I've ever seen.

 

[TonyTheTiger20]

Re: KRACH Ratings

KRACH also has the great property that winning will ALWAYS help your rating and losing will ALWAYS hurt your rating. That is, there are no "bad wins" like in RPI.

No matter how bad the team is that you're playing, your expected winning percentage will always be less than 1. So a win will always add more wins to your win total (1) than your chance of winning the game (like BC has a 0.997-ish chance of beating Union, for example), increasing their rating.

Same with losing -- losing will always lower your rating. Union losing to BC will always add fewer wins to Union's win total (0) than their chances of winning the game (0.003-ish), lowering their rating.

It also gives you a method to see what the percent chance team A has of beating team B: (Team A)/(Team A + Team B).

It's seriously fantastic and digging into it was one of the most interesting things I've ever done because I'm a raging nerd.

 

[ARM]

Re: KRACH Ratings

KRACH is seriously brilliant.

If so, then it shouldn't report SOS at all, because I maintain that piece is flawed. After Minnesota and Ohio State have played four times, it still thinks that the Gophers have played a considerably tougher schedule.

 

[Eeyore]

Re: KRACH Ratings

I used to think KRACH was opaque too until I had to dive deep into it to build the spreadsheet.

Okay, so what is the SoS? The KRACH explanation says that it's the weighted average of the KRACH ratings of your opponents, but my calculations of that come out very different than what is shown either on the USCHO KRACH page, or your version. Using the USCHO page, I come up with a SoS for Minnesota of 386.96 and for Ohio State of 556.70, so KRACH is, at a minimum, opaque in the sense that it's not clear what the weighting on that average is. Further, I'm not sure that just taking a weighted average of your opponents' ratings is a particularly meaningful measure of SoS given the tendency of KRACH to head towards infinity as a team gets really good.

 

[Fishman'81]

Re: KRACH Ratings

The KRACH is indeed transparent if one is willing to break it down...And while it's a natural fact that no statistical metric is perfect, the KRACH is by far the best I've seen of the very few available.

I'd certainly be interested in looking at a better calculation if anyone knows of one.