A new ranking systen for college hockey

[FreshFish]

Re: A new ranking systen for college hockey

Right, depends upon what you are trying to do.

At first it sounded like you were trying to develop a ranking system that (subject to conference auto-bid restriction) would pick the "best" 16 teams that "most truly earned" the right to play in the tournament.

Then someone started talking about 'predictions' and it almost sounded like a transition to card-counting in blackjack. By "predictive" are you trying to fill out the entire bracket at the start of the tournament? or are you allowed to 're-set' each round (as if you were in Vegas) and pick each round's winners?

I thought your original purpose (if I understood it correctly) was more intriguing from a technical standpoint; except that the sample size and relative dearth of out-of-conference opportunities in [men's ice] hockey vis-a-vis [men's] backetball, seems to limit the reliability of your data relative to Ken Pomeroy's work for example.

It would be interesting to see how Ken Pomeroy's basketball analyses would translate to [men's ice] hockey. His work nearly always is a better predictor of tournament success than the seedings.

btw, the way [men's] basketball is seeded, your chances of a #12 seed 'upsetting' a # 5 seed are something like 75%. The NYTimes had a very interesting article last week, for which I am too lazy to look up and post the link.

also, a similar analysis seems to indicate that having a 64-team tournament in women's division I basketball is just silly. There are a lot of really talented men available to fill rosters relative to really talented women (more alternative outlets for women? less potential for a big pro contract as motivator? who knows). Either a 32-team tournament or even a 24-team tournament (top eight get 1st-round bye) would appear to make a lot more sense.

 

[goblue78]

Re: A new ranking systen for college hockey

Well, what I've got (or am heading toward) is nothing more than a set of team power ratings that allows one to answer a range of questions. What you use it for depends on what question you have. So, as we've seen so far:
(1) Who is the strongest team not in the tournament: Wisconsin. Now that doesn't mean Wisconsin deserves to be in the tournament by any means. But based on these strength ratings, that's the answer. Who's the weakest team in the field? Air Force. Who's the weakest non-autobid? RPI. Note that these two answers aren't any different than the pairwise or any other method -- you would hope every ratings method didn't give you different answers to every question!
(2) Who has the highest probability, looked at right now, of getting to the Frozen Four, and by how much? What Regional has the highest probability of supplying the ultimate tournament winner, and by how much? What are the chances of conference X putting 1, 2, 3 or 4 players in the Frozen Four? Which conference has the highest probability of winning the tournament? All of these questions can be quantified by looking at the table in the last post.
(3) What are the chances that at least one four seed beats a one seed? It's 82 percent!
(4) But not only can you answer these questions, you can answer a bunch more besides about the confidence with which these results can be asserted. Thus, North Dakota is ranked highest, but given the relativity paucity of interconference games, how confident are we that their rating is "really" higher, than, say, Merrimack. I suspect most fans are highly overconfident about how confident they should be about such a ranking. One of the things I dislike about KRACH is that it gives us no obvious way to quantify our uncertainty. (There are ways, but it's complicated.)
(5) Who has better ratings? Good offensive teams or good defensive teams? What's the correlation between the two? There was an interesting discussion about Ryan Rondeau's leading the nation in GAA during the ECAC final by the announcers. One guy, I think it was Starman, argued that Rondeau isn't really as good as that number, but that Yale's defense is immensely aided by the fact that the offense keeps the puck for so long. Well, that's a research question -- does having a great offense, statistically, improve your defensive statistics? By how much?

So all I've got here are ratings which can be turned into matchup probabilties of wins. With all due respect to jcarter6976, if the model isn't based on theory I have no idea how you'd go about quantifying that. My theory is fairly simple: teams win games who outscore the other teams and the distribution of goals is close to Poisson. Any question you can answer with offensive and defensive power ratings of teams you can answer with this model -- and it serves as a check on it's own validity.

 

[FreshFish]

Re: A new ranking systen for college hockey

Do you factor in the (relatively) small sample size and the (relative) lack of out-of-conference play?

 

[goblue78]

Re: A new ranking systen for college hockey

Yep. Or rather, the statistical package does.

 

[Patman]

Re: A new ranking systen for college hockey

Do you factor in the (relatively) small sample size and the (relative) lack of out-of-conference play?

Most ranking systems aren't not intrinsically able to account for the uncertainty of the estimates in the presence of a) weak connection ties (lack of out-of-conference play), b) small sample sizes. Granted, their estimates reflects these things and does attempt to balance as best as it can... but most solutions are "maximum likelihood estimates" and can be very non-conservative in nature... for example: extreme ranks earlier in the year.

Its just a way of life in these situations.

 

[goblue78]

Re: A new ranking systen for college hockey

Sure.. but my point is that a maximum likelihood estimate combined with a variance-covariance matrix of the estimates can tell you how uncertain you are. An iterative procedure that yields KRACH can't, at least without a lot of extra calculations. It's one thing to know that estimates are unstable early in the season. It's another thing to quantify that uncertainty.

 

[FreshFish]

Re: A new ranking systen for college hockey

It's one thing to know that estimates are unstable ... It's another thing to quantify that uncertainty.

Interesting to consider how to rank some of the extraordinary achievements of the human mind....probably should start a new thread for this, even if I don't have 5,000 posts yet [sarcastic aside to some of the posters on the poll thread].

Without ranking them, and not being exclusive, I'd include Euclid's geometry, Newton/Leibniz calculus, Maxwell's equations (close to the running for # 1); Heisenberg's and Schrodinger's work; Bohr's model of the atom; the conceptualization behind the Michaelson-Morley experiment (the most influential 'failure' in history, perhaps?), Kekule's formulation of the atomic structure of benzine, Mendeleev's 'discovery' of the periodic table, Cantor's formulation of infinities, Godel's theory, and Einstein's big three. I'm not including anything technological here, that would be a separate list; these are all 'achievements of the imagination.'

 

[goblue78]

Re: A new ranking systen for college hockey

As monumental an achievement as this model is, I feel it falls somewhere short of those... I'd put it somewhere between the Pairwise and sliced bread.