Division I Rutter Rankings for 2013-2014

[EastFan1]

Re: Division I Rutter Rankings for 2013-2014

Here are the first Rutter Rankings of the year for games played through October 6, 2013

  	Team 		Rating 
1 	Minnesota 	3.42
2 	Wisconsin 	1.66
3 	Clarkson 	1.58
4 	North Dakota 	1.48
5 	Boston College 	1.31
6 	Cornell 	1.30
7 	Boston Univ 	1.29
8 	Ohio State 	1.15
9 	Harvard 	1.04
10 	Minn. Duluth 	0.90

Complete rankings can be found here: http://math.bd.psu.edu/faculty/rutter/WomensRankings.html

FAQ

Q. [Insert team here, like Cornell] hasnt played a game yet. Ho'w can they be ranked?
A. The power of Bayesian statistics. Each team has a prior rating based on last year, so early season rankings are possible. As more games are played, the effect of the prior is reduced.

Q. Is home ice advantage included?
A. No. Although I have estimates of home ice advantage for both the entire division and for each team, they are not included in these rankings. It emulates the NCAA criteria in this respect.

Q. Is margin of victory (MOV) included?
A. No, just wins, losses, and ties. Again, the NCAA doesn't use margin of victory and since the games are low scoring, I haven't found a satisfactory way to include MOV.

Q. How do you include ties?
A. If you look at the web page, you will see how ties are included. In short, I estimate a region that includes a probability of a tie which is larger when the teams are closely ranked. No "tie is half a win, half a loss" simplification here.

Q. How does your method compare to RPI?
A. The two are very different. I think my system (and KRACH) are a much better reflection of the quality of teams as statistical models are used as opposed to arbitrary alegbra. But I am biased. Someday, I will do a really complete comparission of the methods.

When is the impact of last season 100% eliminated from your rankings?

 

[pokechecker]

Re: Division I Rutter Rankings for 2013-2014

Actually, this isn't obvious and not all fans know this. You are conflating your personal opinion on what should be more important with objective fact.

Maybe you should start sitting with more intelligent fans? It is obvious from their comments that both Mark Johnson and Brad Frost believe how their teams are playing at the end of the season is what’s most important. That’s good enough for me.

You assume that the team that wins the national championship is by definition the best and then let this assumption do all of your heavy lifting and putting a superficial structure of logic on top of it.

Most, if not all people would say that the team that wins the NCAA tournament is the best (so I am not alone in that regard, it seems you are the one with the odd perception). And when you consider the teams that have won the NCAA tournament have also won their conference tournament (with one exception), and have overwhelmingly also won their conference title (9 of 13 times), who are you trying to kid (besides yourself) that the NCAA champion is not the best team?

 

[pokechecker]

Re: Division I Rutter Rankings for 2013-2014

1) Where are you finding the archived rutter rankings? I don't see them on the website.

Nor do I, I was referring to the rankings (seeding) for the NCAA tournament. In that regard I agree with the Rutter ranking, the NCAA (and pairwise) is not a very accurate ranking.

2) When you say the Rutter rankings didn't predict the national champion, which rutter rankings are you referring to? one at the beginning of the season? one just before the Frozen Four? one updated after the championship game?

The one just before the NCAA tournament. Again, I wasn’t referring to the Rutter ranking, I’m referring to ANY ranking. Let’s take a step back for a moment. What is the purpose of the Ranking? Without answering this question you cannot possibly say how accurate they are. If the purpose is to fuel discussion, any ranking system will suffice. Indeed, the less accurate the ranking, the more likely it will fuel discussion. If it is to secure tenure, obviously the Rutter Ranking is an excellent one.
When I talk about ranking it is for the purpose of selecting and seeding teams in the NCAA tournament. The athletes deserve a fair ranking for their effort. But some people have a different definition of fair from others. Some even believe that “making it easier for my team” is the fair one. I Don’t.
The accepted version of fair seems to be to have the best team square off with the bottom team, the 2nd best team square off with the 2nd to the bottom team, and so on …
The problem comes in when you try to select which team is #2, #3, #4, ….
Which is the sole reason, IMO, to try to have as accurate rating system as possible, to select which teams compete, and how they are seeded, in the NCAA tournament. For me, any other ranking system is frivolous.

Again, I don't know where you are finding the archived Rutter rankings, but to say "6 out of 13 have won the title, that's worse than flipping a coin" is either really misinformed or really disingenuous because "win a national championship" or "not win a national championship" are not the only two results a team can have in a season.

I was referring to the rankings (seeding) for the NCAA tournament. In that regard I agree with the Rutter ranking, the NCAA (and pairwise) is not a very accurate ranking. I was referring to a person who had gone to Vegas and was deciding to place a bet, should he go with the #1 ranked team or somebody else? He may as well as flipped a coin to help him decide, the NCAA ranking system picks the winner less than half the time.
But I can’t say that I agree with the Rutter rankings either. They are claimed to be more accurate, yet there does not seem to be proof to show it.

all the answers to your other questions should be obvious

 

[TonyTheTiger20]

Re: Division I Rutter Rankings for 2013-2014

I was referring to a person who had gone to Vegas and was deciding to place a bet, should he go with the #1 ranked team or somebody else? He may as well as flipped a coin to help him decide, the NCAA ranking system picks the winner less than half the time.

Bro, do you even math?

 

[TonyTheTiger20]

Re: Division I Rutter Rankings for 2013-2014

And are you going to respond to my rebuttal? Or just respond with a trolltastic "duh the answer to your question should be obvious"?

I'm going to copy and paste for you:

Fourthly,

Only six times has the rankings accurately predicted the eventual winner. (less than by flipping a coin or by chance).

You realize the mathematical flaw you made here right? You do know that if a ranking correctly picked the national champion out of a field of, what, 52?, almost half the time, that that's pretty dod gamb good?

Finally,

Only once have the rankings predicted the order of finish for the top four teams (2005).

Because do you realize how hard it is to predict the finishing order of 4 roughly equal items in a set?

Even if:

1) You had JUST 4 teams playing women's hockey,
2) With each team twice as good as the next team (Team A = .53, Team 2 = .27, Team Gamma = .13, Team IV = .07) -- which is not even in the same STRATOSPHERE as reality)
3) Completely ignoring the fact that there are dozens of other teams in reality as opposed to this example!!


Even with ALL THAT, the chances of picking the finishing order of all four correctly by choosing best to worst -- -- are just one in five. One in five! With just a four team league and each team being twice as good as the next best team!

And if you assume four roughly equal teams, that drops to one in TWENTY FIVE. Again, ignoring the fact that there are DOZENS of other teams not even being considered for this 'top 4' argument.

Are you kidding me with this mathematical nonsense?

Yes, you must be, so I don't know why I'm doing this, I really don't, but it's an impressive bit of troll bait I'm running with here.

 

[joegrav]

Re: Division I Rutter Rankings for 2013-2014

Nor do I, I was referring to the rankings (seeding) for the NCAA tournament. In that regard I agree with the Rutter ranking, the NCAA (and pairwise) is not a very accurate ranking.



The one just before the NCAA tournament. Again, I wasn’t referring to the Rutter ranking, I’m referring to ANY ranking. Let’s take a step back for a moment. What is the purpose of the Ranking? Without answering this question you cannot possibly say how accurate they are. If the purpose is to fuel discussion, any ranking system will suffice. Indeed, the less accurate the ranking, the more likely it will fuel discussion. If it is to secure tenure, obviously the Rutter Ranking is an excellent one.
When I talk about ranking it is for the purpose of selecting and seeding teams in the NCAA tournament. The athletes deserve a fair ranking for their effort. But some people have a different definition of fair from others. Some even believe that “making it easier for my team” is the fair one. I Don’t.
The accepted version of fair seems to be to have the best team square off with the bottom team, the 2nd best team square off with the 2nd to the bottom team, and so on …
The problem comes in when you try to select which team is #2, #3, #4, ….
Which is the sole reason, IMO, to try to have as accurate rating system as possible, to select which teams compete, and how they are seeded, in the NCAA tournament. For me, any other ranking system is frivolous.



I was referring to the rankings (seeding) for the NCAA tournament. In that regard I agree with the Rutter ranking, the NCAA (and pairwise) is not a very accurate ranking. I was referring to a person who had gone to Vegas and was deciding to place a bet, should he go with the #1 ranked team or somebody else? He may as well as flipped a coin to help him decide, the NCAA ranking system picks the winner less than half the time.
But I can’t say that I agree with the Rutter rankings either. They are claimed to be more accurate, yet there does not seem to be proof to show it.

all the answers to your other questions should be obvious

So you're talking about the pairwise rankings? Well, a lot of people, the creator of the Rutter Rankings included, seem to feel that the PWR are based off of a faulty rating system in the RPI. I agree with that. I think the RPI is not a very good ratings system. It's far too simplistic. So I guess we're in agreement?

But even with that said, your comments about the predictive ability of the PWR rankings are still quite mathematically flawed. Saying "only 6 of 13 won a title, so it was worse than flipping a coin" is really dumb. Maybe if you compared the results to rolling an 8-sided die, since there are 8 teams in the tournament. A fairer analysis would be to see what the average finishing position is of the team ranked #1 in the tournament, vs. #2, #3 and #4. I'm willing to bet that the average finish is highest for the #1-ranked team.

The other flaw in this argument though is that the bottom four teams aren't seeded. So the theoretical reward for being #1 overall should be getting to face a weaker team in the first round, but that's not the case in NCAA women's hockey; the quarterfinal matchups are seeded geographically. So really, you basically have no point, so far as I can tell.

The accepted version of fair seems to be to have the best team square off with the bottom team, the 2nd best team square off with the 2nd to the bottom team, and so on …
The problem comes in when you try to select which team is #2, #3, #4, ….
Which is the sole reason, IMO, to try to have as accurate rating system as possible, to select which teams compete, and how they are seeded, in the NCAA tournament. For me, any other ranking system is frivolous.

Well, obviously, you do not understand how the first round pairings work, so I can see why you're confused.

The ratings mean different things to different people; I look at them as an interesting mathematical way to stack up how the teams compare and potentially make predictions on how future games will go.

It is obvious from their comments that both Mark Johnson and Brad Frost believe how their teams are playing at the end of the season is what’s most important.

Swell. Johnson and Frost aren't "all fans." Johnson and Frost are paid to do a very particular job, which is win championships. The way to win the championship is to win the NCAA tournament, so peaking at the end of the season is the way they do their job.

That's a separate question entirely from determining who the "best team" is as of November, December, January, or any particular moment in time. Being the best team at that time of the season is not their job, but it's a question/debate that is interesting to fans.

As far as I can tell, the only discernible point you seem to be making is that the RPI/NCAA ratings system should be tweaked to further emphasize the results of games down the stretch. I have to say that I wholeheartedly disagree with you on that one. You play a long, grinding season, and every game in it should count. Otherwise, why play such a long season? Why should fans go to games in October and November? We already have a way to reward teams who are playing their best hockey down the stretch; that's why we have playoffs.

 

[pokechecker]

Re: Division I Rutter Rankings for 2013-2014

Swell. Johnson and Frost aren't "all fans." Johnson and Frost are paid to do a very particular job, which is win championships. The way to win the championship is to win the NCAA tournament, so peaking at the end of the season is the way they do their job.

That's a separate question entirely from determining who the "best team" is as of November, December, January, or any particular moment in time. Being the best team at that time of the season is not their job, but it's a question/debate that is interesting to fans.

the NCAA tournament seems to be the moment in time when nearly everyone except you is interested

do you suppose the Wisconsin Badgers will celebrate their 2nd period last Friday at Ridder? It will probably be the most dominant and best any team plays this year against any opponent in any rink. Do you suppose the Gophers are still celebrating their sweep of Wisconsin?

maybe that's why those teams win and yours doesn't (although I seriously doubt BC is any different from UW & UM in that regard)
really, it's only a problem in your mind

 

[ARM]

Re: Division I Rutter Rankings for 2013-2014

So if you're unhappy with the PWR, why blame Rutter's rankings? They clearly are not the problem. He's done a fine job, so I don't follow your point. Obviously, we all care more about who wins in March, but that is a long way off when it is October.

 

[Eeyore]

Re: Division I Rutter Rankings for 2013-2014

Most, if not all people would say that the team that wins the NCAA tournament is the best (so I am not alone in that regard, it seems you are the one with the odd perception). And when you consider the teams that have won the NCAA tournament have also won their conference tournament (with one exception), and have overwhelmingly also won their conference title (9 of 13 times), who are you trying to kid (besides yourself) that the NCAA champion is not the best team?

I'm not trying to kid myself. What I'm saying is that any argument that states that the team that wins the NCAA championship is automatically the best team is not a statement of fact; it's a value judgment. It is only true in the tautological sense that constructing a definition of "best" will return a value for that team that meets that definition. It doesn't mean anything else. And no matter how many people you cite who believe as you do and no matter what jobs they perform, that isn't going to change. It may be a widely held value judgment, but that's still all it is and all that it can be.

And for what it's worth, there are a lot more people who think like I do than you seem to realize.

 

[Eeyore]

Re: Division I Rutter Rankings for 2013-2014

As far as I can tell, the only discernible point you seem to be making is that the RPI/NCAA ratings system should be tweaked to further emphasize the results of games down the stretch. I have to say that I wholeheartedly disagree with you on that one. You play a long, grinding season, and every game in it should count. Otherwise, why play such a long season? Why should fans go to games in October and November? We already have a way to reward teams who are playing their best hockey down the stretch; that's why we have playoffs.

I'll even go beyond this. I wish we had a system that rewarded the team that can win three straight games in March less than one that can be consistently good for five months. There are a lot of things that go into that aesthetic position. I acknowledge that the majority of American fans disagree with me on this and so I, too, end up valuing playoff championships more highly because that is the stated objective but I wish people thought differently.

I much prefer the approach of European domestic soccer leagues. (Those in other places may work the same; I don't know.) The champion of the EPL or the Bundesliga is the one that finishes the season with the best record. In England they also play a knockout tournament for the FA Cup but this isn't as important in everyone's mind as the league title. This means that every game played is truly important. It also means that top success in domestic play accrues to virtues that I respect more.

As long as I'm yelling at clouds I should also say that I don't really get the American abhorrence of a game ending up in a tie. Sometimes the two teams played so close to equally well that the record should reflect that.

 

[TonyTheTiger20]

the NCAA tournament seems to be the moment in time when nearly everyone except you is interested

do you suppose the Wisconsin Badgers will celebrate their 2nd period last Friday at Ridder? It will probably be the most dominant and best any team plays this year against any opponent in any rink. Do you suppose the Gophers are still celebrating their sweep of Wisconsin?

maybe that's why those teams win and yours doesn't (although I seriously doubt BC is any different from UW & UM in that regard)
really, it's only a problem in your mind

You're making two different arguments here.

1) They don't award titles to teams based on how they play in the middle of the season. However, they DO reward teams for how they played over the course of the season by giving them favorable seedings. That is why mid-season games are important. I think you agree with this.

2) As Joe said, we have a very strong way of rewarding the teams who play well at the end of the year. It's why we have playoffs and don't award an NCAA title to the team at the top of the Pairwise.

You play a season to be eligible to win the title. You play the playoffs the award the title.

Given that it appears everyone here, including you, agrees with this, I'm struggling to find the point you're trying to make.

To your original comment several posts back, the "point" of specifically women's hockey rankings is to identify which 8 teams are deserving of being eligible to win the title, and to seed the most deserving of those teams based on the entire body of work of the season.

Early season games have nothing to do with who is the best at the end of the year, but it does factor in to your seeding for the tournament.

What are you trying to get at here? That the regular season should not count? That the first half of the season should be exhibition games? Or maybe that we should make the conference champions the top 4 seeded teams? If that is your argument, it is a perfectly legitimate one, but get to the point.

What would your ideal situation be? Please, tell us, because it is very tiring arguing with someone who no one can tell what it is they are trying to argue.

 

[TonyTheTiger20]

Re: Division I Rutter Rankings for 2013-2014

In addition I pose to you the following question.

Let's say BC scored a fluky (or not fluky, doesn't matter) goal in overtime to beat Minnesota in the semis last year. Would you argue that Minnesota was not the best team in the country last year?

 

[ARM]

Re: Division I Rutter Rankings for 2013-2014

Let's say BC scored a fluky (or not fluky, doesn't matter) goal in overtime to beat Minnesota in the semis last year. Would you argue that Minnesota was not the best team in the country last year?

Or to make it muddier, the Lamoureux shot that hit the crossbar in OT bounced into the goal and UND eliminated Minnesota. UND then lost in the semis to BC, and the Eagles turned around and lost to BU. Going by "the team playing the best at the end", the only logical answer is that BU was the best team. In any case, they'd own the Championship Belt in addition to that insignificant piece of hardware the NCAA hands out, so our label of "best" is likely to mean little to them no matter where we would bestow it.

 

[wwhyte]

Re: Division I Rutter Rankings for 2013-2014

I'll even go beyond this. I wish we had a system that rewarded the team that can win three straight games in March less than one that can be consistently good for five months.

I much prefer the approach of European domestic soccer leagues. (Those in other places may work the same; I don't know.) The champion of the EPL or the Bundesliga is the one that finishes the season with the best record. In England they also play a knockout tournament for the FA Cup but this isn't as important in everyone's mind as the league title. This means that every game played is truly important. It also means that top success in domestic play accrues to virtues that I respect more.

I agree with a lot of this. It's true (I'm from Ireland) that winning the league counts for more than winning the cup, and I like that system more (though maybe because I'm used to it). However, one reason why it counts so much is because all the teams in the Premiership play each other the same number of times, equal numbers of home and away, on an equal footing. You can genuinely say that the team that comes out on top is the best.

In North America, and with college sports, the long distances and tight travel budgets mean that you can't have everyone play everyone. That's why we need the Pairwise and Rutter and all the other systems: the games themselves don't give a thorough test of quality, so you need some statistical method on top of that. It would be great if there was a superleague with promotion and relegation, and the regional leagues acted as feeders to it, but that's not the system we've got, in part I presume because the NCAA is modeled after the franchise-owned leagues of US professional sports rather than the European model of independent leagues with clubs moving in and out of particular divisions or being promoted or relegated from one league to another.

As it is, my personal feeling is that the Hockey East, WCHA, etc. league titles are the "real" titles and the conference championships are just fun, because in those conferences everyone does play everyone. For example, BC won the HE championship in 2011, but as far as I'm concerned they've never won "Hockey East". But the NCAA tournament is the only true national event, so winning it counts in some sense. Even so, in ARM's example above, I'd say that we would end the year simply not knowing who the best team was but probably agreeing it was the Gophers.

Bring on the superleague!

(Apologies for mixing up conference/league/division above, half of my mind is thinking in European and half is thinking in American).

 

[LakersFan]

Re: Division I Rutter Rankings for 2013-2014

When is the impact of last season 100% eliminated from your rankings?

It takes about 15 games for the effects to be negligible. At that point, the data overwhelms the prior information.

 

[ManBehindTheCurtain]

Re: Division I Rutter Rankings for 2013-2014

It takes about 15 games for the effects to be negligible. At that point, the data overwhelms the prior information.

Just by way of information, it appears you have a missing or broken link (and possibly a typo) on your D1 front page.

The last sentence of the third paragraph reads "For information of the prior distributions, click here", but there is nothing to click at "here".

 

[Eeyore]

Re: Division I Rutter Rankings for 2013-2014

It takes about 15 games for the effects to be negligible. At that point, the data overwhelms the prior information.

Am I correct that the effect of prior seasons never really goes away, just that it converges to zero?

 

[EastFan1]

Re: Division I Rutter Rankings for 2013-2014

Am I correct that the effect of prior seasons never really goes away, just that it converges to zero?

Ivy's won't get to 15 games until sometime in January. What is the max number of games that goes into your formula?

Not sure there is any value in having a carry-over effect unless you are trying to determine the best program, not team, of the past couple few years.

 

[TonyTheTiger20]

Re: Division I Rutter Rankings for 2013-2014

Not sure there is any value in having a carry-over effect unless you are trying to determine the best program, not team, of the past couple few years.

The value is in having some kind of at least somewhat meaningful ranking early in the year.

For example when all the team's have only played like 3 games, Rutter's 'carry-over' is going to give you far more accurate results (even with using some of last year's numbers) than, say, RPI, which will give you weird things until everything starts to tie together.

 

[EastFan1]

Re: Division I Rutter Rankings for 2013-2014

The value is in having some kind of at least somewhat meaningful ranking early in the year.

For example when all the team's have only played like 3 games, Rutter's 'carry-over' is going to give you far more accurate results (even with using some of last year's numbers) than, say, RPI, which will give you weird things until everything starts to tie together.

If you are saying using last years computer rankings may give you a good prediction of the next season in some years, sure. But I'm not so sure about this year.