2010-2011 Rutter DI Computer Rankings Thread

[comefrombehind]

Re: 2010-2011 Rutter DI Computer Rankings Thread

I stand corrected, although Wisconsinites probably don't consider any rink in Minnesota a neutral site!

 

[IceIsNice]

Re: 2010-2011 Rutter DI Computer Rankings Thread

I stand corrected, although Wisconsinites probably don't consider any rink in Minnesota a neutral site!

Attendance per the Box Score was 307. (Me plus 306). I don't think the 20 or so of us who were there supporting the Lakers were enough to create a substantial home ice advantage. The game might have been in Minnesota, but I'd still dare say the "Home Ice Advantage" leaned toward those annoying furry little creatures.

 

[granddaddyscout]

Re: 2010-2011 Rutter DI Computer Rankings Thread

Food for post-season thought -- Cornell won 3-0 this year in Erie, PA. UW's last shutout in this city on a real lake was in 2000. Number of players on the current rosters from those games: Cornell 20, Wisconsin 0.[/QUOTE]

Not sure it matters any more than the 5-0 game in Boston to win it all. Also for comparison sake they share two other common opponents Robert Morris on the road for the Badgers 2 games 14-2 over all vs Home and Away I believe for Big Red 13-3. Then RPI vs the Badgers 13-0 as compared to 12-1 for Cornell. So they can both beat up on the under privileged, but UW has proven they can beat lots of other strong teams as well. Cornell has a great team, I didn't grow up in Wisconsin, and I am more of a WCHA fan than any one team. My point is simply that UW in my opinion has earned the top spot until beaten by Cornell. Cornell's schedule should preclude them from the top spot unless they win it all!! Going forward they need to play tougher teams outside their conference.

 

[LakersFan]

Re: 2010-2011 Rutter DI Computer Rankings Thread

That is an understatement your "lesser competition" has a collective record of 19-52-11 with several of the wins coming against each other. Clearly the computer and the eastern pollsters don't really adjust for the strength of schedule effectively. UW has played 17 games against the 10 teams shown above; the only common opponent is Mercyhurst. Cornell split with them and UW won 7-4. Any ranking that places Cornell above UW at this point can't be taken seriously. Cornell is good, but really has yet to prove how good they are, UW has proven it several times already. Just my opinion

A ranking that orders of the top two teams based on the results of three games can't be taken seriously, in my opinion. My algorithm takes into account ALL the games each team has played, including UW losses to UMD and UM. My algorithm also doesn't reward victories against teams with low ratings as much as it does a win against a quality team. This can be seen in the graph from my website shown below:

Notice that Cornell's rating (the blue line as of 2/1/11) does not increase very much during its long winning streak before the second Mercyhurst game (game days30-45), at which point it jumps a little. You can also see how UW was rewarded for the win over UMD, followed by the penalty of the two ties against UMD and UM. My philosophy is to never penalize a team for a win, but I don't have to give them a lot of credit either. Losses and ties against teams you should defeat will always cost you.

The other thing you can see is that Cornell and UW are in a virtual tie for first place, with just .012 rating points separating the two teams. To put this in perspective, the difference in quality between UW and UM is 47 times greater than the "difference" in quality between Cornell and UW.

 

[ARM]

Re: 2010-2011 Rutter DI Computer Rankings Thread

Attendance per the Box Score was 307. (Me plus 306). I don't think the 20 or so of us who were there supporting the Lakers were enough to create a substantial home ice advantage. The game might have been in Minnesota, but I'd still dare say the "Home Ice Advantage" leaned toward those annoying furry little creatures.

As one whose team just played Bucky in front of 10K fans, I find your attempt to claim Mercyhurst was at a disadvantage in front of a couple hundred apathetic Husky fans and a few curious Gopher fans "Shannonesque".

 

[IceIsNice]

Re: 2010-2011 Rutter DI Computer Rankings Thread

As one whose team just played Bucky in front of 10K fans, I find your attempt to claim Mercyhurst was at a disadvantage in front of a couple hundred apathetic Husky fans and a few curious Gopher fans "Shannonesque".

Hey, that really, really hurt! LOL No, not saying we were disadvantaged, but it's equally hard to claim that this was your typical Badger road game, either. The classification of this game as a neutral site contest seems to me to be remarkably accurate.

 

[granddaddyscout]

Re: 2010-2011 Rutter DI Computer Rankings Thread

A ranking that orders of the top two teams based on the results of three games can't be taken seriously, in my opinion. My algorithm takes into account ALL the games each team has played, including UW losses to UMD and UM. My algorithm also doesn't reward victories against teams with low ratings as much as it does a win against a quality team. This can be seen in the graph from my website shown below:

Well that clears it all up, your graph explains everything clearly!

One question though. How does your algorithm account for one team having a schedule where they play 25 of their 29 games against teams they should beat 10 times out of 10 games as compared to a schedule where a team plays 22 of its 34 games against teams that should win 2-3 times out of ten. In my mind they are virtually incomparable. Playing in the WCHA provides its teams with the best preparation possible for the NCAA run. However, players are more likely to get dinged up. You playing the top team 4 times, it is tough to beat a strong team 4 times. Cornell played 2 games against a top 10 opponent, they went 1-1. UW has played 10 games against UM, UDM, and UND where they are 6-2-2 or 8-2 with the shoot outs. They also played 6 games against OSU and BSU where they are 6-0. Cornell is a very good team with lot's of talent, but try to remember that last year was not representative of the best NCAA women's hockey. Cornell will have to play much stronger teams if it makes it to the Frozen Four. It is possible that the tournament will include 4 WCHA teams, if that is the case UW will have played 13 games against tournament teams while Cornell will have played 2 equivalent games. I don't think you can really adjust for a schedule delta that wide. Just my opinion, I hope Cornell plays UW for the Championship. They may need to play a tough WCHA team in the 1st round though, so we will see who's theory proves to be most accurate.

 

[dave1381]

Re: 2010-2011 Rutter DI Computer Rankings Thread

I actually think granddaddyscout has a point.

The problem people have with the ranking boils down to this: when you take the teams Wisconsin and Cornell have lost to, Wisconsin is 5-2-2 against those teams, Cornell is 1-1. Otherwise both teams have a lot of wins, which for the sake of simplicity let's basically saw are the same (probably Wisconsin's are stronger but oh well). Most posters, myself included, are more impressed by Wisconsin's results than Cornell. The rankings are not.

Lakersfan, you say "A ranking that orders of the top two teams based on the results of three games can't be taken seriously." I agree with that. However, it seems some games should be weighted more heavily in calculating the likelihood than others.

Right now, the model predicts Cornell would win 77% of its games against Mercyhurst. It actually won 1 of 2 games, not such a good fit. Meanwhile, Wisconsin's expected win percentage according to the model is pretty close to the actual -- 9 game sample, so model fits it better.

Put another way -- if you took all the team's rankings as given, Cornell has a sample size of 2 for teams ranked 1-10, and lots of games for teams ranked in the middle. Roughly speaking, the model is setting Cornell's ranking high enough so it's likely to win all those games against teams in the middle. Given the imbalance in the schedule, it seems that these Mercyhurst games should count more than they do. Though I don't know exactly what a suitable weighting scheme would be.

 

[LakersFan]

Re: 2010-2011 Rutter DI Computer Rankings Thread

Take a look at my D-III ratings. RIT is 12-0-2 (the ties are against the number two team). They have a rating of 2.88 (full point higher than UW/Cornell) because in addition to being undefeated, they also have defeated the number four team twice. Why isn't Cornell's (a team with only 1 loss) rating as high as RIT's? Because Cornell's schedule isn't as strong as RIT's schedule (comparatively). Cornell is being penalized for playing what has turned out to be a weak schedule, which is the desired effect.

Wisconsin is 2-1-1 against the number three team (UM) and 2-1-1 against the number 6 team (UMD). Yes, Wisconsin is the best of those three WCHA teams, but not by that wide of a margin. So their rating is not going to be above 2 either.

One thing you can then do is compare common opponents, which in this case is Mercyhurst, taking into account that Cornell has played them twice while UW played them only once. The discussion seems to be centered around how much weight those games have in the ratings/rankings. Here is the answer:

Current Ratings
Cornell 1.7605
Wisconsin 1.7480
Minnesota 1.1581 (for reference)

Ratings with Wisconsin's win over Merychurst removed
Cornell 1.7736
Wisconsin 1.6872
Minnesota 1.1402

Ratings with Cornell's loss over Mercyhurst removed
Cornell 2.1474
Wisconsin 1.7310
Minnesota 1.1448

Ratings with Cornell's win over Mercyhurst removed
Wisconsin 1.7553
Cornell 1.6639
Minnesota 1.1636

Based on these results, it appears that the effect of the three Merychurst games is fairly important, especially Cornell's loss to Mercyhurst. I am not sure how much more impact those games should have on the rating values. The issue that concerns me about changing the weights is that we are having this discussion because you have two teams that have only lost to the top 6 six teams. As soon as one of them has a defeat to a team outside the top 6, the discussion becomes moot. Do we revert the weights back to what they are now or keep them? How many common opponents games need to be played before the weight is decreased? Anytime you have these discussions, you are going to produce weights that allow the computer rankings to match some preconceived human rating/ranking. The weights of RPI are an excellent example of that. I don't want to go down that road.

Each team is getting first place votes in the polls, a good indication that the two teams are very close in quality. An objective computer algorithm that takes into account how all the games are linked together generates a very similar answer. Robin Locke's rankings (http://it.stlawu.edu/~chodr/wchodr/current.html) have Cornell #1 by a clear margin. USCHO RPI has Wisconsin #1 (my unadjusted RPI has Wisconsin #1 as well, but Cornell is right behind them), as does USCHO KRACH. In a perfect world, I would like to see a best of five series between the two teams as that would be the only way to determine which is the team.

Personally, I am hoping Merychurst knocks one of these two teams off in the tournament so we will never have an answer to the question.

 

[comefrombehind]

Re: 2010-2011 Rutter DI Computer Rankings Thread

I would agree. You illustrate pretty clearly a high degree of importance of the Mercyhurst games. The dramatic impact of even just one goal -- by Mercyhurst against Cornell in overtime -- is striking.

 

[granddaddyscout]

Re: 2010-2011 Rutter DI Computer Rankings Thread

It should be interesting to see how last night’s tie impacts the computer ranking. I would think a tie to a below .500 team that lost to Minnesota 5-0 and 3-0 should have a significant impact. Curious to hear the feedback and explanations.

 

[dave1381]

Re: 2010-2011 Rutter DI Computer Rankings Thread

It should be interesting to see how last night’s tie impacts the computer ranking. I would think a tie to a below .500 team that lost to Minnesota 5-0 and 3-0 should have a significant impact. Curious to hear the feedback and explanations.

The site updates overnight. Rutter now has Wisconsin at 1.77, Cornell at 1.50. This implies a 61% chance of Wisconsin beating Cornell.

Additional note: I made a serious of posts on Friday right before the board crashed and lost everything. Will try to reproduce them in due time. If they are recovered, sorry if I end up repeating myself.

 

[dave1381]

Re: 2010-2011 Rutter DI Computer Rankings Thread

I would agree. You illustrate pretty clearly a high degree of importance of the Mercyhurst games. The dramatic impact of even just one goal -- by Mercyhurst against Cornell in overtime -- is striking.

(this is a reproduction of a lost post from Friday)

The dramatic impact you observe is because Cornell would be undefeated if it had beaten Mercyhurst (ignoring Friday's tie against Clarkson). Computer rankings always have trouble with undefeated teams. If you assume games all go according to the model used in the Rutter rankings, the best fit in such a situation would be to give Cornell a rank of infinity. The Rutter rankings avoid this issue by assuming a prior distribution of teams' rankings (based on the previous seasons' results if I remember right), such a prior would give low weight would give Cornell. But depending on which prior was chosen, an undefeated Cornell team's chances of beating Wisconsin could be anywhere from 50-100%. (for the prior chosen, Cornell's prob of beating Wisconsin was 67%,. going by the numbers from Lakersfan's post)

 

[dave1381]

Re: 2010-2011 Rutter DI Computer Rankings Thread

(this is my attempt at reproducing a lost post from Friday)

The issue that concerns me about changing the weights is that we are having this discussion because you have two teams that have only lost to the top 6 six teams. As soon as one of them has a defeat to a team outside the top 6, the discussion becomes moot.

This is not an accurate representation of my critique.

The issue is that Cornell has a much smaller sample of games among the top teams than Wisconsin does. The fact none of them have lost outside the 6 (as of Friday) is just what makes the problem more transparent.

I've raised this issue in the past. Last season at some point, Northeastern had lost all games to the only top 15 team it played (UNH), it's best win was over a team approximately 16th or so, yet Northeastern had a solid top 10 ranking.

Do we revert the weights back to what they are now or keep them? How many common opponents games need to be played before the weight is decreased? Anytime you have these discussions, you are going to produce weights that allow the computer rankings to match some preconceived human rating/ranking. The weights of RPI are an excellent example of that. I don't want to go down that road.

You can pick weights with desirable mathematical properties. That's no different from assuming a normal distribution for the uncertainty in game outcomes as opposed to logit errors. I'm not sure what the right weighing scheme is. One desirable property would be that if all teams did play a round-robin schedule, no weighting would be required.[/quote]

Each team is getting first place votes in the polls, a good indication that the two teams are very close in quality. An objective computer algorithm that takes into account how all the games are linked together generates a very similar answer. Robin Locke's rankings (http://it.stlawu.edu/~chodr/wchodr/current.html) have Cornell #1 by a clear margin. USCHO RPI has Wisconsin #1 (my unadjusted RPI has Wisconsin #1 as well, but Cornell is right behind them), as does USCHO KRACH. In a perfect world, I would like to see a best of five series between the two teams as that would be the only way to determine which is the team.

I didn't mean to single out your ranking for criticism. I've looked at them all. I think they're all flawed in the same way.

 

[dave1381]

Re: 2010-2011 Rutter DI Computer Rankings Thread

(this is an attempt at reproducing my last lost post from Friday)

I want to attempt to explain my problem with the available computer models in something closer to language that statisticians would follow.

Going into this weekend, I observe that Wisconsin is 5-2-2 against the top 6, Cornell is 1-1, and apart of this both teams are undefeated (Wisconsin's schedule being slightly differ). I would then conclude Wisconsin is clearly the better team than Cornell. I think most posters here would agree. The computer rankings as of Friday morning did not agree -- they said Cornell and Wisconsin were about the same. Why the discrepancy?

Based on past postings, I anticipate LakersFan would say that I'm not cognitively capable of processing every game this season like the computer, so I use a heuristic above that gives misleading results.

But I think I can put into words why the Rutter rankings considered Cornell and Wisconsin to be equals -- Wisconsin effectively lost 3 teams to top 6ish teams. These top 6ish teams each lost 3 teams to top10ish teams. Cornell is undefeated against a bunch of top 10ish teams. Given the structure assumed about how results are determined, Wisconsin must be similarly ranked than Cornell.

Now let me offer a couple arguments for why I might think Wisconsin is better than Cornell that aren't captured by the model.

(1) I think any of these models would suggest we have a better idea of where Wisconsin stands than Cornell, because Wisconsin has nine games against top teams while Cornell has two. So how do you resolve this uncertainty? One way is to use other information. I have some priors based on 10 years of women's hockey results, and results from other sports. I consider Cornell this year akin to Harvard in 2008 or Mercyhurst in 2007. Both these teams spent a lot of time at No. 1 and didn't win a single NCAA game outside of their conference. So based on these priors, most posters here are pretty confident that Wisconsin is better than Cornell. Of course this isn't particularly constructive criticism for computer models.

(2) Maybe the right model to think of is one with two parameters for each team, an upper bound and lower bound, or maybe a mean and a variance, though I gather it can be difficult to identify more than 1 parameter per team. I think a lot of posters would believe that Wisconsin is a team that has a high mean is consistent, that Cornell is a team who's upper bound isn't quite at Wisconsin's level, but still is consistent and manages to avoid losing to all the ECAC teams. Modelling-wise, Minnesota and UMD are capable of playing as well as Wisconsin but are more inconsistent (for Minnesota, this is more due to early season injuries). I think posters would argue that Minnesota and UMD play up to Wisconsin and play down to teams worse than them. Modeling this may be tough though.

 

[LakersFan]

Re: 2010-2011 Rutter DI Computer Rankings Thread

I just want to say that is a great discussion and I have a lot of thoughts and ideas. Unfortunately, I also have 100+ Intro STAT homeworks and labs sitting on my desk waiting to be graded, among other things that pay the bills. So a longer response will be coming later in the week.

Based on past postings, I anticipate LakersFan would say that I'm not cognitively capable of processing every game this season like the computer, so I use a heuristic above that gives misleading results.

You, me, and 99.999% of our fellow human beings.

(2) Maybe the right model to think of is one with two parameters for each team, an upper bound and lower bound, or maybe a mean and a variance, though I gather it can be difficult to identify more than 1 parameter per team. I think a lot of posters would believe that Wisconsin is a team that has a high mean is consistent, that Cornell is a team who's upper bound isn't
quite at Wisconsin's level, but still is consistent and manages to avoid losing to all the ECAC teams. Modelling-wise, Minnesota and UMD are capable of playing as well as Wisconsin but are more inconsistent (for Minnesota, this is more due to early season injuries). I think posters would argue that Minnesota and UMD play up to Wisconsin and play down to teams worse than them. Modeling this may be tough though.

For games played through 2/5, here are the top 3 teams in terms of the standard deviation of their mean rating estimate. In other words, it can be considered a measure of how certain the model is of the rating, the larger the value the more uncertain the model.

Team SD Rating
St. Cloud State .43 -1.29
Cornell .40 1.52
Wisconsin .37 1.80

The team with the lowest standard deviation is Vermont (.28) and the average standard deviation is .32. UM is ranks 7 (.34) and UMD is 8 (.33). This is not the same as estimating how a team's rating fluctuates from game to game (which I assume has a sd of .71).

Not sure how this relates to Dave's point above, as I think some (if not most) of the variation is due to position in the rankings. The closer you are to the top or bottom, the more variable your estimate because you are not playing teams above or below you. The correlation between absolute value of the rating and the standard deviation is .80.

More later on possible weighting ideas.

 

[OnMAA]

Re: 2010-2011 Rutter DI Computer Rankings Thread

And here I thought I was done with Stats after graduating some 30 years ago

Great discussions, but there is one variable the models can/do not account for unless past HTH results are already part of the data set. Matchups between teams and how certain teams matchup against each other. There are often a lot of intangibles. I give you an example. Last week UNH beat BU. Granted BU still outshot UNH, but I'm convinced one of the intangibles there was an Ex UNH player on BU.

How would Cornell matchup vs Wisconsin. Now there is a debate that is more interesting to the average Hockey Fan compared to this dry number crunching discussion.

 

[Watson Rink]

Re: 2010-2011 Rutter DI Computer Rankings Thread

It seems you mean "outside their region" as Harvard defeated five of the six Hockey East teams it faced that season.

However, your fundamental point is perfectly true, in that schedules are very parochial: not only are there very few East-West match-ups (with Harvard's admittedly limited experience with Minny and UMD since 2006-07 having resulted in a perfect 0-4-0 road record and an almost perfect 3-0-1 home record), but even within conferences the non-league games are very parochial (Harvard plays nearby HE schools whereas Cornell plays nearby CHA schools so that H and C have no common opponent outside the ECAC). So the sample size of overlapping opponents must be disappointingly small.

 

[dave1381]

Re: 2010-2011 Rutter DI Computer Rankings Thread

It seems you mean "outside their region" as Harvard defeated five of the six Hockey East teams it faced that season.

However, your fundamental point is perfectly true, in that schedules are very parochial: not only are there very few East-West match-ups (with Harvard's admittedly limited experience with Minny and UMD since 2006-07 having resulted in a perfect 0-4-0 road record and an almost perfect 3-0-1 home record), but even within conferences the non-league games are very parochial (Harvard plays nearby HE schools whereas Cornell plays nearby CHA schools so that H and C have no common opponent outside the ECAC). So the sample size of overlapping opponents must be disappointingly small.

I needed an additional word for clarity -- I meant neither team won an NCAA TOURNAMENT game out of conference. The 2007 Mercyhurst team lost to UMD in the quarters. 2008 Harvard beat Dartmouth in the quarters and lost to Wisconsin at the Frozen Four. I should also enter 2006 UNH into the discussion (which did win an NCAA quarterfinal against an average Harvard team, but lost to Minnesota in the Frozen Four).

 

[dave1381]

Re: 2010-2011 Rutter DI Computer Rankings Thread

And here I thought I was done with Stats after graduating some 30 years ago

Great discussions, but there is one variable the models can/do not account for unless past HTH results are already part of the data set. Matchups between teams and how certain teams matchup against each other. There are often a lot of intangibles. I give you an example. Last week UNH beat BU. Granted BU still outshot UNH, but I'm convinced one of the intangibles there was an Ex UNH player on BU.

How would Cornell matchup vs Wisconsin. Now there is a debate that is more interesting to the average Hockey Fan compared to this dry number crunching discussion.

There are always variables these models won't control for. But often times these effects are 2nd order, and when forecasting results, it's important to get the relative team strengths right first.

I think Green Bay's Super Bowl win is a case in point. Probably lots of fans didn't see it coming because they were an 8-6 team. Patriots fans probably had a better idea since they saw how thoroughly they demolished Pittsburgh on the road and then struggled to beat Green Bay sans Rodgers at home. But the analysis at football outsiders made it clear to me even back in Week 15 that Green Bay was going to be the toughest game of the season for the Patriots, and that this performance was no fluke.