2010-2011 Rutter DI Computer Rankings Thread

[LakersFan]

We are back for another year of algorithm based fun. For the complete rankings and information on how they are calculated, please visit the website: http://math.bd.psu.edu/faculty/rutter/WomensRankings.html

For games played through 10/10/10

   	Team  		Rating  
1 	Mercyhurst 	1.7743 	
2 	Minnesota 	1.4368 
3 	Wisconsin 	1.2998 
4 	Quinnipiac 	1.1193 	
5 	Ohio State 	0.9929 
6 	Boston College 	0.9259 	
7 	Harvard 	0.6314 	
8 	UMD	 	0.5116 	
9 	Cornell 	0.5085 	
10 	Providence 	0.4357

Yes, Quinnipiac is ranked above OSU as the started the season ranked a little higher.

I have added another page to the website this year describing the uncertainty in the rankings: http://math.bd.psu.edu/faculty/rutter/CurrentD1Unc.html

Since I use a statistical model to estimate a teams quality rating, there is some degree of uncertainty in the estimates. Very rarely are these estimates of uncertainty reported, but they do contain some interesting information. Here is the Top 10 from that page:

   	Team  		25th 	75th   	Percent Top 8
1 	Mercyhurst 	1 	6 	86.8
2 	Minnesota 	2 	7 	82.5
3 	UMD	 	2 	10 	70.4
4 	Wisconsin 	3 	10 	69.0
5 	Quinnipiac 	3 	10 	67.0
6 	Boston College 	4 	12 	57.8
7 	Ohio State 	4 	12 	57.4
8 	Harvard 	5 	18 	43.7
9 	Cornell 	6 	19 	39.4
10 	Princeton 	10 	25 	21.1

The table is sorted by Percent Top 8. This is determined by looking at simulations of all 33 teams' quality ratings and ranking them over 10,000 times. Percent Top 8 is the percentage of times these simulated rankings have that team in the top 8, the same number of teams that make the NCAA tournament. The 25th and 75th percentile represent the innerquartile range of all the simulated rankings, that is the middle 50% of the rankings. So, for at least half the time, Merychurst was ranked between first and sixth.

Can the Percent Top 8 column be used to predict the probability that a team will be in the top 8? Yes, IF the team played an infinite number of games against all of the other teams and the true quality rating did not change over time. That is a big "if." In reality, the teams play a finite number of games against a subset of all the teams, and the Percent Top 8 calculation does not take into account future opponents.

You will note that Ivy league teams have the widest spread of rankings, as they have not played any games yet and their ratings are based on last season's results. These results will improve over time as more games are played. For example, in 2008-2009, Wisconsin finished ranked number 1 with an interquartile range of [1,2] and a 100% of being in the top 8.

 

[scrambledlegs]

Re: 2010-2011 Rutter DI Computer Rankings Thread

For games played through 10/10/10

   	Team  		Rating  
1 	Mercyhurst 	1.7743 	
2 	Minnesota 	1.4368 
3 	Wisconsin 	1.2998 
4 	Quinnipiac 	1.1193 	
5 	Ohio State 	0.9929 
6 	Boston College 	0.9259 	
7 	Harvard 	0.6314 	
8 	UMD	 	0.5116 	
9 	Cornell 	0.5085 	
[B]10 	Providence 	0.4357 [/B]

I can't complain because Providence is ranked , however this must make OnMaa very happy as he has been lobbying for the Q!

Yes, Quinnipiac is ranked above OSU as the started the season ranked a little higher.

 

[OnMAA]

Re: 2010-2011 Rutter DI Computer Rankings Thread

I can't complain because Providence is ranked , however this must make OnMaa very happy as he has been lobbying for the Q!

You got my allegiances way off, but I did put Quinnipiac in the top ten (deservedly so) based on a 4-0 record. However, they should IMHO not be ranked ahead of UMD or BU, as Quinnipiac's strength of schedule does not match that of BU and UMD. The Rutter ranking also shows Clarkson ranked 30th. Clearly a reflection of their poor start and not of their real strength as perceived by many including yours truly.

The statistical rutter method done by Lakersfan will become more accurate once more games are recorded in the books. Watched these rankings in years past and found them to be A: good and B: impartial. Keep up the good work Lakerfan.

 

[dave1381]

Re: 2010-2011 Rutter DI Computer Rankings Thread

Hi Lakersfan,

Thanks for the nice addition to the rankings.

One question -- your method isn't actually taking into account the uncertainty in the ranking parameter estimates for each team, right? It sounds like you're taking the parameter estimates as given, and then calculating the expectation of each team finishing in the top 8 given the parameters. Just wanted to be clear.

 

[LakersFan]

Re: 2010-2011 Rutter DI Computer Rankings Thread

One question -- your method isn't actually taking into account the uncertainty in the ranking parameter estimates for each team, right? It sounds like you're taking the parameter estimates as given, and then calculating the expectation of each team finishing in the top 8 given the parameters. Just wanted to be clear.

No, it is a true measure of the uncertainty. I apologise in advance for all the statistical terminology I am about to post.

Since I use a Bayesian approach to estimate a team's rating, it is very easy to take a random walk through the joint distribution of all the teams' ratings. I do this as part of the estimation process, and usually report the median of each team's rating as a point estimate. I could easily report a 95% credibility interval (Bayesian form of confidence interval) for each team's rating. For example, the 95% CI for Wisconsin's rating (after their great win over Ohio State, wait, wrong sport) on Sunday morning is (-0.06, 2.87). By itself, that interval is scary not only because it falls below 0, which would indicate an average team, but it is also difficult to compare to the other teams' ratings, since they would be effected/correlated with Wisconsin's. Looking at 33 simultaneous CI hurts my head.

Instead, for all 30,000 visits to the joint distribution, I rank the ratings, therefore looking at how Wisconsin compares to all the other teams for that 1 simulation from the joint distribution. That seems like a lot of work, but computers are fast and enjoy endless, repetitive tasks. Therefore, I have 30,000 samples from the distribution that describes the rank of Wisconsin with respect to the other 32 teams. I could plot a histogram of those 30,000 values, but I decided to summarise the distribution by looking at the middle 50% and the probability (calculated via simulation) of being in the top 8.

If you look at today's report, you see that none of the teams are ranked number one more than 25% of the time. To me, this indicates that the top 7 teams are indistinguishable from each other at this point in the season. Given the limited number of games that have been played and the fact that many of the top teams have lost a game or two, this is not a surprise.

There will be a quiz on Wednesday

 

[ARM]

Re: 2010-2011 Rutter DI Computer Rankings Thread

There will be a quiz on Wednesday

I plan to just audit this course. Nice work.

 

[LakersFan]

Re: 2010-2011 Rutter DI Computer Rankings Thread

For games played through October 17, 2010

   	Team            Rating  	
1 	Wisconsin 	1.3296 	
2 	UMD	 	1.2343 	
3 	Boston College 	1.1127 	
4 	Ohio State 	1.1007 	
5 	North Dakota 	1.0811 	
6 	Mercyhurst 	1.0510 
7 	Boston Univ.	1.0278 	
8 	Providence 	0.7194 	
9 	Minnesota 	0.6559 	
10 	Harvard 	0.6314

Where is Cornell? Well, they have a rating of .51 based on last year's regular season. Their rating hasn't changed, the other teams have just jumped over them. Once they start playing games, they will settle in where they belong. Still early in the season and the ratings are very sensitive.

 

[LakersFan]

Re: 2010-2011 Rutter DI Computer Rankings Thread

For games played through October 24, 2010

  	Team 		Rating 	
1 	Wisconsin 	1.8320 	
2 	UMD	 	1.3278 	
3 	Boston Univ. 	1.1463 	
4 	Mercyhurst 	1.1330 	
5 	Boston College 	1.0245 	
6 	Cornell 	0.9519 
7 	North Dakota 	0.9248 
8 	Quinnipiac 	0.7220 	
9 	Minnesota 	0.7055 
10 	Harvard 	0.6283

 

[dave1381]

Re: 2010-2011 Rutter DI Computer Rankings Thread

Lakerfan -- thanks for the explanation. I didn't get a chance to read the response until now. Now I understand.

One more quick question: in the past when I've used your rankings to calculate playoff probabilities, I've calculated the inverse normal cdf of the difference in team rankings. Is that still correct -- i.e. the prior/posterior distributions are all multivariate normal? Or is it just a decent approximation? I guess I'm also ignoring the issue that you have ties in the regular season and not the postseason.

 

[LakersFan]

Re: 2010-2011 Rutter DI Computer Rankings Thread

Lakerfan -- thanks for the explanation. I didn't get a chance to read the response until now. Now I understand.

One more quick question: in the past when I've used your rankings to calculate playoff probabilities, I've calculated the inverse normal cdf of the difference in team rankings. Is that still correct -- i.e. the prior/posterior distributions are all multivariate normal? Or is it just a decent approximation? I guess I'm also ignoring the issue that you have ties in the regular season and not the postseason.

Yes, that is the easy way to calculate the probability of one team defeating another. The model for predicting a single game's outcome is univariate, while the distribution that describes the teams' ratings is multivariate. Now, to be 100% correct, you would calculate the probability of team A beating team B for every 30,000 combinations of the two ratings, and then find the average the results. And you would have an answer very similar to the one calculated from the point estimates.

 

[LakersFan]

Re: 2010-2011 Rutter DI Computer Rankings Thread

For games played through 10/31/10

  	Team 	Rating 	
1 	Wisconsin 	2.0486 
2 	UMD 	        1.5473 	
3 	Cornell 	1.3705 
4 	Mercyhurst 	1.1801 	
5 	Ohio State 	0.9873 	
6 	Boston Univ. 	0.9834 	
7 	North Dakota 	0.8624 	
8 	Boston College 	0.5805 	
9 	Minnesota 	0.5137 	
10 	Dartmouth 	0.4874

I will update the ratings after Tuesday's games.

 

[dave1381]

Re: 2010-2011 Rutter DI Computer Rankings Thread

Ok, at this point it looks to me like Cornell, Mercyhurst, BU/BC are all a class above in their conferences right now, and most of the regular season excitement in terms of making the top 8 is going be jostling among WCHA teams. The WCHA will get 4 teams, barring an upset of Cornell in the ECAC tournament or BU/BC in Hockey East. Of course other teams are capable of improving in ECAC/Hockey East, but no one else in those leagues will improve enough fast enough to earn an NCAA at-large bid.

 

[LakersFan]

Re: 2010-2011 Rutter DI Computer Rankings Thread

Ok, at this point it looks to me like Cornell, Mercyhurst, BU/BC are all a class above in their conferences right now, and most of the regular season excitement in terms of making the top 8 is going be jostling among WCHA teams. The WCHA will get 4 teams, barring an upset of Cornell in the ECAC tournament or BU/BC in Hockey East. Of course other teams are capable of improving in ECAC/Hockey East, but no one else in those leagues will improve enough fast enough to earn an NCAA at-large bid.

With St. Lawrence and Harvard appearing to be below average teams this year (at least so far), the quality of the non-conference games have taken a hit. Below is the list of non-conference games between top 10 (according to my rankings on 11/6) that have or will occur. I may have missed a couple, and it will of course change when the top 10 changes.

Wisconsin-Mercyhurst
Bemidji State-Mercyhurst x2
Cornell-Mercyhurst x2
BU-North Dakota x2

Four of those games (BS-MC, BU-ND) would not have been thought of as top 10 non-conference games before the season, I think. This is going to make it difficult for any ranking system to rank the WCHA vs. the east, as there are so few games against the top teams in each league. Also makes Merychurst's schedule look a little stronger.

Please, for the sake of computer models everywhere, let OSU join the CHA when Penn State goes DI so each WCHA team can gain four (hopefully meaningful) non-conference games.

 

[dave1381]

Re: 2010-2011 Rutter DI Computer Rankings Thread

Regarding disappointing teams for the sake of nonconference matchups, topping the list has to be UConn (UMD's only nonconference opponent) and Clarkson (a Minnesota opponent) have to be tops on the list. They were both top 10 teams last year and are struggling mightily this year.

SLU was average last year so they don't really belong in this conversation. I agree Harvard has disappointed so far, but in terms of the rankings they've lost to the No. 4 team and beaten the worst team --- really their ranking (aside from the prior) is mainly determined by one game, the tie against Yale. I expect they'll be more of a 9-15 team rather than a below average team.

Mercyhurst looks stronger, not just because of the direct head-to-head matchups with Bemidji, but also Wayne State's wins over Bemidji and North Dakota are impressive.

I'd never have imagined the WCHA would have 6 of the top 9 teams, that's for sure.

 

[LakersFan]

Re: 2010-2011 Rutter DI Computer Rankings Thread

For games played through 11/7/10

  	Team 	Rating 	
1 	Wisconsin 	1.6556 	
2 	UMD		1.4347 	
3 	Boston Univ.	1.3097 
4 	Cornell 	1.1924 	
5 	North Dakota 	1.0688 	
6 	Mercyhurst 	0.8699 	
7 	Minnesota 	0.8207 	
8 	Ohio State 	0.7148 	
9 	Bemidji State 	0.5747 	
10 	Boston College 	0.5441

 

[LakersFan]

Re: 2010-2011 Rutter DI Computer Rankings Thread

For games played through 11/14/10

  	Team 	        Rating 	
1 	Wisconsin       1.6200 	
2 	UMD 	        1.4444 	
3 	Cornell         1.2325 	
4 	Mercyhurst	0.9746 	
5 	Boston Univ. 	0.8056 	
6 	North Dakota 	0.7980 	
7 	Dartmouth       0.7854 	
8 	Minnesota       0.7262 	
9 	Bemidji State 	0.7247 	
10 	Providence      0.4188

 

[ARM]

Re: 2010-2011 Rutter DI Computer Rankings Thread

I think we're about 8 weeks into the season in terms of official games, half that for the Ivy League, and at this time the Rutter Rankings favor WCHA teams in several cases where the RPI prefers HEA participants. Looking at the top 17 in both, which represents those with a positive Rutter Rating and an RPI > .500, we have:

  	Team 	 	Rating 	RPI.Rank   RPI
1 	Wisconsin 	1.6641 	1 	  0.6543
2 	Cornell 	1.3689 	2 	  0.6376
3 	Minn. Duluth 	1.2658 	3 	  0.6195
4 	Mercyhurst 	1.1157 	4 	  0.5888
5 	Minnesota 	0.8660 	8 	  0.5566
6 	North Dakota 	0.7886 	6 	  0.5775
7 	Bemidji State 	0.7019 	11 	  0.5432
8 	Boston Univ. 	0.6881 	5 	  0.5801
9 	Ohio State 	0.4373 	14 	  0.5313
10 	Dartmouth 	0.4306 	9 	  0.5498
11 	Boston College 	0.4180 	7 	  0.5671
12 	Harvard 	0.2947 	10 	  0.5448
13 	Minn. State 	0.2528 	16 	  0.5090
14 	Providence 	0.2504 	13 	  0.5322
15 	Northeastern 	0.1359 	12 	  0.5326
16 	Quinnipiac 	0.1221 	15 	  0.5120
17 	New Hampshire 	0.1212 	17 	  0.5059

In the cases where the ordinal they assign differs by three or more, we have:

Rutter 	Team 	 	 	RPI 	League 	RPI-Rutter
5 	Minnesota 	 	8 	WCHA 	3
7 	Bemidji State 	 	11 	WCHA 	4
8 	Boston U 	 	5 	HEA 	-3
9 	Ohio State 	 	14 	WCHA 	5
11 	Boston College 	 	7 	HEA 	-4
13 	Minnesota State 	16 	WCHA 	3
15 	Northeastern 	 	12 	HEA 	-3

Usually, we see these two tighten up as the season progresses. As some WCHA teams are running out of non-conference games, I'm not sure if they will line up or not. The key differences at this point are Minnesota, BSU, BU, and BC, because those are the rankings where one has a team in an 8-team field, the other has them on the outside. With 20 games to go, it is likely that there will be at most a couple of teams that would receive a different in/out fate with a different NCAA ranking system.

 

[LakersFan]

Re: 2010-2011 Rutter DI Computer Rankings Thread

For games played trough 11/22/2010

  	Team 	            Rating 	
1 	Wisconsin 	    1.6641 	
2 	Cornell 	    1.3689 	
3 	Minnesota Duluth    1.2658 	
4 	Mercyhurst 	    1.1157 	
5 	Minnesota 	    0.8660 	
6 	North Dakota 	    0.7886 	
7 	Bemidji State 	    0.7019 	
8 	Boston University   0.6881 
9 	Ohio State 	    0.4373 	
10 	Dartmouth 	    0.4306

 

[LakersFan]

Re: 2010-2011 Rutter DI Computer Rankings Thread

In the cases where the ordinal they assign differs by three or more, we have:

Rutter 	Team 	 	 	RPI 	League 	RPI-Rutter
5 	Minnesota 	 	8 	WCHA 	3
7 	Bemidji State 	 	11 	WCHA 	4
8 	Boston U 	 	5 	HEA 	-3
9 	Ohio State 	 	14 	WCHA 	5
11 	Boston College 	 	7 	HEA 	-4
13 	Minnesota State 	16 	WCHA 	3
15 	Northeastern 	 	12 	HEA 	-3

Usually, we see these two tighten up as the season progresses. As some WCHA teams are running out of non-conference games, I'm not sure if they will line up or not. The key differences at this point are Minnesota, BSU, BU, and BC, because those are the rankings where one has a team in an 8-team field, the other has them on the outside. With 20 games to go, it is likely that there will be at most a couple of teams that would receive a different in/out fate with a different NCAA ranking system.

I think they key is that my rankings look at the actual results of each game individually, while RPI evaluates the records as a whole and not individual game results. Also, my rankings penalize for a tie against a bad team more than RPI does. For example, BU tying Vermont twice hurts BU in my rankings more than RPI. Same with BC's ties against St. Lawerence and Vermont.

I would also like to point out than when you take into account uncertainty, 50% of the time BU is ranked between 6 and 11, while BC is ranked between 8 and 14 50% of the time. That may mean that 50% of the time the difference between RPI and my rankings is not as bad as ARM says it is. Always striving to be less wrong half the time, that's my motto

 

[TonyTheTiger20]

Re: 2010-2011 Rutter DI Computer Rankings Thread

Always striving to be less wrong half the time, that's my motto

As a math major who loves statistics, this is brilliant