What's new
By:
USCHO Fan Forum

This is a sample guest message. Register a free account today to become a member! Once signed in, you'll be able to participate on this site by adding your own topics and posts, as well as connect with other members through your own private inbox!

  • The USCHO Fan Forum has migrated to a new plaform, xenForo. Most of the function of the forum should work in familiar ways. Please note that you can switch between light and dark modes by clicking on the gear icon in the upper right of the main menu bar. We are hoping that this new platform will prove to be faster and more reliable. Please feel free to explore its features.

Octonion Power Rankings

O

octonion

New member
Poisson model, pooled NCAA divisions, home/away/neutral factors.

div = NCAA division
str = team strength
ofs = offensive strength
dfs = defensive strength
sos = strength of schedule

Code: 
  rk |             school              | div |  str  |  ofs  |  dfs  |  sos  
-----+---------------------------------+-----+-------+-------+-------+-------
   1 | minnesota                       |   1 | 4.230 | 1.916 | 0.453 | 1.489
   2 | quinnipiac                      |   1 | 3.858 | 1.536 | 0.398 | 1.460
   3 | miami                           |   1 | 3.469 | 1.402 | 0.404 | 1.458
   4 | minnesota-state                 |   1 | 3.443 | 1.753 | 0.509 | 1.502
   5 | north-dakota                    |   1 | 3.321 | 1.727 | 0.520 | 1.526
   6 | denver                          |   1 | 3.259 | 1.837 | 0.564 | 1.548
   7 | st-cloud-state                  |   1 | 3.216 | 1.720 | 0.535 | 1.499
   8 | new-hampshire                   |   1 | 3.173 | 1.687 | 0.532 | 1.470
   9 | umass-lowell                    |   1 | 3.113 | 1.565 | 0.503 | 1.428
  10 | notre-dame                      |   1 | 3.084 | 1.605 | 0.521 | 1.482
  11 | union                           |   1 | 3.012 | 1.570 | 0.521 | 1.448
  12 | rensselaer                      |   1 | 2.931 | 1.551 | 0.529 | 1.496
  13 | wisconsin                       |   1 | 2.868 | 1.369 | 0.477 | 1.496
  14 | providence                      |   1 | 2.863 | 1.544 | 0.539 | 1.482
  15 | boston-college                  |   1 | 2.808 | 1.761 | 0.627 | 1.437
  16 | western-michigan                |   1 | 2.786 | 1.295 | 0.465 | 1.459
  17 | nebraska-omaha                  |   1 | 2.679 | 1.760 | 0.657 | 1.495
  18 | air-force                       |   1 | 2.673 | 1.534 | 0.574 | 1.349
  19 | ferris-state                    |   1 | 2.625 | 1.463 | 0.558 | 1.475
  20 | dartmouth                       |   1 | 2.618 | 1.526 | 0.583 | 1.462
  21 | cornell                         |   1 | 2.541 | 1.377 | 0.542 | 1.502
  22 | yale                            |   1 | 2.527 | 1.576 | 0.624 | 1.486
  23 | niagara                         |   1 | 2.508 | 1.494 | 0.596 | 1.354
  24 | colorado-college                |   1 | 2.507 | 1.798 | 0.717 | 1.567
  25 | michigan-tech                   |   1 | 2.493 | 1.611 | 0.646 | 1.516
 
Re: Octonion Power Rankings

octonion said:
Poisson model, pooled NCAA divisions, home/away/neutral factors.

div = NCAA division
str = team strength
ofs = offensive strength
dfs = defensive strength
sos = strength of schedule

Code: 
  rk |             school              | div |  str  |  ofs  |  dfs  |  sos  
-----+---------------------------------+-----+-------+-------+-------+-------
   1 | minnesota                       |   1 | 4.230 | 1.916 | 0.453 | 1.489
   2 | quinnipiac                      |   1 | 3.858 | 1.536 | 0.398 | 1.460
   3 | miami                           |   1 | 3.469 | 1.402 | 0.404 | 1.458
   4 | minnesota-state                 |   1 | 3.443 | 1.753 | 0.509 | 1.502
   5 | north-dakota                    |   1 | 3.321 | 1.727 | 0.520 | 1.526
   6 | denver                          |   1 | 3.259 | 1.837 | 0.564 | 1.548
   7 | st-cloud-state                  |   1 | 3.216 | 1.720 | 0.535 | 1.499
   8 | new-hampshire                   |   1 | 3.173 | 1.687 | 0.532 | 1.470
   9 | umass-lowell                    |   1 | 3.113 | 1.565 | 0.503 | 1.428
  10 | notre-dame                      |   1 | 3.084 | 1.605 | 0.521 | 1.482
  11 | union                           |   1 | 3.012 | 1.570 | 0.521 | 1.448
  12 | rensselaer                      |   1 | 2.931 | 1.551 | 0.529 | 1.496
  13 | wisconsin                       |   1 | 2.868 | 1.369 | 0.477 | 1.496
  14 | providence                      |   1 | 2.863 | 1.544 | 0.539 | 1.482
  15 | boston-college                  |   1 | 2.808 | 1.761 | 0.627 | 1.437
  16 | western-michigan                |   1 | 2.786 | 1.295 | 0.465 | 1.459
  17 | nebraska-omaha                  |   1 | 2.679 | 1.760 | 0.657 | 1.495
  18 | air-force                       |   1 | 2.673 | 1.534 | 0.574 | 1.349
  19 | ferris-state                    |   1 | 2.625 | 1.463 | 0.558 | 1.475
  20 | dartmouth                       |   1 | 2.618 | 1.526 | 0.583 | 1.462
  21 | cornell                         |   1 | 2.541 | 1.377 | 0.542 | 1.502
  22 | yale                            |   1 | 2.527 | 1.576 | 0.624 | 1.486
  23 | niagara                         |   1 | 2.508 | 1.494 | 0.596 | 1.354
  24 | colorado-college                |   1 | 2.507 | 1.798 | 0.717 | 1.567
  25 | michigan-tech                   |   1 | 2.493 | 1.611 | 0.646 | 1.516
Click to expand...

Any description as to how you are coming by these numbers?
 
Re: Octonion Power Rankings

For a fairly simple exposition of how to do Poisson modeling, see https://dl.dropbox.com/u/5755704/ranking.doc (I wrote it a couple of years ago, so it refers to the 2011 season.) The only difference between this and what Octonion has done is normalization to create the rankings, I think. But he or she will, I'm sure, correct me if I'm wrong.
 
Re: Octonion Power Rankings

Sure, it's a mixed-effect Poisson regression model. I'm assuming that the goals scored by a team is modeled as a Poisson distribution depending on the team's offensive strength (random effect), opponent's defensive strength (random effect), home/away/neutral factor (fixed effect) and year (fixed effect). Offense and defense are nested within NCAA divisions, and these are pooled over the last 14 years.

As an example, I get than teams score about 7% more goals and allow about 7% fewer goals at home. This implies by the Pythagorean expectation (with exponent about 2.2) that the home team's winning percentage should be about 57%, and this is exactly what we see.

-Chris
 
Last edited:
Re: Octonion Power Rankings

Interesting, Octonion. What do you get out of the 14 year pooling (given year fixed effects) other than more accurate (presuming invariance) estimates of the home team effect? If I understand you correctly, are you assuming that, in essence, a team's offensive and defensive prowess rises and falls by a fixed (estimated) amount, or are the defense and offense allowed to move independently from year to year?
 
Re: Octonion Power Rankings

So, was the model applied to previous years? If so, how did it compare to the NCAA tournament results (especially in regards to insulated teams that do not play each other such as MN and QU this year)?

I assume the reason to rank the teams is to give some insight into how the teams will perform. If the purpose is for something else, what is it?
 
Re: Octonion Power Rankings

gopheritall said:
So, was the model applied to previous years? If so, how did it compare to the NCAA tournament results (especially in regards to insulated teams that do not play each other such as MN and QU this year)?

I assume the reason to rank the teams is to give some insight into how the teams will perform. If the purpose is for something else, what is it?
Click to expand...

To start a discussion on the internet. Duh. :D
 
Re: Octonion Power Rankings

It's a first model - with sports you do have to be really careful regarding any rule changes. In particular, can the rule changes be absorbed into the fixed yearly effect?

The strength of a team's offense and defense are separately estimated for each season, but the overall pool strength of D1, D2 and D3 is assumed to be consistent. The home field advantage is assumed to be consistent, too, but that could easily be allowed to vary by team. That makes a difference for some (but not most) teams.
 
Re: Octonion Power Rankings

It predicted 11/15 (73.3%) of the NCAA D1 tournament outcomes correctly last year, fitting on only non-tournament games. Is that good or bad?
 
Re: Octonion Power Rankings

octonion said:
It predicted 11/15 (73.3%) of the NCAA D1 tournament outcomes correctly last year, fitting on only non-tournament games. Is that good or bad?
Click to expand...

In last year's tournament, I believe that all 4 top seeds qualified for the FF, and in the FF, everything went according to PWR seeding. That means that, at worst, the PWR predicted the same 11/15 (and I am fairly sure that UND was a #2, so that makes 12 right atleast), and it's not designed as a predictive model. Draw your own conclusion.
 
Re: Octonion Power Rankings

octonion said:
It predicted 11/15 (73.3%) of the NCAA D1 tournament outcomes correctly last year, fitting on only non-tournament games. Is that good or bad?
Click to expand...

Its a small sample size. I've thought about yearly correlation but then you get into the trickery of time-series behavior. I'm a little leery of a mixed-effect model for this... and with the constraint my thoughts would be that it would be hard to calculate... while one can always recode the predictors with respect to constraint, that a "2" may appear in the data is a little leery.

Also, if you are using 58 separate random effects, I would be shocked if the computation wasn't difficult in some measure.... oh, wait... random effect within year and then moving value for each. I suppose but I'm not sure what's gained or if model strength actually borrows over. "Nested within NCAA divisions" so every team by year comes from the same pool for offense and then again for defense?

As the complexity increases the ease of modeling by standard software also increases. GLMM models can be nasty if you throw enough into the works.
 
Numbers said:
I am sorry. I had thought that Minny and Ferris were #1s. Interesting that they each defeated #1 seeds from their own conference...
Click to expand...

No. Ferris beat #3 seed Denver, then #4 Cornell to get to Tampa, then #1 Union to get to the final.
 
Re: Octonion Power Rankings

Sure, one tournament is a small sample. What would be your metric for measuring relative predictive ability? Octonion is likely better than KRACH even now, as KRACH doesn't account for winning margins. That's why KRACH has Quinnipiac ranked over Minnesota.
 
Re: Octonion Power Rankings

Relative predictability for any set of games is measured as sumof(f(prob(win)-win)), where the sum is taken over all out-of-sample games and f() is a penalty function which is always positive and, ideally punishes longshots, like f(x)=x^2. Thus, a "pure" PWR based metric that predicts the higher seed always wins (or a pure metric based on any ranking system that predicts the higher seed will always win, generates either f(1) or f(0) for every game. Your system, by contrast, will generate f(1-pr(win)) when you get it right and f(pr(win)) when you get it wrong. Thus, predicting a game as 50-50 will generate f(.5) no matter what the outcome. Nicely, the binomial nature of the probabilities also will tell you whether or not your predictive score is within the tolerance of the probabilities: for any game with true probability p, the standard error is sqrt(p(1-p)). Since games are independent, you can then form a compounded expected predicted accuracy over N games which rises as the square root of N. This will at least tell you something about whether or not you've overfitted the model.

One more thing: it's true that your model uses score and KRACH doesn't, but KRACH uses who won the game and your method doesn't. Unless the Poisson model is exactly right (and it isn't), it's up in the air which is more accurate for predicting the outcome of actual games. I tested this by resimulating the season using only the Poisson model and looking at team records: try it, you will find some surprises.
 
Last edited:
octonion said:
Sure, one tournament is a small sample. What would be your metric for measuring relative predictive ability? Octonion is likely better than KRACH even now, as KRACH doesn't account for winning margins. That's why KRACH has Quinnipiac ranked over Minnesota.
Click to expand...

Thank you for not responding to substantive issues, as a journal referee I would not be pleased. Remember that in the future should you proceed.
 
You must log in or register to reply here.
Back
Top