NCAA Selection Process

[FlagDUDE08]

Re: NCAA Selection Process

Update as of games completed 9 November 2013 at 23:00 Aleutian Standard Time:

Quality Wins Bonus:
5.00 Minnesota
4.75 St. Cloud State
4.50 Quinnipiac
4.25 Providence
4.00 Boston College
3.75 Michigan
3.50 Northeastern
3.25 Wisconsin
3.00 Miami
2.75 Ferris State
2.50 RPI
2.25 Notre Dame
2.00 LSSU
1.75 Bowling Green
1.50 Yale
1.25 Brown
1.00 St. Lawrence
0.75 North Dakota
0.50 UMASS Lowell
0.25 Clarkson

And the tournament field:

Boston College
Minnesota
St. Cloud State
Providence

Miami
Quinnipiac
Michigan
St. Lawrence

Northeastern
Wisconsin
Notre Dame
Boston University

Ferris State
Yale
UMASS Lowell
AHA Champ (Air Force)

Keep in mind that BU has thus far racked up 7.5 RatingsPI points in Quality Wins ALONE.

 

[FlagDUDE08]

Re: NCAA Selection Process

Update after games played on 10 November 2013:

Quality Wins Bonus:
5.00 Minnesota
4.75 St. Cloud State
4.50 Providence
4.25 Quinnipiac
4.00 Boston College
3.75 Michigan
3.50 Northeastern
3.25 Miami
3.00 Ferris State
2.75 Wisconsin
2.50 Notre Dame
2.25 LSSU
2.00 Bowling Green
1.75 RPI
1.50 North Dakota
1.25 Yale
1.00 Brown
0.75 St. Lawrence
0.50 UMASS Lowell
0.25 Clarkson

And the tournament field:

Boston College
Minnesota
St. Cloud State
Providence

Miami
Quinnipiac
Michigan
St. Lawrence

Notre Dame
Wisconsin
Northeastern
Boston University

Ferris State
North Dakota
UMASS Lowell
AHA Champ (39 - Air Force)

 

[FlagDUDE08]

Re: NCAA Selection Process

Alright, I took a look at the selection criteria again, and I think I may have found an issue with my code:

Quality Wins Bonus
The Quality Wins Bonus is implemented to reward performance against the strongest competition, as measured by the top 20 teams in the RPI (calculated per above).

The maximum bonus of 5.00 points in any single game is earned with a win against the #1 team in the RPI. A bonus of 4.75 points is earned for a win against the #2 team, 4.50 is earned for a win against the #3 team, and so on until the smallest bonus of 0.25 points, which is earned for a win against the #20 team.

As a tie is one-half of a win, the bonus for a tie against any top 20 team is one-half of the corresponding win bonus against the same team.

The amount of bonus for any game is multiplied by the appropriate game-site factor consistent with the calculation of the RPI. If the win (or tie) takes place on the road the amount of bonus is multiplied by a factor of 1.2. If the win (or tie) takes place at home the amount of bonus is multiplied by a factor of 0.8. The bonus is unchanged for a win (or tie) at a neutral site.

Once the total bonus points have been calculated they are divided by the total weighting of all games played per the RPI calculation (with road wins/home losses weighted with a factor of 1.2 and home wins/road losses weighted with a factor of 0.8). The resulting Quality Wins Bonus (QWB) is added to the original RPI to obtain the final RPI for each team. This is the value to be used in comparing any two teams, along with the other selection criteria.

Read more: http://www.uscho.com/2013/09/20/roa...-tournament-selection-criteria/#ixzz2kNB6e7fE

Specifically, it has to do with the last two paragraphs of information, and perhaps I have been weighting the quality wins bonus a little too firm. Instead of a full 5.00 points for beating Minnesota, it would be your full potential weighted win percentage with that game counted in. Therefore, if you beat Minnesota on the road, but lost another game at home and another on the road, you would get 6.0/3.2 bonus points, 5.00 for beating Minnesota * 1.2 because it was on the road so it is weighted, and the 3.2 because you have 1.2+1.2+0.8. Does that make more sense to anyone?

 

[RHamilton]

Re: NCAA Selection Process

Specifically, it has to do with the last two paragraphs of information, and perhaps I have been weighting the quality wins bonus a little too firm. Instead of a full 5.00 points for beating Minnesota, it would be your full potential weighted win percentage with that game counted in. Therefore, if you beat Minnesota on the road, but lost another game at home and another on the road, you would get 6.0/3.2 bonus points, 5.00 for beating Minnesota * 1.2 because it was on the road so it is weighted, and the 3.2 because you have 1.2+1.2+0.8. Does that make more sense to anyone?

That looks like the correct interpretation to me.

I started implementing the changes in my PWR predictor and ran in to a couple things I don't like. First, it seems that tying an opponent at home has a better effect on your winning percentage than tying away, which should not be the case in my opinion. If my understanding of the weighting is correct, a tie is half a win, so a tie at home is worth 0.4 out of 0.8 points, while a tie away is worth 0.6 out of 1.2 points. I know you had a post earlier where you suggested that ties were always out of 1 point, which objectively makes more sense to me, but that's not how I'm interpreting the weighting. I couldn't find any basketball precedent (no ties), but I did find this resource which listed point values for ties.

As an example of this issue, RPI has two ties at home this season. Currently RPI's winning percentage is .739 (6.8 out of 9.2). If our ties were moved to be away ties, our winning percentage drops to .720 (7.2 out of 10). Under your compromise, our winning current percentage would be .708, (6.8 out of 9.6), while if they were away ties it'd bloom to .75 (7.2/9.6) which correctly weights the importance of away ties.

I additionally ran in to a snag when removing games that could have a negative effect on RPI. It seems to me that there could be a situation where you beat an inferior team twice in a home and home, but you would only want to remove the home win, as .8/.8 has a weaker impact on win percentage than a 1.2/1.2 win -- and that difference could help to counteract the negative effect on OWP and OOWP. I don't really like the implications of this, as a bad team is a bad team, but I suppose beating a bad team away is marginally harder than beating a bad team at home. Basketball should have this problem too; any idea how it's handled there?

 

[Dutchman]

Re: NCAA Selection Process

Update after games played on 10 November 2013:

Quality Wins Bonus:
5.00 Minnesota
4.75 St. Cloud State
4.50 Providence
4.25 Quinnipiac
4.00 Boston College
3.75 Michigan
3.50 Northeastern
3.25 Miami
3.00 Ferris State
2.75 Wisconsin
2.50 Notre Dame
2.25 LSSU
2.00 Bowling Green
1.75 RPI
1.50 North Dakota
1.25 Yale
1.00 Brown
0.75 St. Lawrence
0.50 UMASS Lowell
0.25 Clarkson

And the tournament field:

Boston College
Minnesota
St. Cloud State
Providence

Miami
Quinnipiac
Michigan
St. Lawrence

Notre Dame
Wisconsin
Northeastern
Boston University

Ferris State
North Dakota
UMASS Lowell
AHA Champ (39 - Air Force)

So, let me see if I have this correct. If the NCAA were going to use the new formula to determine the selections for the 2013-2014 NCAA D-I College Hockey Playoffs, and the decision were made today, we would select 6 teams from Hockey East and two teams from the ECAC. Boy, that sounds fair, especially given last year This number stuff is going from bad to worse. Who at the NCAA is making this decision? Are they running any simulations to understand the sensitivity of the numbers? Have they applied these new rules to past years to see how things might have turned out? I doubt Yale and Union would have made the playoffs last year.

 

[FlagDUDE08]

Re: NCAA Selection Process

That looks like the correct interpretation to me.

I started implementing the changes in my PWR predictor and ran in to a couple things I don't like. First, it seems that tying an opponent at home has a better effect on your winning percentage than tying away, which should not be the case in my opinion. If my understanding of the weighting is correct, a tie is half a win, so a tie at home is worth 0.4 out of 0.8 points, while a tie away is worth 0.6 out of 1.2 points. I know you had a post earlier where you suggested that ties were always out of 1 point, which objectively makes more sense to me, but that's not how I'm interpreting the weighting. I couldn't find any basketball precedent (no ties), but I did find this resource which listed point values for ties.

As an example of this issue, RPI has two ties at home this season. Currently RPI's winning percentage is .739 (6.8 out of 9.2). If our ties were moved to be away ties, our winning percentage drops to .720 (7.2 out of 10). Under your compromise, our winning current percentage would be .708, (6.8 out of 9.6), while if they were away ties it'd bloom to .75 (7.2/9.6) which correctly weights the importance of away ties.

I additionally ran in to a snag when removing games that could have a negative effect on RPI. It seems to me that there could be a situation where you beat an inferior team twice in a home and home, but you would only want to remove the home win, as .8/.8 has a weaker impact on win percentage than a 1.2/1.2 win -- and that difference could help to counteract the negative effect on OWP and OOWP. I don't really like the implications of this, as a bad team is a bad team, but I suppose beating a bad team away is marginally harder than beating a bad team at home. Basketball should have this problem too; any idea how it's handled there?

With ties, it is specifically defined by the NCAA as one-half win and one-half loss. You can use the distributive property to create 1/2 (win and loss), and that's how you get that.

As for the removals, you pretty much have to take everything into account. Sometimes it's only one game that gets removed. Best advice I have it to take it one game at a time. If the RatingsPI improves without the game, leave it out.

 

[FlagDUDE08]

Re: NCAA Selection Process

Update after games played on 10 November 2013, adjusted for the discussion we just had regarding how the Quality Wins Bonus works:

Quality Wins Bonus:
5.00 Minnesota
4.75 St. Cloud State
4.50 Providence
4.25 Quinnipiac
4.00 Boston College
3.75 Michigan
3.50 Northeastern
3.25 Miami
3.00 Ferris State
2.75 Wisconsin
2.50 Notre Dame
2.25 LSSU
2.00 Bowling Green
1.75 RPI
1.50 North Dakota
1.25 Yale
1.00 Brown
0.75 St. Lawrence
0.50 UMASS Lowell
0.25 Clarkson

And the tournament field:

Minnesota
St. Cloud State
Providence
Boston College

Quinnipiac
Michigan
Miami
Northeastern

Wisconsin
Ferris State
Notre Dame
LSSU

Bowling Green
Yale
North Dakota
AHA Champ (37 - Air Force)

 

[Priceless]

Re: NCAA Selection Process

So, let me see if I have this correct. If the NCAA were going to use the new formula to determine the selections for the 2013-2014 NCAA D-I College Hockey Playoffs, and the decision were made today, we would select 6 teams from Hockey East and two teams from the ECAC. Boy, that sounds fair, especially given last year This number stuff is going from bad to worse. Who at the NCAA is making this decision? Are they running any simulations to understand the sensitivity of the numbers? Have they applied these new rules to past years to see how things might have turned out? I doubt Yale and Union would have made the playoffs last year.

1) This is November 11 and we still have months to go in the season.
2) Criteria change almost every year. Yes, I am certain the NCAA ran plenty of simulations before approving these changes. As I recall, when these changes were announced CHN ran a simulation using last season's results and the field remained unchanged but some seeds were different.

Flaggy, you might want to check the other thread

 

[RHamilton]

Re: NCAA Selection Process

With ties, it is specifically defined by the NCAA as one-half win and one-half loss. You can use the distributive property to create 1/2 (win and loss), and that's how you get that.

Great, I like that interpretation on multiple levels.

As for the removals, you pretty much have to take everything into account. Sometimes it's only one game that gets removed. Best advice I have it to take it one game at a time. If the RatingsPI improves without the game, leave it out.

Indeed, not a big adjustment, more of an interesting observation.