How to improve the Pairwise

[Shirtless Guy]

Since the last change, I have always found it strange that only head to head games can flip a comparison from the team with the higher RPI, because a superior common opponent record isn't enough by itself because tiebreaker goes to RPI. With the RPI calculation so flawed and skewed specifically for hockey to avoid negative impact games, it doesn't make sense to put so much emphasis on something so arbitrary. Does anyone have a suggestion on how to change the current formulation to find a 4th criteria to allow flipping a comparison without head to head games? My first thought was to incorporate RPI w/QWB and RPI w/o QWB since not every team gets equal shots at QWB and only so much of that can be the responsibility of the team itself. If we are so confident in the RPI we choose to use, it would make sense to use both numbers in some fashion.

Other thoughts? I'd like to gather a few ideas and incorporate them into the last 3 years of the current system and see how it would have changed the field.

 

[ExileOnDaytonStreet]

Re: How to improve the Pairwise

[Insert 'use KRACH instead of RPI' argument here]

 

[Shirtless Guy]

Re: How to improve the Pairwise

[Insert 'use KRACH instead of RPI' argument here]

get conferences to reduce number of conference games to have more interconference matchups to allow for better cross conference data...

 

[UML Puck Hawk]

Re: How to improve the Pairwise

What are people's thoughts on using H2H instead of RPI as the first tiebreaker for comparison. This year as an example, Lowell held a 3-1 H2H comparison, but only a 0.003 RPI advantage. They tied with an identical number of comparisons won, but Lowell owned the compare over BU. I understand why RPI is used, and should be in most cases, but when a team holds the actual comparison (holds com-op and H2H) wouldn't it make sense to break the tie with the comparison win itself in stead of RPI?

 

[ExileOnDaytonStreet]

Re: How to improve the Pairwise

What are people's thoughts on using H2H instead of RPI as the first tiebreaker for comparison. This year as an example, Lowell held a 3-1 H2H comparison, but only a 0.003 RPI advantage. They tied with an identical number of comparisons won, but Lowell owned the compare over BU. I understand why RPI is used, and should be in most cases, but when a team holds the actual comparison (holds com-op and H2H) wouldn't it make sense to break the tie with the comparison win itself in stead of RPI?

I would say that there's no issue at all, since comparing two teams directly is sort of the point of the PWR system. But H2H already has a fairly large weight in the system as it is, given how many comparison points you get for each victory. Making it the first tie-breaker almost seems redundant.

 

[Shirtless Guy]

Re: How to improve the Pairwise

What are people's thoughts on using H2H instead of RPI as the first tiebreaker for comparison. This year as an example, Lowell held a 3-1 H2H comparison, but only a 0.003 RPI advantage. They tied with an identical number of comparisons won, but Lowell owned the compare over BU. I understand why RPI is used, and should be in most cases, but when a team holds the actual comparison (holds com-op and H2H) wouldn't it make sense to break the tie with the comparison win itself in stead of RPI?

If a team has both ComOp and H2H while losing RPI, they will flip the comparison. The biggest problem I have is flipping it can only occur if teams play head to head. Lowell gets 3 pts for H2H wins, BU gets 1, UML gets 1 more for RPI advantage...ComOp was actually a draw so UML wins that comparison 4-1.

 

[Shirtless Guy]

Re: How to improve the Pairwise

I truly don't believe there is a better way.

Moving this over from the other pairwise thread...

The pairwise system is the best way in general, but that doesn't mean how we arrive at the numbers used can't be done better and that a 4th comparison can't be determined that is equally valuable to the 3 we already use...I think it is a relatively fair system, but after investing a ton of time into creating my own calculator, I certainly see flaws.

For example...the result of every game directly impacts the weight of Opp's W% and OppOpp W%. Should UMD's Opp W% be determined on whether or not they win a game? Or does it make sense that the 2nd and 3rd components of RPI should not be weighted based on the outcome of the game?

Also, with so much emphasis on the OppOpp W%, should that calculated valued exclude games against the team we're using, as in UMD's OppOpp W% should exclude all games played with UMD. Doesn't that make sense? Wouldn't that favor teams who are closer to .500 in conference but play in a tough conference because everyone else gets a better OppOpp W% value, the largest component of RPI now, while a top team in a weaker conference gets penalized for winning most of their games within their conference, driving down the OppOpp W% component?

 

[UML Puck Hawk]

Re: How to improve the Pairwise

If a team has both ComOp and H2H while losing RPI, they will flip the comparison. The biggest problem I have is flipping it can only occur if teams play head to head. Lowell gets 3 pts for H2H wins, BU gets 1, UML gets 1 more for RPI advantage...ComOp was actually a draw so UML wins that comparison 4-1.

Maybe I didn't explain what I was getting at well, they tied with 52 comparisons. The current tiebreak upon a tie in the pairwise is RPI. I was saying who holds the comparison should get the advantage as opposed to settling that tie with RPI again.

I understand how the PWR works, I am commenting on the tiebreak that occurs when teams have the same number of compares won. I am saying the team that owns the comparison between the two tied teams should receive the higher seeding.

RPI currently (1. decides one portion of who wins a compare, 2. breaks the tie in the event the comparison is tied, 3. breaks the tie in the event two teams have the same number of comparisons won). I am commenting on 3, saying perhaps a better tiebreak here would be who owns the comparison between the two tied teams.

 

[billmich88888]

Re: How to improve the Pairwise

I take it that we don't want to (re) introduce another comparison factor, like record in last 20 games?
reward teams coming in hot?

 

[Spartanforlife4]

Re: How to improve the Pairwise

get conferences to reduce number of conference games to have more interconference matchups to allow for better cross conference data...

Oh, so now the Big Ten has the right idea on something!

Fairly certain baseball has a penalty for losing to teams in the bottom 50 or so. Of course, there are around 300 teams, so maybe make it the bottom 10 for hockey?

 

[Shirtless Guy]

Re: How to improve the Pairwise

Maybe I didn't explain what I was getting at well, they tied with 52 comparisons. The current tiebreak upon a tie in the pairwise is RPI. I was saying who holds the comparison should get the advantage as opposed to settling that tie with RPI again.

I understand how the PWR works, I am commenting on the tiebreak that occurs when teams have the same number of compares won. I am saying the team that owns the comparison between the two tied teams should receive the higher seeding.

RPI currently (1. decides one portion of who wins a compare, 2. breaks the tie in the event the comparison is tied, 3. breaks the tie in the event two teams have the same number of comparisons won). I am commenting on 3, saying perhaps a better tiebreak here would be who owns the comparison between the two tied teams.

fair enough, I agree with that...not positive you're right that 3 isn't who owns the head-to-head comparison win, but if it isn't I agree with that change.

 

[tape]

Re: How to improve the Pairwise

The biggest problem I have is flipping it can only occur if teams play head to head.

This is my biggest concern as well. When a team's schedule contains at most about 1/3 of Division I, H2H just doesn't come up that often, and with RPI the tiebreaker, it means ComOp doesn't even matter unless there's also H2H.

Imagine a scenario where teams are extremely close in the RPI - Lowell and BU this year are a great example since Lowell is only up on BU by .0003. In reality, they played head-to-head, but suppose they didn't, and suppose that rather than being tied in ComOp, BU obliterated Lowell in that category - say BU was 8-1 against common opponents while somehow Lowell went 1-8 (I know it's done by adding the win% against each team but you get the point). It almost seems unfair that two teams in a virtual tie in RPI but with such a huge discrepancy in ComOp would be decided by the RPI in such a scenario.

 

[ExileOnDaytonStreet]

Re: How to improve the Pairwise

This is my biggest concern as well. When a team's schedule contains at most about 1/3 of Division I, H2H just doesn't come up that often, and with RPI the tiebreaker, it means ComOp doesn't even matter unless there's also H2H.

Imagine a scenario where teams are extremely close in the RPI - Lowell and BU this year are a great example since Lowell is only up on BU by .0003. In reality, they played head-to-head, but suppose they didn't, and suppose that rather than being tied in ComOp, BU obliterated Lowell in that category - say BU was 8-1 against common opponents while somehow Lowell went 1-8 (I know it's done by adding the win% against each team but you get the point). It almost seems unfair that two teams in a virtual tie in RPI but with such a huge discrepancy in ComOp would be decided by the RPI in such a scenario.

True all around.

Still better than the "oh whatever, let's just take straw polls until we like it" method used by the basketball committee.

 

[Patman]

Re: How to improve the Pairwise

A few considerations...

Anything done must be able to be done by a fastidious layperson. As awful as things are, the RPI can be computed in a long afternoon with a simple calculator.

The goal of the ranking is unclear. The TUC factor was meant to consider performance against a more select set of opponents. Of course you have to preselect before you select and all that fun.

The means of evaluation are currently limited. Even if you blank out the scores and simulate what is to judge good and bad? Will we accept an aggregate testing bed? What is the value of mis-selection.

What is it we value in these teams? The capacity to be the best amongst all? What does that mean? Is it just the best group amongst the entire selection? The last and the first are not the same.

----

I can always set up a synthetic data testbed. That's actually quite easy. Modeling may take the better part of the afternoon but nothing I can't do.

 

[UML Puck Hawk]

Re: How to improve the Pairwise

fair enough, I agree with that...not positive you're right that 3 isn't who owns the head-to-head comparison win, but if it isn't I agree with that change.

USCHO and College Hockey News break it with RPI. That said we don't really know what the committee would use, except if we look at history.

 

[fightingsiouxfan16]

Re: How to improve the Pairwise

theres really nothing wrong with it. conference winners get AQs which i know is controversial to begin in. But win ur games. Thats all that matters. Bad losses are gonna hurt u

 

[Shirtless Guy]

Re: How to improve the Pairwise

If you introduce a fourth criteria, you open the system up to that many more 2-2 pair ties which would result in more ties broken by RPI.

I disagree with this...for the final PWR this year there were 244 comparisons that were a 1-1 tie broken by the RPI tiebreaker. I need more time to analyze how often this happen with teams with similar RPI values (within 0.0100 pts or so). The H2H comparison is also not calculated as a 1-0 victory, it is a whole comparison point per win so unless the head to head match up is 1-0 or 1-0-1, it is impossible for a comparison to be tied 2-2 without a specific comparison like ComOp being a draw.

I don't know how you improve on the pwr/pairwise but I do think additional criteria should be added for when you play someone and where they end up at the end (Beating UNO early was a big help but later when they took the fall we found out where they actually should have been). Maybe that is already factored in, I don't know. Maybe a weighting factor for games later in the year, better idea where the teams actually are.

Another is prime players in and out of the lineup- or injuries. I have no idea how you would factor this in mathematically but it is a big factor.
SCSU played QU at the beginning of the season on their home turf with a young D core. 3 freshmen D (Schuldt, Borgen and Lizotte) and without our best player Ethan Prow. A very small sophomore Ilvonen in for Prow and Nevalainen just back from a concussion. So maybe something to consider for injuries/healthy? Doesn't QU have a player out (Anas) I think so they are not the same team as they are with him, but they have depth. Other teams don't.

A big one was last year. I thought Miami was the best team in the field. They get beat by Providence but they are missing their top two players, Blake Coleman (suspended for a hit in the NCHC championship game) and Riley Barber ( injury).

Same with North Dakota, they lost Mark MacMillan due to an injury late in the season (Brodzinski blew his kneecap/leg(J/K) off blocking a shot). He also hurt another ND player on the same PP.
Brodzinski himself hurt his ankle in the Mich Tech game and it really affected his play against North Dakota in the Regional final.

Last year the Blackhawks almost didn't make the playoffs when Kane was out and when he came back they went on a tear. Being healthy at the end of the year obviously helps your rating and how your team is peaking or not.

included as carry over from the other thread.

 

[manurespreader]

Re: How to improve the Pairwise

My issues with it are as follows:
I think the OppOpp is weighted too high and I'd find some way to lower it slightly.
Second, Since there is so much parity in the game, the QWB doesn't accurately reflect the quality of teams you beat. I mean really, number 20 is not so much different than number 30. I'd lower the QWB cutoff to top 10 teams vs top 20 or 15.
three, it seems to me that once a team clinches a spot, they don't play quite as hard as they did previously. it's not egregious, but enough to make a difference. so i'd find a way to reduce the effect of beating a team that has already clinched.

 

[tape]

Re: How to improve the Pairwise

(copied from the other thread)

If you introduce a fourth criteria, you open the system up to that many more 2-2 pair ties which would result in more ties broken by RPI.

If the fourth criteria is one in which all or most teams will have data, it will result in fewer ties overall, because H2H comes up in maybe 1/3 of team comparisons and often less (depending on a team's conference they're playing maybe as few as 12 or as many as 20 opponents over the course of the season). That means that currently, most comparisons are decided only by RPI and common opponents, and since the RPI is the tiebreaker, that means most comparisons are really decided only by RPI because common opponents alone isn't enough to overcome RPI.

 

[hockeykrazy]

Re: How to improve the Pairwise

theres really nothing wrong with it. conference winners get AQs which i know is controversial to begin in. But win ur games. Thats all that matters. Bad losses are gonna hurt u

I brought up injuries or missing key players (World juniors, suspensions) and when(young inexperienced in the fall(freshmen) vs spring) you play strong/weak teams are big factors.
Not sure how you would do this mathematically. Maybe weighting factors?

The biggest thing would be to get the NCAA to let them play more games (non-conference) later in the year. It is weighted way too heavily on results early in the year.

SCSU swept Minnesota in Nov but I don't know if we get the same result if we play them in Feb.