Changes to Pairwise

[Biddco]

Re: Changes to Pairwise

Let's just put the four most lucrative programs in the Frozen Four every year. That will make the sponsors happy, ESPN happy, the arena happy and the NCAA happy.

Are you saying that what these proposed changes will do?

 

[LTsatch]

Re: Changes to Pairwise

It'll be Minnesota, Wisconsin and North Dakota every year. The 4th team will either be Michigan, New Hampshire, Maine or a private school that can show revenue above $2.75M/year

Then some team (like Yale) will show up and beat them all, ruining everyone's perfect world. Theses new rules seem to allow more personal opinion of the Committee into the equation, which would not be an improvement to the current system.

 

[jcarter7669]

Re: Changes to Pairwise

Then some team (like Yale) will show up and beat them all, ruining everyone's perfect world. Theses new rules seem to allow more personal opinion of the Committee into the equation, which would not be an improvement to the current system.

Ylae... skating circles around Wisconsin, Minnesota, and North Dakota

 

[The Exiled One]

Re: Changes to Pairwise

Complex formulae require more subjective decisions (do we add these extra variables? Do we average them? What multiplying coefficient do we use?), so it's impossible to add extra variables or criteria without increasing the risk of being more subjective.

Which measurements being selected and how they are weighted can be subjective, but like Patman pointed out, they each have an equal opportunity to be gamed by any individual team. So, they can only be subjective when applied retroactively. Going forward, all criteria are objective.

That being said, the NCAA is effectively now using criteria to modify behavior in the same way that the government uses taxes to modify behavior. The question isn't "how are they modifying criteria to manipulate the field?", it's "how are they using criteria to manipulate scheduling?" In this case, I agree with the goal of increasing road trips to smaller schools (though my school has typically been in the middle).

 

[purpleinnebraska]

Re: Changes to Pairwise

Just that a win on the road is worth more than a win at home.

But why should it be? The Pairwise already takes into account which teams are bad and which are good. Why should a road win over a bad team get a bonus attached to it? A road win over a good team doesn't need a bonus; the win itself was already a "bonus." And what is the road win bonus? The proponents of this like to argue that it's just "objective math," but it's not even math until someone puts a mostly subjective number on it.

It's also hard to see the long-term wisdom of telling some of the few revenue-producing teams in our sport that they should play road games in a 1200 seat building that's never full. In football and basketball, the rules are pretty clear: If you're a mid-major, make sure your road uniforms are clean and ready to go.

 

[jcarter7669]

Re: Changes to Pairwise

In football and basketball, the rules are pretty clear: If you're a mid-major, make sure your road uniforms are clean and ready to go.

When was the objective to be like College football and basketball?

 

[Priceless]

Re: Changes to Pairwise

Are you saying that what these proposed changes will do?

Have no idea what the changes will do because we haven't seen the details yet. But if our goal is to make certain people happy, that's the answer.

 

[LynahFan]

Re: Changes to Pairwise

It's not objective. It's just formulaic...

I am starting to wonder if this was a compromise of the form "we will give you this if you give us that". The schools that may advantage from a home favoring system would be disadvantaged with a quality wins construction and vice versa. Quality wins would incentivize power conferences. Quality non-conf would only incentivize the top teams to play each other.

In any system, if the "model" is not true then it can be gamed. Current pairwise is gameable but its not so certain how, especially with so many conference tilts kind of averaging things out for a league. If you believe krach then its more important for PWR to pick up wins against lesser competition as krach favors team w strong SOS and PWR doesn't as much.

Granted, the strength of said gaming was never clear. Adding factors usually creates more undesirable incentives than it solves due to the additional complexity and de-weighting factors that were more useful to capture things.

The key question is whether or not the new system captures team ability more than the old knowing people will take advantage of the system.

I am pessimistic here. I see gameable features here being crafted by people who aren't sure what they are doing.

The fact that i would have some difficulty with the idea of calibrating these factors appropriately should give pause.

Given that schedules are largely set years in advance, good luck to anyone trying to game the system - we don't even know what the criteria will be THIS year, much less 5 years from now - not to mention trying to guess which teams will be strong vs. weak in that timeframe.

 

[FlagDUDE08]

Re: Changes to Pairwise

Quality wins bonus just sounds like they're removing the 10 game minimum from the TUC requirement. So long as the formula for determining RatingsPI isn't being messed with, I'm not seeing the issue.

The bottom line is that a formula is set when the season begins for who is in and who is out. If you don't think you can win your conference tournament, play to the formula. It's the same for everyone. Win your games, and you'll get in.

 

[TigerFan86-87]

Re: Changes to Pairwise

It'll be Minnesota, Wisconsin and North Dakota every year. The 4th team will either be Michigan, New Hampshire, Maine or a private school that can show revenue above $2.75M/year

Actually, if ESPN and the NCAA have say in it, you can leave North Dakota, Maine, and New Hampshire out of it (great hockey programs, no recognition otherwise). They'd choose the FF out of the B1G teams, Notre Dame, UConn, Boston College, and that's about it.

 

[Priceless]

Actually, if ESPN and the NCAA have say in it, you can leave North Dakota, Maine, and New Hampshire out of it (great hockey programs, no recognition otherwise). They'd choose the FF out of the B1G teams, Notre Dame, UConn, Boston College, and that's about it.

ESPN likes a Cinderella story every once in a while.

 

[camilo]

Re: Changes to Pairwise

But why should it be? The Pairwise already takes into account which teams are bad and which are good. Why should a road win over a bad team get a bonus attached to it? A road win over a good team doesn't need a bonus; the win itself was already a "bonus." And what is the road win bonus? The proponents of this like to argue that it's just "objective math," but it's not even math until someone puts a mostly subjective number on it.

OK, let's do apples to apples.

A road win over a good team is harder than a home win over a good team - do you disagree? Why shouldn't that be acknowledged?
A road win over a bad team is harder than a home win over a bad team - do you disagree? Why shouldn't that be acknowledged?

It's also hard to see the long-term wisdom of telling some of the few revenue-producing teams in our sport that they should play road games in a 1200 seat building that's never full. In football and basketball, the rules are pretty clear: If you're a mid-major, make sure your road uniforms are clean and ready to go.

Tell that to the poor schmuck who plays for the smaller school who spends so much more time away from home and class. This is a college sport after all, big time programs or not. Send your kid to UA Huntsville or one of the Alaska schools and tell me you think it's ok for them to spend 2-3X as much time away from home and class.

I know that finances rule the NCAA, but that doesn't make it right or that we should just agree about it.

I say "forcing" or "incentivizing" larger programs to travel more is a valid concern of the college hockey system.

 

[Patman]

Given that schedules are largely set years in advance, good luck to anyone trying to game the system - we don't even know what the criteria will be THIS year, much less 5 years from now - not to mention trying to guess which teams will be strong vs. weak in that timeframe.

They aren't scheduled years in advance... Many series are one year ahead at best. If this were football I'd agree but even they are able to game the system. edit #2: They do so mainly by avoiding tough teams in non-conf play.

----

Maine Mom... My advanced degree is in statistics. I referee papers for JQAS (Journal for the Quantitative Analysis of Sports). This is as close to my job without it actually being my job. But yes, my true gripe is that I'm a fan of the hard luck River Hawks... That must be why. Not my interest in estimation methodology, simulation, statistical practice, understanding model forms, and understanding how and why methodology can fail

For that matter, this topic should be worthy of a blog post (for a friend's stat blog) when the fine details come out... Or when I get around to looking at them.

Still not going to make up for never having put these methods under the microscope as such but its mostly because I don't want to go through that headache.

edit: I just saw your other post. I'm hopeful the board nukes it, even if I get a reprimand for the above.

 

[amherstblackbear]

Re: Changes to Pairwise

It's not objective. It's just formulaic...

This.

In ranking hockey teams, there's nearly an infinite number of combinations of parameters and weights. Say 100 of them are considered "realistic." Subjectivity enters into the equation right there. Then narrow the 100 possibilities down to one. What makes that one the best? Again, it's probably subjective. I guess the NCAA could try to identify the model that best predicts (retro-dicts?) results from the past x number of seasons . . . you could defend that choice as being non-arbitrary. But that's not what's going on here.

ABB's law says that any deviation from an existing formula will inevitably be to the advantage of the person/program doing the tinkering.

For that reason alone, I oppose ad hoc tinkering with pairwise. Unless we're going to wipe the slate clean and have a "once-and-for-all" conversation of whether predictive power is the best way to judge ranking systems, and how best to achieve predictive power. Considering KRACH never got off the ground with the NCAA, I don't see that happening.

So . . . I say leave well enough alone.

 

[purpleinnebraska]

Re: Changes to Pairwise

OK, let's do apples to apples.

A road win over a good team is harder than a home win over a good team - do you disagree? Why shouldn't that be acknowledged?
A road win over a bad team is harder than a home win over a bad team - do you disagree? Why shouldn't that be acknowledged?

Tell that to the poor schmuck who plays for the smaller school who spends so much more time away from home and class. This is a college sport after all, big time programs or not. Send your kid to UA Huntsville or one of the Alaska schools and tell me you think it's ok for them to spend 2-3X as much time away from home and class.

I know that finances rule the NCAA, but that doesn't make it right or that we should just agree about it.

I say "forcing" or "incentivizing" larger programs to travel more is a valid concern of the college hockey system.

I don't know that there's much one way or another on a road win versus a home win. As Patman noted earlier in this discussion, "Home ice advantage does exist but it's not extreme." So we should change the system for this? The Pairwise already rewards you for beating a good team, wherever the game takes place.

As for the "finances rule the NCAA" - Finances rule everything. This isn't an NCAA issue. To tell Minnesota, "You need to make more money-losing trips" is a bad strategy, whether it's hockey, academics, marching band, whatever. I just don't see how the sport benefits from having a team that makes money give up home dates to go play at a place where the financial model makes it impossible to make money.

The UAH and Alaska travel will always exist, because of the schools' locations. Changing the Pairwise isn't going to move those schools one inch closer to Mankato or Bowling Green. If a kid doesn't like to travel, he better skip the WCHA in general, and those schools in particular, under any Pairwise configuration.

 

[MinnesotaNorthStar]

Re: Changes to Pairwise

Tell that to the poor schmuck who plays for the smaller school who spends so much more time away from home and class. This is a college sport after all, big time programs or not. Send your kid to UA Huntsville or one of the Alaska schools and tell me you think it's ok for them to spend 2-3X as much time away from home and class.

I know that finances rule the NCAA, but that doesn't make it right or that we should just agree about it.

I say "forcing" or "incentivizing" larger programs to travel more is a valid concern of the college hockey system.

Do you mean the same poor schmuck that probably just spent the last couple years in juniors or at the USNTDP where they probably spent more time away from home and class?

If you want to "force" or "incentivize" the big schools into going on the road more...don't schedule them without a return trip.

 

[CLS]

Re: Changes to Pairwise

I don't know that there's much one way or another on a road win versus a home win. As Patman noted earlier in this discussion, "Home ice advantage does exist but it's not extreme." So we should change the system for this? The Pairwise already rewards you for beating a good team, wherever the game takes place.

I'd like to hear a more quantitative statement from Patman on that; what is the home ice advantage?. If, say, the most home-heavy schedule is 16 home and 14 on the road and the most road-heavy schedule us 14 home and 16 on the road, then I'd agree it doesn't make much difference. But it appears to me that the home/away breaks are more uneven than that. And if that's the case, then even a small advantage in each home game can have a large cumulative effect. If a coin comes up heads 50.001% of the time, you can go broke (or make a fortune) if you flip it often enough.

As for the "finances rule the NCAA" - Finances rule everything. This isn't an NCAA issue. To tell Minnesota, "You need to make more money-losing trips" is a bad strategy, whether it's hockey, academics, marching band, whatever. I just don't see how the sport benefits from having a team that makes money give up home dates to go play at a place where the financial model makes it impossible to make money. ...

That's not what they're telling Minnesota (or the small schools that travel to Minneapolis if they want to play Minnesota). They're saying "Finances say that Minnesota will have more home games than away games. We don't want that to be a competitive advantage."

 

[Patman]

Re: Changes to Pairwise

I'd like to hear a more quantitative statement from Patman on that; what is the home ice advantage?. If, say, the most home-heavy schedule is 16 home and 14 on the road and the most road-heavy schedule us 14 home and 16 on the road, then I'd agree it doesn't make much difference. But it appears to me that the home/away breaks are more uneven than that. And if that's the case, then even a small advantage in each home game can have a large cumulative effect. If a coin comes up heads 50.001% of the time, you can go broke (or make a fortune) if you flip it often enough.

That's not what they're telling Minnesota (or the small schools that travel to Minneapolis if they want to play Minnesota). They're saying "Finances say that Minnesota will have more home games than away games. We don't want that to be a competitive advantage."

The CHODR regression model seems to suggest 5% GF more, 5% GA less... so about a net .3 goals or so... maybe a little less. I forget what that means in terms of equally comparable team win %.

---

One time I ran a model where each team had a home ice component... the big home ice advantage that year (and I forget how far back this was) was Sacred Heart. Squirrelly results will always appear regardless of the method chosen.

 

[joecct]

Re: Changes to Pairwise

According to CHN
Weight a win @ home or a loss on the road by 0.8
Weight a win on the road or a home loss by 1.2

Also a sliding bonus for beating a top 20 (on Selection Sundae??) RPI team from 5.0 to 0.25.

CHN crunching numbers to how it pans out. Somewhere, wherever Jon Whelan is, the CPU usage just tripled.

 

[The Exiled One]

Re: Changes to Pairwise

I'd love to see this applied retroactively to last season (or couple of seasons) to see how the results would have been affected. Of course, it's important to remember that these new criteria were not devised to adjust the field, but to adjust future scheduling behavior. So, even if last year's field would not have been altered, it has no bearing on the NCAA's justification for the criteria.