Right. Not a cliff, a very gentle slope. Dumb question on my part.
Not dumb at all! The fact that we had a TUC cliff in the past meant this was definitely a reasonable thing to wonder.
Anyway, I figured out the difference between my numbers and USCHO's. It amounts to being an order-of-operations question.
In my calculation, what I'd been doing for the last several years was fully calculating NPI (or I guess RPI before this season) without involving QWB, and then once the calculation was complete, tacking on a QWB to every team's final NPI. For what it's worth, I know my numbers in prior years have been correct because I had a contact on the committee who was able to compare my numbers with what the NCAA was using.
This year, USCHO's Pairwise looks to be calculating each team's NPI
while including QWB in the NPI calculation iterations, instead of tacking it on at the end as a "final bonus" of sorts.
So, I have been doing this:
[ (Result1 x 0.25) + (oNPI1 without QWB x 0.75) ]
[ (Result2 x 0.25) + (oNPI2 without QWB x 0.75) ]
[ (Result3 x 0.25) + (oNPI3 without QWB x 0.75) ] divided by total included games,
AND THEN + the team's overall QWB
[ .................................................. ................ ]
[ (Resultn x 0.25) + (oNPIn without QWB x 0.75) ]
While USCHO is now doing this:
[ (Result1 x 0.25) + (oNPI1 including QWB x 0.75)
+ Game1QWB ]
[ (Result2 x 0.25) + (oNPI2 including QWB x 0.75)
+ Game2QWB ]
[ (Result3 x 0.25) + (oNPI3 including QWB x 0.75)
+ Game3QWB ] divided by total included games
[ .................................................. ......................................... ]
[ (Resultn x 0.25) + (oNPI3 including QWB x 0.75)
+ GamenQWB ]
This means that everyone's NPI is naturally higher, in part because when you remove bad wins, you're removing "bad wins" relative to a QWB-inflated NPI. And since removing bad wins is iterative, that gets compounded a few times while you remove more and more bad wins. An NPI of 0.515 with this way of calculating ends up meaning that wins over half of the D1 programs award a quality win bonus.
Intuitively the way USCHO is calculating it doesn't really seem correct to me, because QWB was always explained to me as being a "bonus" that is tacked onto your final RPI/NPI, rather than something included in the iterative calculations. Doing it the way USCHO has it also means you can have weird end results such as a game being a Quality Win, but also being removed as a Bad Win -- and indeed, that's already happening: Yale's wins over Princeton, Mercyhurst, and St. Lawrence, which would otherwise provide a Quality Win Bonus to Yale's NPI are instead being removed because they're Bad Wins.
In the end, the "Right" way to calculate this is whatever the NCAA says it is, and given that I doubt more than 5 people on the planet understand NPI well enough to even have this in-depth of a conversation about it, I really don't care what the "Right" way ends up being. I already have changes ready to push live if USCHO's numbers are right. The only thing I care about -- and I'm sure the USCHO folks have the same goal -- is to just match how the NCAA has decided to calculate it so we can provide the women's hockey community with a reliable way to follow along with how the tournament will be selected.
I reached out to a couple important people with this info to see if we can all get on the same page. If USCHO's right, then at least I can explain how they got the numbers they did and I can push my changes out to match theirs.
