Well...I certainly didn't intend this to be a debate on ratings systems.
Without getting too deep into inferential stats (lest the Scooby character blows a gasket), I'll take a swing at this with some food for thought. Weighting functions in inferential statistics are never arbitrary. Typically a researcher will examine historical data and allocate more “weight” or influence to some elements of a set to reduce measurement error or inherent bias. For example, the NCAA D1 Men's Hockey Committee consulted with experts within both coaching and analytics fields and weighted road wins over home wins relative to their perceived importance. Same could be said of the QWB, the committee added another IV to incentivise scheduling top 20 teams. Not sure what you mean by the QWB as "statistical double jeopardy".
With regard to the RPI patch, it is common practice in empirical model building to assume (implicitly) that the postulated model does not describe all observations. The intent is to build a model that describes the majority of the observations. This is exactly the aim of robust statistical procedures like the RPI. KRACH uses logistic regression which is not robust, especially with it's lack of team relevant predictors. For example, one problem with the KRACH formulation is that it simply does not handle unbeaten teams. If one team has a non-zero KRACH rating, and an unbeaten team has beaten it, then the unbeaten teams' KRACH rating cannot be high enough to yield the required expected wins. Moreover, KRACH's odds ratio calculations are based upon an assumption that output data from it's inherent syllogistic paradigms are normally distributed. Is this true? Is this rational? Not in the real world. Non-robust tools that violate the assumption of homogeneity of variance are subject to large residual variances. That means a fix is needed.
Arguably the RPI model although much maligned is a robust procedure that ensures the best fit across a large sample population. One could also argue that the KRACH minus the application of a probability density model using MLE, is more retrodictive than the RPI system based upon the NCAA's intervention to also implicitly reduce residual variance.
If you accept that odds are a way of expressing the relative strengths of teams and odds are ALWAYS consistent between teams (such that if A:B and B:C express the odds of Team A beating Team B and Team B beating Team C) then KRACH is your answer to ratings that express the teams' actual wins and losses. Odds ratios are one way of determining SOS, but it is not the only way. I tend to agree with the NCAA that the RPI SOS is more accurate as a function of W%.
The ultimate purpose of the RPI formulation is to help the NCAA determine which teams go to the post-season tournament based upon how they are seeded. It's used by nearly every NCAA sport with one recent exception. IIRC in 2018, they introduced the NCAA Evaluation Tool for NCAA Men's BB (Women's BB kept their version of RPI). Stat gurus took two years to design it.
It's noteworthy that they decided to retain W% and home and away weightings while adding additional factors: Team Value Index, Net Efficiency, Winning Percentage, Adjusted Win Percentage and Scoring Margin. This looks very similar to the RPI intent to reward teams who beat other good teams. Unfortunately for KRACH enthusiasts, they didn't even give it a sniff for some of the reasons I've mentioned. Sorry about the length of this thing.
