Copyright 2004, Paul Kislanko

A better hammer?

Mathematically, the most concise way to define the "Pefrormance Against" algorithm is to set up a square matrix W whose elements are defined by

Wi,j = wins by team i over team j
and an N by 1 vector V that characterizes team i by an arbitrary metric. Then the "performance against" (PA) algorithm is just:
PA = [ W × [ W × V ]* ]*
When Vi is defined as team i's OWP, the result is the PASOS rating. If there is a different definition of SOS than OWP, that could be used instead - it would still result in a "Performance againt (some) SOS" ranking.

The [ W × V ]* result is the value of a win over team i, it is not a characterization of team i's opponents, so the measurement that characterizes schedule strength uses an N × N matrix G, where

Gi,j = number of games between teams i and j
and the PA algorithm uses that to define
dSOS = [ G × [ W × V ]* ] *
Note: [ result ]* indicates that in the product the results for team i are normalized by the total number of games played by team i:
[ ]*i = result i / (games played by team i).

A better screwdriver?

As we mentioned in Beyond the SOS the basic "performance against" algorithm can be used as a tool to analyze any method, using the value (not the ranking) from any other method as a replacement for OWP.

When we define Vi to be team i's PASOS, RPI, or ISR value the aSOS value ( [ W × V ]* ) is the value assigned to a win over that team and the dSOS value is a characterization of the team's schedule strength by the rating system we used to define V.

How well the "performance against" algorithm correlates to the method's ranking is a measure of how self-consistent a method is. More precisely, the difference between "performance against" rankings and the method's rankings are a measure of how accurately the method reflects strength of schedule.

For instance, we can compare the RPI to itself using "Performance Against RPI" by using the RPI as the input to the algorithm:

And for the ISR we get:

It should come as no surprise that the PA-PASOS correlates more nearly with the PASOS than PA-ISR or PA-RPI do with their respective ratings since we're using the same tool to measure the method that was used to produce its results:

It's important to understand that in these examples from the 2004 baseball season we've used the "Performance Against" algorithm to compare each method to itself. When we use a different measurement to compare them to each other, we find that the PASOS is marginally better than the RPI at ranking teams, and the ISR is better than either.

The PASOS method suggests that we also test the PASOS by using aWP (cube root of PASOS) vs PA-AWP, which correlates almost exactly when a power series is used: