By Harrison Chase and Austin Tymins
One of the coolest things to come out of sports analytics in the past few years has been the win expectancy curves that have popped up on various sites. After reading an article at 538 last year where they aggregated those curves for every NBA team, we were inspired to do the same for NFL teams, with a few statistical additions.
Scraping data from pro-football-reference.com, we found the average win expectancy at each second of every game for all 32 NFL teams. A few notes:
Games do not begin at 50% for each team – rather, they use Vegas lines to infer a prior win probability for each team.
Games either end at 100% win probability (if they won), 0% win probability (if they lost), or 50% win probability (if they tied). We cut off the graph at the end of the fourth quarter for all games.
Below is a chart showing the average win probability at any point in the game for all 32 NFL teams for the first 6 weeks of the 2015 NFL season. Because we don’t have the amazing visualization tools (or funding) of 538, we couldn’t post a nifty graphic where one could select a team and have that team’s curve light up. However, we have highlighted the curves of the Seattle Seahawks and Carolina Panthers as they will be interesting for future discussions. If any of our readers want to see a particular curve, please let us know on Twitter and we can easily highlight that curve for you.
We’ve selected these two curves for several reasons. Firstly, it is a nice visualization of our model and shows each observed game ending in the fourth quarter, regardless of how overtime turns out. Seattle is 2-4, but two of those losses came in overtime, which is why at the end of our graph their average win expectancy is 50%, not 33%. The other reason we chose these two curves is that they are remarkably similar up until the end of the game, despite the fact that Carolina is undefeated while Seattle only has two wins. This goes to show that Seattle has been absolutely awful in the fourth quarter so far this year.
So this chart is pretty cool (or at least we think so), but what else can we do with this data? We thought it would be fun to create power rankings because everyone has loved our power rankings posts in the past. One initial thought was to base our power rankings on the area under each team’s curve, a kind of average win probability across all games. The main issue with this of course is that we would not be controlling for strength of schedule. To adjust for this, we decided to use a variation of Colley’s matrix method.
The adjustment we made, that differentiates our ratings from regular Colley rankings, is that instead of looking at games won and games lost we look at expected games won and expected games lost, using the average area under the curve to calculate this. We also lightly controlled for home field advantage by adjusting each team’s average curve depending on if they had played more home or away games – however, this adjustment had marginal effect as teams have played roughly the same home and away games. Plugging in these expected games won and expected games lost, we can solve the Colley matrix and get our Probabilistic Outcomes by Win Expectancy Regularization (POWER) Rankings (shoutout to Will Ezekowitz for the name). The results are below:
These rankings are by no means perfect, but they do pass the smell test. The Patriots are number one, as they should be. The Packers are a close second and after that there is a sizable drop off. Of course, I have no doubt that your favorite team is ranked too low and you can spot a million things that should be different. Rather than talk about each team individually, we’ll just list answers to some possible concerns.
- This method does not care about head-to-head results
Seattle is higher than Carolina, the Giants are higher than the Eagles, Arizona is higher than Pittsburgh…there are many examples of teams that lost a particular matchup being higher than the team they lost to. Our method, like the Colley method, does not care about head-to-head matchups but rather how a team does on the aggregate. There could be several reasons why the winner of a head-to-head matchup is below the loser: perhaps the winner actually played better (had a high win expectancy throughout the game), perhaps the winner was playing at home and barely won, or perhaps that win was the sole bright spot in an otherwise dark season. Any fan of the game knows that the so-called “transitive property” has severe limitations, and that’s why we didn’t get bogged down in these comparisons.
- This method does not weigh recent results more heavily
This could either be a good or bad thing. On one hand, we do not want to overreact and become victim to a recency bias. One the other hand, if something has fundamentally changed about a team (i.e. Jamaal Charles going down for the season) we may want to weigh their last game more heavily. We will look into adjusting for this in the future.
- This method has a bias towards teams Vegas likes
Because the win probability charts from pro-football-reference.com use Vegas lines to calculate the win probability, they are biased towards the team Vegas has as favorites. This bias might actually make our rankings look more reasonable, as they essentially have a Vegas informed prior. Whether this is good or bad is again up for debate.
At the same time, this method does have it’s benefits. As it is built off the Colley method, it has many of the same positive attributes of that model. Rather than explain the benefits myself, I will just quote the abstract of the paper outlying the methodology behind Colley rankings:
“The scheme adjusts effectively for strength of schedule, in a way that is free of bias toward conference, tradition, or region.”
One noticeable drawback of the Colley matrix method is that is does not account for margin of victory. Our model actually does indirectly by looking at win expectancy, so teams that win big will be rewarded. It also takes into account the obvious notion that a 10 point lead at the end of the 1st quarter is not nearly as valuable as the same size lead going into the 4th quarter. In addition, it does not assign linear weight to margin of victory as Massey’s method does. Our method is actually fairly similar to Chase Stuart’s Game Scripts, which looks at the average point lead (instead of win expectancy) for a team throughout the game. However I don’t believe he has tried to use those game scripts to generate power rankings, although using the Massey method with game scripts seems like a natural extension for what he has already done.
There are many ways to improve this model that we will consider over the next few weeks/months. We could try to make each team’s curve an exponentially weighted moving average of their past games, optimizing the model to find what weighting gives it the most predictive power. We could also use the log5 method to try to predict future games – although before we do that we would want to also optimize how we want to change the spread depending on who is home or away.
Besides improving the model, we can also apply it other sports. Remembering the 538 article from before, given their data it would be relatively straightforward to calculate POWER rankings for NBA teams last season. Doing it in season might a bit more tricky as there isn’t a great source for win expectancy throughout the game but we will look into it.
As always, our twitter (@Harvard_Sports) and email are open for thoughts/comments/feedback of any kind. If you have legitimate reasons reasons for why you believe a team should be ranked higher/lower let us know and we can see if it would be beneficial to account for those factors in our model.