By Julian Ryan

There are relatively few possessions in an NFL football game, so in short, turnovers matter. While a right back on the soccer pitch can be forgiven for a mislaid pass or a point guard in basketball can get away with the occasional double dribble, if a quarterback throws an interception, it can often change the course of the game. Just ask Tony Romo.

However, aside from Matt Schaub’s natural aptitude for the pick six, how random are turnovers? How much of a team’s turnover differential can be explained by teams protecting the football on offense and causing turnovers on defense, and how much is just plain dumb luck?

First, it is important to stress just how much turnovers matter in deciding a single game of football. Taking data from pro-football-reference from 2002-2013, if a team has even one more turnover than the other team over the course of a game, it will win 69.6% of the time, compare that to home teams winning about 57.2% of the time. Winning percentage only increases as the turnover differential mounts up (ties excluded in table below):

Turnover Differential |
Wins |
Losses |
Win% |

+1 |
747 |
327 |
69.6% |

+2 |
608 |
117 |
83.9% |

+3 |
342 |
35 |
90.7% |

+4 |
186 |
6 |
96.9% |

+5 |
68 |
3 |
95.8% |

+6 |
18 |
0 |
100.0% |

+7 |
5 |
0 |
100.0% |

If we expand to look at turnover differential on a seasonal level, its importance is even clearer. Regressing season wins against seasonal turnover differential indicates that each additional positive turnover is worth about 0.2 wins with an R-squared value of a whopping 0.419. That suggests that the seasonal turnover differential of a team alone – ignoring points scored, points against, strength of schedule and everything else that affects winning percentage – explains 41.9% of variation in regular season win totals from 2002-2013. Again, turnovers matter a lot.

Some of a team’s turnover differential is due to talent —Aaron Rodgers throwing very few interceptions and J.J. Watt strip-sacking anything that moves come to mind as examples—and some is due to luck, because things like this happen. To ascertain how much each of these two factors contribute to turnover differential over the course of a season, we can borrow from the pioneering work of Tom Tango. If we take it as given that luck is independent of talent, then the following formula will hold:

Var (Observed TO Differential) = Var (TO Differential from Luck) + Var (TO Differential from Talent)

Tango then concludes what percentage of wins in baseball is due to skill because we observe the left hand side and can calculate the variance due to luck. This latter amount is what the variance would be if turnovers (or baseball wins for Tango) had nothing to do with skill whatsoever. For baseball wins, this is equivalent to flipping a coin for each game so a season’s win total is a binomial distribution with 162 trials and a success probability of 0.5. Calculating the variance of this ‘luck distribution’ allows us to assign the remaining variance to talent. To repeat this analysis for turnovers, we need to find our own ‘luck distribution’.

The Poisson distribution is used to model rare independent discrete events over a period of time with a constant (low) probability of the event occurring in each period, like earthquakes. It makes sense to model total turnovers per game this way, for if we consider there to be a constant probability of turnover per possession and an average number of possessions per game, then the total per game should follow a Poisson probability density. This isn’t perfect as possessions per game is not constant and turnovers may shorten drives and thus increase possessions per game (and also are not independent) but on the macro level the data almost perfectly matches the Poisson model. The bar graph below shows the observed probability density of total turnovers per game alongside what we would expect if total turnovers followed a Poisson distribution with mean 3.39 (the average in our data set).

Under the assumption that total turnovers per game is now Poisson with a mean of 3.39, we can now calculate our “luck distribution”. Under this distribution, neither team is better or worse at producing or creating turnovers, so any change of possession happens entirely by luck. This, together with the fact that the sum of independent Poisson random variables is itself a Poisson random variable (with the mean equaling the sum of the means of the original variables), means we can split up this Poisson into two halves: one for the number of turnovers created on defense, and the other for turnovers conceded on offense. In words, for a single team in a single game, the turnovers it “forces” are a Poisson random variable with mean 1.69 as are the turnovers it “commits”.

Each team plays 16 games per season, so the total seasonal turnovers “forced” is the sum of 16 independent Poissons with mean 1.69, which is hence itself a Poisson with mean 27.1. The same process holds for turnovers “committed”. Seasonal turnover differential would thus simply be the difference between these two independent random variables.

The difference between two independent Poisson variable is not itself Poisson, but because they are independent (which implies covariance equals zero) it is easy to calculate the variance of seasonal turnover differential under our luck distribution, since an individual Poisson’s variance is equivalent to its mean. After calculating the observed variance of seasonal turnover differentials, we are able to see how much of that number is due to plain dumb luck (confidence intervals calculated by considering the error in calculating the mean total turnovers per game in our 3072 game sample):

Variance |
Value |
Percentage |
95% Confidence Interval |

Var (Observed Seasonal TO Differential) |
99.0 |
– |
– |

Var (Seasonal TO Differential from Luck) |
54.2 |
54.7% |
(53.7%, 55.8%) |

Var (Seasonal TO differential from Talent) |
44.8 |
45.3% |
(46.3%, 44.2%) |

So after a few hundred words of statistics, we arrive at a whopping conclusion that just over half of seasonal turnover differential is due to luck. That’s huge, especially when you consider that (from earlier) seasonal turnover differential explains over 40% of seasonal winning percentage.

At first glance, this does seem very high to me but evidence for this magnitude is the extraordinary year-to-year variability in turnover differential, which you would expect if luck was a mega factor as my analysis suggests. While starting quarterbacks absolutely play a role in turnover differential (Tom Brady throws fewer picks than Chad Henne) and tend to be fairly constant from year-to-year, nevertheless the correlation between turnover differential last year and this year is only 0.086 which is not significant at the 5% level.

Ultimately in a 16-game season, there’s just a whole lot of luck involved with winning football games and for all that commentators will talk about defensive schemes forcing turnovers this season, it’s just as important to be lucky as to be good.

Nice work.

I’m surprised that 45% of the variance from turnovers is NOT luck. I’ve done some of my own research into turnovers and luck, and I only found completion percentage as the only possible predictor of turnovers (picks in this case).

http://thepowerrank.com/2014/01/31/how-to-predict-interceptions-in-the-nfl-backed-by-surprising-science/

It would be nice to dig up some data on whether teams that have more sacks tend to cause more fumbles, as Brett Thiessen found in college football.

http://mgoblog.com/content/maximizing-your-fumble-luck

What else can we uncover of this 45%?

Great analysis and right on. What would be interesting is to look at this by TYPE of turnover. Are fumbles more random than interceptions (likely so)?

A little bit of ‘effect and cause’ here. Turnovers lead to losses, but losing – especially late – also causes turnovers, as teams try desperate things in order to erase a deficit before the final gun sounds.

Good work!

Two potential follow-ups:

1. What if you broke turnovers down into interceptions and fumbles lost? Intuitively, interceptions and total fumbles (lost + recovered) seem predominantly skill-based, whereas fumbles recovered depends on the weird bounces a football takes.

2. If interceptions are largely skill-dependent, could you use the odds ratio (see http://angrystatistician.blogspot.com/2013/03/baseball-chess-psychology-and.html) to calculate the probability a given QB throws an interception against a given defense? Ex: Brees has a 2.7% int rate this year, and the Packers’ D has a 4.1% int rate this year, so in a league with an int rate of 2.3%, we’d expect Brees to have an int rate of 4.7% vs. the Packers; that is, he’d throw 1 interception every 21 passes.

Just found this on Google and it’s an excellent discussion. Ultimately the issue with turnovers is that in the majority of cases they happen because the offensive team has lost control of the ball. There are of course situations where a quarterback throws a good throw, a defensive player reads it and makes a brilliant play (Malcolm Butler is an obvious example against the Seahawks) but it’s more common for a turnover to occur from a fumble or an errant throw.

In both of those cases the offense isn’t controlling the ball so where it ends up is largely a matter of circumstance. You can say defenses that are on their toes and read things quicker can tip the odds in their favour, but nobody can make the quarterback throw it into an area where it could be caught and they certainly can’t control where the ball goes after a fumble (and therefore who recovers it).