Optimal Play Calling in the NFL: A Data-Driven Approach

By Bill Lotter

In case you haven’t heard, people have been scrutinizing a recent play call by Pete Carroll.  Don’t worry, I’m not going to talk about that.  But I thought it would be interesting to take a step back and look at play calling in general.  How should a coach go about choosing a play?  In the end, I hope to convince you that it’s one of the most interesting mathematical problems in sports.  And obviously play calling is one of the most important problems in football.  So pretty much, math is the most important thing in football.  Commutative property, right?

Let’s say its third down and your favorite team is somewhere between their own 20 yard line and midfield (meaning that they’ll punt if they don’t convert and we don’t need to worry about safeties).  And let’s simplify the play call to either a ‘generic’ pass or run.  Well, to start off, the average yards gained per pass attempt is greater than that for the run.  So should you always pass?  Putting aside the game theory misstep for a second, always passing isn’t optimal because you need to consider the full distribution of yards gained by run/pass and not just the mean.  For instance, you are more likely to gain at least one yard if you run than if you pass, which is useful to know on 3^rd and 1.

Below is the distribution of yards gained over the past six years, which I calculated using data from Pro Football Reference.  The curves show how likely it is that you gain exactly X yards by run and pass.

Below is a graph with the same data, just viewed a different way.  It shows the probability that you will gain more than X yards.  For example, there is about a 12% chance that a run will go longer than 10 yards versus 29% for a pass.

So as mentioned, if it’s 3rd and 1, there’s a higher chance of converting if you run.  But getting the first down isn’t the ultimate goal.  More than a first down, you want points.  And more than points, you want a win.

To simplify further analysis, let’s assume that we’re in the midst of a ‘normal’ game and we’re not near the end of a half. Thus, our goal is to just score points and not let our opponent do so.  In order to make decisions, we need to know the amount of ‘points’ a certain field position is worth.  If you’re right near the goal line on first down, your position is worth nearly 7 points because it’s highly likely that you score a touchdown (insert Pete Carroll joke).  Similarly, we can define a point value to each yard line.  This quantification is called the “Expected Points,” an idea developed by Brian Burke at Advanced Football Analytics, among others.  In the middle of the field, the Expected Points on a first down is approximately linear with a slope of about 1 point per 15 yards.  For every 15 yards you gain, on average, you will score 1 more point than your opponent.  As another point of reference, the Expected Points for having a first down on your opponent’s 35 is about 3 points.

With the quantification of field position, we can address our play call.  This decision obviously depends on our yards to go.  For both run and pass, we can estimate the chance that we would get a first down and then weight this by the Expected Points of being at this new field position.  Once again, we have to take into account the entire distribution. Even though we’re more likely to gain 1 yard running than passing, the overall chance of having a large pass play might still make it preferable to pass on 3rd and 1.

To assess the cost of not gaining a first down, we need to know how costly it will be to punt.  Let’s assume our net punt will be 37 yards, which is around the NFL average.  The cost of not converting, then, is just the Expected Points our opponent will have when they get the ball 37 yards down the field.

Ok.  Almost done.  What else do we need to consider?  That’s right, Mr. Football Talkinghead, TURNOVERS.  Turnovers are so costly because they ‘flip the field’ and could give the opponent great field position (meaning high Expected Points).  Even more, defensive players have been known to try to return fumbles/interceptions and get even more yards.  Crazy, right?  From 2008-2014, about 2.6% of passes were intercepted, 0.9% of pass plays resulted in a fumble, and 0.8% of run plays led to a fumble.  And using calculations from SportsQuant, the average net yards exchanged by an interception is -3 yards, and for a fumble it is -7 yards.  This means that the average interception will result in a field position for the opponent 3 yards better than the line of scrimmage on the interception play.

Now we have all our pieces together and just need to churn the numbers.  To quantify run vs. pass, we can just take the difference in the Expected Points for both options.  Without further ado, here are the results…

What does this mean?  For 3^rd and 1, we would expect to make ~0.57 more points overall if we decided to run vs. pass.  This might seem like a small number, but for one play-call on a random third down in midfield… you could see how things could add up quickly.  Overall, the shape of the graph isn’t very surprising:  run if it’s short, throw if it’s long.  But it’s interesting how much more beneficial a run is than a pass on third and short.  And the breakeven is about at 5 yards.  How many teams do you know that would run the ball on 3^rd and 5?

Of course, there are so many assumptions built into these calculations and it was meant more of a proof of concept than anything else.  In reality, you aren’t just picking run or pass, but which specific run play and which specific pass play.  And the distribution of yards gained isn’t only a function of the specific play you call, but also of the defensive play that is called.  Knowing that teams are likely to run on 3^rd and short, defenses will stack the line, which will make it much harder to gain this yard, making passing a more viable option.  For those familiar with game theory, we would expect convergence towards a ‘Nash equilibrium’, although it seems the NFL is far from it (see Advanced Football Analytics).  Also, the distribution of yards gained will depend strongly on your player personnel.  Your pass distribution will look a lot different if Peyton Manning is your quarterback than if you’re stuck with J.P. Losman.  (No offense J.P, you were just the first one to pop into my head, which is kinda weird…) Anyway, all of these considerations can be accounted for and estimated, leading to a most interesting, team-specific problem that needs the use of game theory, optimization, probability, machine learning… I could go on.  For how much a win is worth and how much a win is dependent on play-calling, you can only imagine the advantage that a rigorous, mathematical play calling scheme could add in addition to the intuitive, experiential knowledge that coaches already have.