By Kurt Bullard
If you were to construct a Mount Rushmore of quintessential sports debates, the existence of the hot hand may make its way up there, especially in the basketball world. Proponents point to performances like Klay Thompson’s 37point third quarter in January against the Kings, former HSAC copresident John Ezekowitz’ paper showing that a small hot hand factor exists, or that one time they couldn’t miss playing pickup basketball. Detractors point to the lack of existence of momentum in sports in general, as well as Ezekowitz’ first paper on the subject which he has since revised.
But people outside of the basketball world have also started to embrace the debate. At the Sloan Sports Conference this year, Brett Green and Jeffrey Zwiebel tested whether or not the “hot hand” existed in different facets of baseball using panel data, finding that it existed for batters to some extent in the MLB.
I thought I’d extend the principle to see if there’s any type of phenomenon in football, specifically for quarterbacks.
My strategy was to emulate John Ezekowitz’ paper from 2014, doing as best as I could to translate his paper from basketball to football. John had access to much more data than I do, since football spatial data tends to be under lock and key. Simple playbyplay data does not capture how far a pass went in the air; for example, a tenyard screen pass is counted the same as a tenyard out to the sideline. However, I do have the air yards data for Peyton Manning from 2006 through Week 8 of the 2015 season, meaning that I know how far the pass went in the air as opposed to where the play ended. So, since I can’t perform an analysis on all QBs, I will instead perform a case study focusing on Peyton Manning only.
The case study will aim to answer two questions:

Did Peyton believe that he had the hot hand? (i.e. did Peyton throw more difficult passes once he was “hot”?)

Did Peyton experience the hot hand? (i.e. did Peyton throw better when he was hot?)
The methodology for measuring heat was derived from Ezekowitz’ paper from 2014, in which he first uses multiple regression to calculate a player’s “complex heat,” which he measures as the following
Complex Heat = Actual % of last n shots – Expected % of last n shots
This way, Peyton isn’t penalized as being “cold” for taking dangerous, low probability shots down the field, nor is deemed “hot” for completing easy dumpoffs. Rather, Manning is judged for he has done recently conditional on the difficulty of the passes. For this analysis, I kept the number of passes (n) equal to 4, which is what John’s paper settled on.
I then ran a regression to calculate a player’s marginal heat gained or lost on each pass. I looked at the following variables
Variables of Interest
Air Yards
No. Pass of Game
Lateral Position of Pass
DVOA
Out of those four variables, only lateral position of the pass was not significant in predicting the completion likelihood of a pass.
Then, I calculated all of the residuals and then summed up the marginal heat of the previous four passes to find a player’s cumulative heat entering a pass. Then, with the complex heat calculated, I could then test the two aforementioned questions.
The first requires a simple regression of the difficulty of the pass against the cumulative heat. Since heat takes into account the last four passes that one completed, the first four passes of the game were thrown from the analysis since you could not properly calculate heat.
The heat coefficient is positive here, which implies that the hotter someone is, the less likely it is that they try a difficult pass, since the pass has a higher likelihood of being completed. This result is somewhat counterintuitive, but one possible explanation could be that if someone is hot, the drive is going well and they do not need to force things down the field. Nonetheless, a player’s heat is not predictive of the next pass’ difficulty, with a adjusted Rsquared of less than 0.1%.
The second question—whether or not Peyton actually got hot—requires simply inserting the heat term into the first regression we did to predict completion likelihood, and seeing whether or not the heat term is significant.
The heat term is significant, but it’s actually negative, meaning that Peyton was actually less likely to complete a pass if he had a higher complex heat. This “negative” heat most likely is just regression to the mean, but nonetheless, it shows that there’s no evidence here that Peyton had any symptoms of a hothand during his career.
There are obvious improvements to this type of study, but are tough to do without sufficient, advanced tracking data. For one, other QBs are crucial in seeing whether or not a hot arm exists on average. You also can’t discern coverage or pass rush from this data, which is critical in determining the likelihood of completions. You also can’t tell how far back the QB is from the LOS, which could also add to the difficulty of the pass.
Nevertheless, it doesn’t look like the hot “arm” was much a thing for Peyton. But with two Super Bowl wins and a firstballot HoF resume, it’s not like he needed it.
Kurt,
Pretty good article. I do have two questions though.
How well did your variables predict completion likelihood? It seems to me more variables could be added, provided the data is available, to better predict how difficult a pass attempt is. While coverage data is probably not out there for you, pass defense probably plays a role, as do things like the receiver to whom the ball is thrown.
Did you try changing up your value for n? I’m not familiar with how you are coding all of this, but I think it would be pretty easy to calculate complex heat for many values of n and rerun your regressions. It may be that some effect exists but does not show up with your specific n.
I also believe that the negative coefficient on heat should not represent “regression to the mean”, as in that case the coefficient would be 0 if I understand this correctly, just as flipping a coin and getting heads 4 times in a row does not make it less likely to get tails on the fifth flip. I could be wrong but that is what my intuition tells me.