By John Acton
Some of the most controversial analysis of Deflategate has focused on the Patriots’ fumble rate since 2007 (the year that the current rules about football handling were implemented, after Tom Brady circulated a letter to other quarterbacks to petition the NFL to make the change). In particular, Warren Sharp has put forward the notion that the Patriots’ fumble rate has been suspiciously low, and that the players who had previously played for the Patriots had significantly higher fumble rates than they had when in a Patriots’ uniform. Sharp’s methodology has been criticized by a variety of sources for using both bad data and a statistically problematic methodology. That being said, the gist of the question Sharp is asking has validity. If the same players fumble less for the post-2007 Patriots than for other teams (or for the pre-2007 Patriots), it would raise questions about whether the Patriots were doing something that made fumbling less likely.
There are 10 players who had at least one rushing attempt or reception for the Patriots both before and after 2007, 45 players who had at least one rushing attempt or reception for the post-2007 Patriots and at least one other team, and 8 players who fall in both categories. I compiled the data for the number of combined rushing attempts and receptions (from here on called “touches”) for all of these players, categorizing them into touches with the pre-2007 Patriots, touches with the post-2007 Patriots, and touches with other teams.
Unfortunately, it is very difficult to get any statistically significant results based on this information, because fumbles are very rare among both good and bad players. As such, it takes a very large sample size for good and bad fumble rates to distinguish themselves at a 95% confidence interval. That being said, when comparing the 3 categories between these different players, the following statistically significant results are found:
- Kevin Faulk had a lower fumble rate post-2007 than pre-2007. (0/433 vs. 16/1001 p-value: 0.00804)
- Benjarvus Green-Ellis had a lower fumble rate with the post-2007 Patriots than with the Bengals. (0/588 vs. 5/545 p-value: 0.0198)
- Benjamin Watson had a lower fumble rate with the Browns and Saints than with the pre-2007 Patriots. (1/196 vs. 5/95 p-value: 0.00736)
This doesn’t show us very much. It’s worth noting that Faulk and Green-Ellis are the only two players to record more than 400 touches in more than one category, and both of them show up as statistically significant, going eye-raising amounts of time without a single fumble for the post-2007 Patriots. There are also some very suspicious-looking rates that still miss statistical significance, such as Tom Brady (9/262 vs. 17/277), Matt Cassel (3/77 vs. 17/166), and Danny Amendola (0/92 vs. 4/208). On the other hand, Benjamin Watson’s significant improvement with the Browns and Saints gives precedent for an individual’s fumble rate drastically changing over time for reasons that have nothing to do with the inflation of the ball.
However, in the aggregate, there is significantly more information to believe that the post-2007 Patriots are fundamentally different. The 10 players who played for the Patriots both before and after 2007 had 42 fumbles on 2,023 touches pre-2007. After 2007, these same 10 players had their fumble rate nearly cut in half, fumbling only 18 times on 1,729 touches. This drop has a p-value of 0.01174, and suggests that something changed to drastically cut these 10 players’ fumble rates.
The aggregate difference between players who played for the post-2007 Patriots and their performance with other teams is less pronounced, but still statistically significant. After 2007, these 45 players combined for 32 fumbles on 4,506 touches in a Patriots uniform. With other teams, these same players combined for 154 fumbles on 13,340 touches for a p-value of 0.01108.
There may not be enough data to clearly prove that the Patriots fumble rate is substantially different from the rest of the NFL’s, and it’s always possible that the Patriots changed some variable in their play-calling or personnel decisions around 2007 that created this positive impact. But the data clearly shows that in general, post-2007 Patriots players were not able to replicate their lack of fumbling in any other time or place in their career. This may not exonerate Sharp’s methods, but it does help his point: the data raises serious questions about whether or not the Patriots fumble rate has been kept artificially low.