By Harrison Chase and Kurt Bullard
Last year, we wrote various posts on creating a model to predict which NFL coaches would be fired at the end of the season. So, with nothing to do but wait for the regular season to begin, we decided to renew our predictions for the upcoming season to see which coaches may be spending their last seasons in their respective cities.
For those of you who aren’t familiar with our model, we’ll go through our methods in building this model before delving into the results.
Data:
We used available data after the last work stoppage in the NFL in 1983 in order to train our model. For this year’s rendition, we added in last season’s results to better refine our model. Seven teams changed leadership this offseason. We marked down five of them as having fired their coaches (Falcons, Bears, Jets, Bills, Raiders); we decided not to report Jim Harbaugh or John Fox as having been canned, given reports that the two mutually decided to part ways with their respective organizations.
Variable Selection:
Last year, we hypothesized that copious variables may affect the chances a coach gets fired by the end of the year. Despite the fact that we felt good about our inputs, we worried that we might be overfitting a bit, and so we decided to use several methods of cross-validation to evaluate our model. After testing various models using leave-one-out and 10-fold cross-validation methods, we found nine variables to be explanatory of firing odds:
· Season winning percentage
· The difference between current season winning percentage and the winning percentage by team the year before the coach was hired
· How often the coach has made the Divisional Round with the team
· Strength of Schedule (per SRS from Pro Football Reference)
· How often the coach has won the Super Bowl
· Whether a new GM has taken over the organization
· Separate dummy variables for whether a coach is in his first year, second year or, third year
All of the variables make intuitive sense, as they reflect absolute performance, relative performance compared to expectations, track records of success, and how much of the owner’s goodwill the coach has used up.
Projecting Performance:
The problem with predicting the odds of coach turnover before the season is that you have to predict a team’s performance for the upcoming season. Two of the variables in the model – winning percentage and change in winning percentage – are determined by team strength; therefore, the predictions of the model largely depend on how good you think each team will be. But simply put, predicting the NFL season is an incredibly hard task. Trust us, we’ve tried.
Since this post is not about predicting the season, but rather testing our coaching turnover model, we’ll use two different methods to come up with an estimated ranking (expressed in terms of SRS) for each team to hedge the uncertainty of the season. After we determine these teams’ rankings, we can use them to compute win probability for each game by running a logistic regression so that we can simulate the season.
The first method we used to come up with each team’s SRS was to regress last year’s value back to the mean. To determine how much to regress each team’s SRS, we regressed a team’s SRS from year T + 1 against the team’s SRS in year T to find a correlation coefficient. We found on average that teams regress more than halfway to the mean (SRS = 0.46*PRIOR_SRS), showing how fleeting success is in the NFL.
This is a conservative way of estimating team strength, but that doesn’t mean it is flawless. Regression to the mean underestimates variance in team skill and also doesn’t take into account roster turnover year-to-year, with teams like the 49ers – who experienced a mass exodus of talent – benefiting the most from this system.
The second method we used was simply mapping ESPN’s power rankings to an SRS. We amassed all of the SRS rankings for each team since 1983 and then assigned the 95th-percentile value (9.88) of these values to ESPN’s top team – the Seahawks – and assigned the 5th-percentile value to the Titans, ESPN’s worst team. For the other teams, we mapped out SRS evenly to fill in the spread. Obviously, these rankings aren’t perfect either, since this is a subjective measure and the makers are prone to their own biases. We realize that not everyone will agree with ESPN’s rankings – even we have our complaints – but it’s tough to find a preseason ranking list that not one person has a problem with.
Results:
Below are the estimated probabilities for each team changing its head coach at or before the end of the season. These were produced by simulating the season 1,000 times and taking the average of the expected fired percentage at the end of each of the seasons.
Team |
Mean-Regressed Probabilities |
Power Ranking Probabilities |
Arizona Cardinals |
16% |
7% |
Atlanta Falcons |
7% |
7% |
Baltimore Ravens |
2% |
1% |
Buffalo Bills |
5% |
9% |
Carolina Panthers |
18% |
13% |
Chicago Bears |
10% |
11% |
Cincinnati Bengals |
24% |
14% |
Cleveland Browns |
16% |
33% |
Dallas Cowboys |
16% |
9% |
Denver Broncos |
4% |
3% |
Detroit Lions |
13% |
11% |
Green Bay Packers |
2% |
1% |
Houston Texans |
5% |
7% |
Indianapolis Colts |
5% |
1% |
Jacksonville Jaguars |
32% |
43% |
Kansas City Chiefs |
7% |
7% |
Miami Dolphins |
40% |
42% |
Minnesota Vikings |
16% |
26% |
New England Patriots |
1% |
0% |
New Orleans Saints |
23% |
21% |
New York Giants |
16% |
23% |
New York Jets |
6% |
11% |
Oakland Raiders |
9% |
13% |
Philadelphia Eagles |
10% |
8% |
Pittsburgh Steelers |
20% |
12% |
San Diego Chargers |
9% |
8% |
San Francisco 49ers |
8% |
11% |
Seattle Seahawks |
1% |
0% |
St. Louis Rams |
35% |
45% |
Tampa Bay Buccaneers |
26% |
46% |
Tennessee Titans |
42% |
65% |
Washington Redskins |
28% |
26% |
As we mentioned before and as is apparent here, the projections using the ESPN Power Rankings are more extreme because regression underestimates variance in team performance. Nonetheless, we can compare our results to Vegas odds for who will be the first coach fired – something we believe will correlate highly with odds of being fired in general.
-
The coach sitting on the hottest seat according to both the mean-regressed and power ranking projections is Ken Whisenhunt, who BetOnline gives the second highest odds to be the first coach canned. Despite it only being his second year on the job, the Titans seem destined to be at the bottom of the league only two years after they posted a respectable 7-9 record with Mike Munchak.
-
Jay Gruden of the Redskins, Mike Pettine of the Browns, and Lovie Smith of the Buccaneers are three other second-year coaches to whom our model attributes a high chance of being fired, primarily due to the fact their teams are projected to do very poorly – 27th, 28th and 31st by ESPN rankings respectively. BetOnline gives these coaches, respectively, the 1st, 4th and 7th greatest chances of getting fired, with Gruden perhaps getting a boost due to being part of the dysfunctional Redskins organization.
-
Joe Philbin faces relatively high odds of being fired, as he is no longer in the three-year grace period usually given to head coaches. He has been perfectly middling as head coach of the Dolphins, with a 23-25 record over those three years. Despite a recent one-year contract extension, he has to be feeling pressure to at least make the playoffs. Philbin faces a set of lofty expectations – some of which are really high. The Dolphins have made big splashes in free agency over the past two years, so one more year of falling short of the Patriots may be the last straw for Joe.
-
Both of our models give Gus Bradley the fourth-highest odds of being fired, one spot higher than BetOnline. Despite the fact that the Jaguars have won a combined seven games over his two-year tenure and don’t seemed poised for a breakout year, Jacksonville has ostensibly been in a rebuilding project even in the days before Bradley, which lowers his expectations significantly.
-
One coach that our model thinks has a greater chance of getting fired than the betting markets do is Jeff Fisher of the Rams. Despite only being given the 8th best odds from BetOnline, our model pegs him as the third most likely by both SRS projections. He too, like Philbin, is entering his fourth year without a winning season or playoff appearance. The fact that he plays in one of the toughest divisions can’t be forgotten, but his seat should be getting warm also.
While we’ll be able to refine our model once the season starts and not have to rely on questionable ranking procedures to predict the season, these odds are the best we can do now. Hopefully these coaches enjoy the last month of comfortable sitting before some come face-to-face with the hot seat.
***Here is a link to that data we used. Although the variables are poorly named, most of the main ones should be clear (and you can also create new ones if you aren’t sure what certain ones are). We hope that you will take a look at this data and see if you can find any way we can improve our model. Feel free to either get in touch with us by shooting us an email or following HSAC on Twitter.
2 Comments