This weekend, Southampton make the journey to Wembley to take on Chelsea in the FA Cup Semifinal. The semifinal is an opportunity for over 33,000 Saints fans to watch their team in the national stadium and dream of lifting the oldest domestic cup competition in world football. However, Southampton are also deep in the midst of a relegation battle, sitting in 18th place and 5 points from safety with 5 matches to play.
Some fans might argue that this FA Cup run distracted Southampton from the more important task of Premier League survival while others would argue that it’s a welcome distraction and an opportunity for a team to gain form. Fans of teams that exit the FA Cup early often shrug it off and focus on the potential benefits of an early FA cup exit. There are isolated examples of teams using an FA Cup run to help improve league form but also examples of teams that have fallen apart in the league while chasing FA Cup glory. Last season, Millwall used their run to the Quarterfinal of the FA Cup (defeating 3 Premier League teams en route) as a catapult to promotion out of League One. On the other hand, in 2013 Wigan Athletic were relegated from the Premier League 4 days after defeating Manchester City to win the FA Cup. More recently, in 2016 Crystal Palace reached the final of the FA Cup, but won only 2 league matches in the second half of the season, falling from 5th to 16th.
But how much truth is there in these claims? Do teams that make deep runs in the FA Cup improve their league form, or does the wear and tear of playing in multiple competitions catch up to them (and their league form suffers as a result?). In 2015, the Guardian ran a study investigating this, but only used finalists from the previous 5 seasons (10 teams total), which is a very small sample size. I wanted to expand upon this study and test to see if there was statistical significance behind these theories. I started by identifying teams in the 2005/6 to 2016/17 FA Cup that went much further in the competition than they would have otherwise been expected to based on their league status and position. I did this in the following manner:
1.) Teams that reached at least the Final and finished outside the top 6 of the Premier League.
2.) Teams that reached at least the Semifinal and finished outside the top 12 of the Premier League.
3.) Teams that reached at least the Quarterfinal and played that season in the Football League Championship.
4.) Teams that reached at least the 5th Round and played that season in either Football League One or Football League Two.
Note: There were a few teams that qualified for the 5th Round or beyond while playing in the 5th tier National League (like Lincoln City and Sutton United last season), their results were not included in this analysis.
For example, in the 2015/16 season the teams we identified as having longer than expected cup runs were Crystal Palace (who reached the Final), Watford (Semifinal), Reading (Quarterfinal) and Shrewsbury Town (5th Round). Overall, there were 45 teams that met this criteria during the 2005/6 to 2016/17 seasons. We then tested to see if these teams league form was affected by the FA Cup run as follows:
1.) We selected all league matches starting from 5 days before the 3rd Round to 5 days after the team was eliminated from the competition and calculated the average points per match during that stretch. The 5 days was chosen because team selections for league matches immediately before and after important FA Cup ties might be impacted by the cup run. Despite the fact that teams in League One and League Two enter in the 1st round (early November), the 3rd round was chosen because that is when the rounds start to occur more frequently (once every 3 weeks as opposed to once every 5).
2.) We then calculated the average points per match in all other matches during that season, and subtracted it from our result in part 1. We then averaged all of these differences to create our test statistic.
3.) We then selected a random sample of matches of the same number as the number of matches in step 1 that did not overlap with the matches during the FA Cup run and calculated the average points per match during that random sample. We also calculated the average points per match for all games outside that random sample and subtracted the difference for all teams.
4.) We repeated Step 3 10,000 times to create a null distribution.
5.) We compared our result from step 2 with the results of the 10,000 simulations.
We found that on average, teams that made deep runs in the FA Cup attained .0185 points per match fewer than in matches outside of that run. When we compare this to the results of the 10,000 simulated test statistics, we get the following plot:
We find that of the 10,000 simulations run, 22.2% of them have averages that are less than our test statistic of .0185. What this means is that we do not have enough evidence to conclude that teams who make deep runs in the FA Cup have their league form impacted.
This result lies contrary to conventional wisdom of many Premier League managers who choose to rotate their squads heavily in the early rounds to rest players for important Premier League matches. If there is no tangible downside to making a deep run in the FA Cup (and increase your chances of winning a rare trophy), why not put out stronger teams and go all out to win the competition?
If you have any questions for Andrew, please feel free to reach out to him on Twitter @andrew_puopolo or by email at firstname.lastname@example.org.