How Much Does NFL Seeding Really Matter?

By David Roher

This post also appears on Deadspin.

The sixth-seeded Packers are in the Super Bowl, making them the first team since the ’05 Steelers to accomplish the feat. And they’re not the only squad to outperform their slot β€” over half of the playoff games thus far have gone to the road team. That has pundits questioning not only the wisdom of the NFL’s playoff structure, but also whether seeding itself matters all that much. We can answer the latter, the exact value of an NFL playoff seed.

The process is a little more complex than it might first seem, as we can’t just look at historical playoff results. They’re biased, since the better seed usually belongs to the better team. Instead, we have to be more abstract: what exactly makes getting a top spot valuable? Three things: byes, home-field advantage, and strength of schedule. The top team in each conference not only gets a guaranteed second-round berth, but also a guarantee that they will face, at home, the worst-ranked teams remaining in the next two games.

To start determining the effect of strength of schedule, I looked at the average quality of each AFC and NFC seed since 2002 (the year after realignment). Winning percentage is a decent enough statistic to measure over a nine-year period, but it inflates the quality of the teams with the better records: while “peaking at the right time” is an overblown concept, a team’s strength at the start of the playoffs can be much different than it was in the middle of the regular season. My alternative is Weighted DVOA, a Football Outsiders stat. DVOA stands for Defense-adjusted Value Over Average – it measures the team’s success on each play compared to what an average team would have done in that situation. The weighted version more heavily considers recent games, which provides us with a picture of a team’s quality right as they enter the playoffs.

After scaling the stat back to winning percentage, we have an initial step in determining seed value that’s pretty interesting in its own right:

How Much Does NFL Seeding Really Matter?

The AFC’s supremacy here is striking, as is the jump in quality from the fourth to the fifth seeds. It doesn’t show up here, but there is an especially strong year-to-year change in quality in the wild card seeds. Sometimes they were the best team in the league, while other times they were the Seahawks.

The next step is to imagine (guys, this won’t work unless you all imagine) a hypothetical playoff bracket in which each team has the corresponding winning percentage in the above graph, and that no team has a matchup advantage over any other. Home-field advantage is a constant 57% (meaning that the home team is about 1.33 times more likely to win than they otherwise would be). As I said above, simulating this bracket straight through isn’t enough – the better seeds will win more often just because they’re better teams. The solution is to compare the results of a “correctly seeded” bracket to a randomly seeded one. To find the value of the first seed, I took the difference between the chance that the top “correct” team made it to the Super Bowl when it was seeded conventionally, and the average chance that it made the Super Bowl in the 720 total ways we could rearrange the seedings. I then repeated the process for the other seeds and for each conference. Here are the results:

How Much Does NFL Seeding Really Matter?

The vertical axis shows how the teams’ odds are affected by their seeding relative to the average – take, for example, the team with a DVOA winning percentage of 73%. If we could in some way eliminate seeding, this team would only have a 27% chance of making the Super Bowl. But as the AFC’s top seed, they have a 47% chance of making the Super Bowl. That means they’re 2.41 times more likely to make it. By comparison, the bottom seed is .33 times more likely (or three times less likely). This is a huge difference: the team in the top AFC spot, when throwing team quality out the window, is over seven times more likely to reach the Super Bowl than the bottom seed.

So what accounts for the difference? You might think, given the sharp drop after the second seed, that it’s the bye game. Indeed, that’s the biggest part, and this method doesn’t even consider the increased team strength that results from a week off. But it’s not everything – if it were, i.e. no home-field advantage or increased opponent strength, then the top seed would only be four times more likely to make the Super Bowl than the bottom team. This means that home-field and schedule strength nearly double the top-bottom gap by themselves. Getting to play a weaker team, in your building, is a big deal.

In theory, the NFL playoff system stands in sharp contrast to its regular season: after sixteen games of the “Any Given Sunday” crap, the teams on top of the pile get their richly deserved award. It’d be the exact opposite of baseball, where the regular season is predictable but all hell breaks loose in the playoffs. But this isn’t really the case. Trying to make a hierarchy out of the NFL’s regular season is kind of like putting a broken glass sculpture back together. The “top” teams might get an advantage, but the randomness of the regular season often means that those squads aren’t actually the best. So why have the structure at all? There isn’t much of an alternative; random seeding isn’t exactly fair either. One nice option might involve increasing the playoff teams to eight per conference, eliminating the bye. More below-average teams would get in, but the resulting system, one without byes, might actually be fairer. It might even be a compromise in the current 18-game-season debate. And who wouldn’t want four more games of playoff football?

The sixth-seeded Packers are in the Super Bowl, making them the first team since the ’05 Steelers to accomplish the feat. And they’re not the only squad to outperform their slot β€” over half of the playoff games thus far have gone to the road team. That has pundits questioning not only the wisdom of the NFL’s playoff structure, but also whether seeding itself matters all that much. We can answer the latter, the exact value of an NFL playoff seed.

The process is a little more complex than it might first seem, as we can’t just look at historical playoff results. They’re biased, since the better seed usually belongs to the better team. Instead, we have to be more abstract: what exactly makes getting a top spot valuable? Three things: byes, home-field advantage, and strength of schedule. The top team in each conference not only gets a guaranteed second-round berth, but also a guarantee that they will face, at home, the worst-ranked teams remaining in the next two games.

To start determining the effect of strength of schedule, I looked at the average quality of each AFC and NFC seed since 2002 (the year after realignment). Winning percentage is a decent enough statistic to measure over a nine-year period, but it inflates the quality of the teams with the better records: while “peaking at the right time” is an overblown concept, a team’s strength at the start of the playoffs can be much different than it was in the middle of the regular season. My alternative is Weighted DVOA, a Football Outsiders stat. DVOA stands for Defense-adjusted Value Over Average – it measures the team’s success on each play compared to what an average team would have done in that situation. The weighted version more heavily considers recent games, which provides us with a picture of a team’s quality right as they enter the playoffs.

After scaling the stat back to winning percentage, we have an initial step in determining seed value that’s pretty interesting in its own right:

How Much Does NFL Seeding Really Matter?

The AFC’s supremacy here is striking, as is the jump in quality from the fourth to the fifth seeds. It doesn’t show up here, but there is an especially strong year-to-year change in quality in the wild card seeds. Sometimes they were the best team in the league, while other times they were the Seahawks.

The next step is to imagine (guys, this won’t work unless you all imagine) a hypothetical playoff bracket in which each team has the corresponding winning percentage in the above graph, and that no team has a matchup advantage over any other. Home-field advantage is a constant 57% (meaning that the home team is about 1.33 times more likely to win than they otherwise would be). As I said above, simulating this bracket straight through isn’t enough – the better seeds will win more often just because they’re better teams. The solution is to compare the results of a “correctly seeded” bracket to a randomly seeded one. To find the value of the first seed, I took the difference between the chance that the top “correct” team made it to the Super Bowl when it was seeded conventionally, and the average chance that it made the Super Bowl in the 720 total ways we could rearrange the seedings. I then repeated the process for the other seeds and for each conference. Here are the results:

How Much Does NFL Seeding Really Matter?

The vertical axis shows how the teams’ odds are affected by their seeding relative to the average – take, for example, the team with a DVOA winning percentage of 73%. If we could in some way eliminate seeding, this team would only have a 27% chance of making the Super Bowl. But as the AFC’s top seed, they have a 47% chance of making the Super Bowl. That means they’re 2.41 times more likely to make it. By comparison, the bottom seed is .33 times more likely (or three times less likely). This is a huge difference: the team in the top AFC spot, when throwing team quality out the window, is over seven times more likely to reach the Super Bowl than the bottom seed.

So what accounts for the difference? You might think, given the sharp drop after the second seed, that it’s the bye game. Indeed, that’s the biggest part, and this method doesn’t even consider the increased team strength that results from a week off. But it’s not everything – if it were, i.e. no home-field advantage or increased opponent strength, then the top seed would only be four times more likely to make the Super Bowl than the bottom team. This means that home-field and schedule strength nearly double the top-bottom gap by themselves. Getting to play a weaker team, in your building, is a big deal.

In theory, the NFL playoff system stands in sharp contrast to its regular season: after sixteen games of the “Any Given Sunday” crap, the teams on top of the pile get their richly deserved award. It’d be the exact opposite of baseball, where the regular season is predictable but all hell breaks loose in the playoffs. But this isn’t really the case. Trying to make a hierarchy out of the NFL’s regular season is kind of like putting a broken glass sculpture back together. The “top” teams might get an advantage, but the randomness of the regular season often means that those squads aren’t actually the best. So why have the structure at all? There isn’t much of an alternative; random seeding isn’t exactly fair either. One nice option might involve increasing the playoff teams to eight per conference, eliminating the bye. More below-average teams would get in, but the resulting system, one without byes, might actually be fairer. It might even be a compromise in the current 18-game-season debate. And who wouldn’t want four more games of playoff football?

About the author

harvardsports

View all posts

3 Comments

  • Nice work, hopefully it starts some good debate. Two things related to your first figure. 1) The Seahawks weren’t a wildcard team, they won their division. 2) The jump from the 4 seed to 5 seed makes plenty of sense because the 4th seed is the weakest division winner (e.g. Seahawks), while the 5th seed is the best wildcard team. Often, the 5th seed has a better record than the 4th seed.

  • Thanks for reading, Dave. I realize that about the Seahawks, that was just a joke — it seems that whenever they make the playoffs, wild card or division winner, they’re a below average team. Also, very much with you on 4th-5th seed thing; that’s why I said “striking” and not “surprising.” That’s the way it is, but not the way it should be.

Leave a Reply

Your email address will not be published. Required fields are marked *