Undeserving Champions: Examining Variance in the Postseason

By Julian Ryan and Barrett Hansen

To this unbiased observer, European soccer has unequivocally the fairest system when it comes to deciding a league champion: by the end of the season, each team has played all the others both at home and away. This means all teams play the exact same schedule, and thus whoever wins the most games by definition deserved to win the league. While there is an inherent element of luck to winning an individual soccer match, a team will never be accused of sneaking a championship because it must have earned its victory (unless it pays referees, but that’s really just Italy).

In the four major US sports leagues though, a regular season is supplemented with a postseason to determine the final champion. As once elucidated by Jim Mora, playoffs are an integral part of American sports culture and regularly attract some of the largest TV audiences in the world. This system is in place for the sake of both practicality—can you imagine a round-robin NFL season?—and the exhilaration fans gain from its “win or go home” aspect.

However, even the most ardent playoff advocate will admit that sometimes this system results in an undeserving winner (most explicitly, the 2007 New York Giants). Because of the variance associated with any individual sports game, the shortened postseason creates opportunities for teams with worse talent to pull off a string of upsets through a mixture of form and often luck (David Tyree) and be crowned champions. The question we wanted to address is, of the four leagues, which most consistently produces the least deserving champions.

To tackle this, we looked at past seasons dating back to 1995, when the MLB first implemented the eight-team playoff system, and thus when all the four leagues were approximately using the system in use today. The NBA has made some small alterations to their seeding rules, and the NFL has expanded to four divisions per conference, but each have kept the same number of playoff teams. The MLB in 2012 added the one-game playoff between the two wild cards, a small change that may have slightly aided the Giants’ World Series run last year but in total has been a minor enough change for us to include 2012 in our analysis.

For each season in each league, we considered the Pythagorean win expectation for each team to give an approximation of their regular season performance and then calculated the rank and z-score of each postseason champion’s regular season performance. So, for example, the 2013 Chicago Blackhawks had a Pythagorean winning percentage of .698, which was the best in the league and gave them a z-score of 2.26, implying that they were over two standard deviations better than the league average during the regular season. However, the Pythagorean metric does not take into account the teams’ regular season strength of schedule; consequently, we also analyzed the champions’ “simple rating.” The Simple Rating System (SRS)—devised by sports-reference.com—aims to provide a figure of how many runs/goals/points a team differs compared to league average for the season, while explicitly taking into account a team’s strength of schedule. It can be considered to be a proxy for the point spread between two teams if they played at a neutral site (and were playing at their season-average level). So when the 2007 Patriots (SRS of +20.1) played the 2007 Giants (SRS of +3.3) in the Super Bowl, had they both played at their average level of regular season performance, the Patriots would have been expected to win by about 16.8 points. Computing the SRS rank and z-score for champions as well, we found the following results:

Pythagorean Expectation

Simple Rating System

League Champions from 1995-2013

Mean Rank

Median Rank

Mean Z-Score

Mean Rank

Median Rank

Mean  Z-Score

NFL

5.3

4.5

1.34

4.7

2.5

1.37

MLB

5.8

4

1.10

6.5

7

1.05

NBA

3.3

2

1.47

3.3

2

1.48

NHL

4.0

2.5

1.34

4.2

3

1.27

 

As we expected, the NBA seems to produce postseason champions most aligned with regular season performance. The long format of the seven-game series and reduced likelihood of upsets on account of the high number of possessions have resulted in the best regular season team winning eight of the nineteen possible championships.

Vividly remembering the LA Kings’ and Baltimore Ravens’ unlikely triumphs, we naively assumed that the NFL and the NHL would battle it out for last spot. However, our analysis revealed that the MLB most consistently produces champions most disparate from their regular season performance. What is remarkable is just how bad the MLB playoffs really are. Owing to the length of its 162 game season, one might think that regular season performance would actually be a fairly good indicator – better, for example, than in the NFL where strength of schedule can have a huge impact – of the overall quality of a team. Given this fairly reasonable assumption, if you chose the eight best regular season teams, or even eight of the top ten because you require four from each league, and then just asked each team to draw straws to determine the World Series, the average winning team would be better determined than by the current system. Only three times has the best team from the regular season ended up winning the World Series.

The last 18 years of sports have produced some truly fantastic teams, but there have also been some complete duds, and we would like to take this opportunity to individually call out the least-deserving champion of each of the four major sports. In the NBA, the honors unquestionably go to the 1995 Houston Rockets, who were barely half a standard deviation above the league average and were the eleventh-best regular season team. The NHL has a few out of the blue champions, and honorable mention should go to 1995 Devils, 2007 Hurricanes and 2009 Penguins, but all things considered, no one can touch the 2012 Kings. As the eleventh-best team of the regular season and proud holders of an eight seed, they went on a statistically improbable away-game winning streak, only losing one game outside Los Angeles en route to their Stanley Cup triumph. The craziness of that postseason was only amplified by the ninth-best and sixth-seeded New Jersey Devils making the finals as the Eastern Conference champs.

When it comes to the NFL, no one steals Super Bowls like Eli Manning. The 2007 Giants were playing the team with the highest z-score for both Pythagorean expectation and SRS of any champion in our dataset across all four leagues and still managed to win as the eleventh-best team (by SRS, thirteenth by Pythagorean). However, that team is second on the list to the other Eli Manning triumph. The 2011 Giants, who scraped into the playoffs at 9-7, remarkably actually scored fewer points than their opponents in the regular season and were thus a below average regular season team before tearing through the playoffs behind a dominant defensive line, Eli “somehow-more-rings-than-his-brother” Manning and another ridiculous catch.

Last but never least, the MLB can lay claim to the least deserving postseason winner. Noteworthy runners-up include the 2012 Giants (apparently a solid franchise name for sneaking championships) who pitched their way to victory and the 2003 Marlins, who rallied to victory from the wildcard spot behind Dontrelle “ D-Train” Willis and his unorthodox motion, but neither can touch the least meritocratic winner sports might ever see: the 2006 St Louis Cardinals. The cardinals were shockingly the twenty-first ranked team by SRS, over half a standard deviation below the league average, roughly equivalent to the 2012 Arizona Cardinals of the NFL. They needed a loss from the Astros in the final game of the season to barely slip into the playoffs with the third-worst record for a postseason team in MLB history at 83-78, coming off a September when they went 12-17. However, for some reason the gods were smiling down on St Louis as they pulled off three huge upsets—including an epic game seven against the Mets in the NLCS—to win it all.

So as we move into October and the time of the year when everyone talks about how clutch various baseball players are, never underestimate the power of the MLB to produce some truly appalling victors. But hey, isn’t the craziness why we watch in the first place?

(For another look at measuring variance across the four major sports, check out this piece from Tangotiger.)

2 thoughts on “Undeserving Champions: Examining Variance in the Postseason

Leave a Reply