By Andrew Cohen
As a devoted Harvard basketball fan, I was disappointed on Selection Sunday. By now, you have probably already seen the shot. Harvard had an NCAA tournament berth in its hands, and had it stripped by Princeton. My mourning period was interrupted an hour later when Harvard’s name entered the conversation for an at-large tournament berth. While it was obviously a long shot, I relentlessly investigated the tournament resumes of the other schools on this year’s bubble. For the most part, Harvard was stacked against high-profile power conference schools with decent records and tons of opportunities to play and defeat tournament-caliber teams.
Now I know that Harvard was never a serious contender for an at-large berth. But after watching ESPN College Game Day for 24 straight hours after the loss to Princeton, I can guarantee that Harvard received consideration by the selection committee. That is, the selection committee had to choose between a power conference team like USC, with a measly record of 19-14 but 10 wins this season against Pac-10 teams, and a mid-major team like Harvard with a 23-6 record, but given a chance to play only 5 tournament-ready teams. In this study, I attempt to bring a quantitative analysis to the mid-major/power conference debate for teams on the bubble.
March Madness is all about upsets. The underdogs are the lifeblood of the tournament. According to some (experts included), if it can be determined that a power conference and mid-major school are completely equal in tournament qualification, the berth should be awarded to the mid-major, the perennial collegiate underdog, the George Masons and the Davidsons of the world. After all, these schools are generally untested, haven’t been given the opportunity to prove themselves, and performed close to flawlessly throughout the season. But who actually performs better in the tournament: the mid-major or the power conference schools?
To assess whether power conference teams on the bubble outperform their mid-major counterparts, the dataset chosen included the “last four” at-large selections of every tournament field for the last ten years (2001-2010). Seeding was used to determine this last four because, prior to this year, the final four teams added to the tournament were not publicly disclosed. Any at-large selections holding the same seed as any of the last four were also included in the dataset. Over the ten years, these criteria included 29 power conference teams and 24 mid-major teams.
Tournament results were quantified by teams’ Performance Against Seed Expectations (PASE). As defined by Pete Tiernan of ESPN:
PASE measures the average number of wins a team attains above or below the number its seed position would dictate that it achieves. PASE is calculated by tallying the positive or negative differences between actual and expected wins at each seed position. The total of these differences is then divided by the number of appearances to arrive at an average number of games the team either over-performs or under-performs per tournament.
To further our understanding of the PASE metric, let’s look at an example from the data
set. The Arizona Wildcats of 2009 were seeded 12th and advanced to the sweet sixteen. The Wildcats won 2 tournament games that year, compared to average wins accumulated by a 12 seed of 0.48. Thus, their PASE was 2 – 0.48 = 1.52. To learn more about PASE, a short ESPN article can be found here.
The average power conference PASE was 0.166, and the average mid-major PASE was -0.053. A two sided unpaired t-test confirmed that this small difference in bubble performance between the power conference and mid-major schools was not statistically significant at the α = .1 level.
A slight outlier in this data is the Missouri 2002 team that went to the elite eight as a 12 seed. The only team in the data set to win three games, Missouri Tigers boasted a PASE of 2.52 that year. Led by current NBA journeyman Kareem Rush, the Tigers perhaps inflate the power conference PASE statistic to boost it slightly higher than the mid-major PASE. It is also worth noting that George Mason does not appear in this data set. While George Mason was selected as an at-large bid in 2006 when it made its Final Four run as an 11 seed, the last four at-large teams in 2006 were seeded 12th and 13th.
We can interpret this data to mean that power conference bubble teams tend not to outperform the mid-major squads. As discussed earlier, a constant debate among ESPN College Game Day analysts over the weekend was whether or not to give preferential treatment to underexposed mid-major teams on the bubble, if it could be determined that a given power conference and mid-major team had equal tournament resumes. While this TV debate gave Harvard fans false hope (and UAB and VCU fans real hope), history shows no indication of a difference in tournament performance. With the four team expansion in effect this season, the bubble was regarded as overpopulated with equally qualified teams. According to the predictions of ESPN’s Joe Lunardi, the Selection Committee contemplated doling out the last four spots among 11 teams. (The list is: Saint Mary’s, Clemson, Virginia Tech, USC, Alabama, Georgia, Boston College, UAB, Harvard, Missouri State, and VCU. Clemson, USC, UAB, and VCU received the last four bids). When forced to make this choice between 6 power conference schools and 5 mid-major schools, the Committee chose two of each. While this simple study cannot evaluate the correctness of the committee’s last four picks, it can conclude that the exclusion of a team based solely on conference status is unjust.
This has been an issue for a long time – while it’s politically correct to say that conference affiliation has no bearing on at-large selections, comparing resumes of potential at-large teams does seem to come down to conference affiliation. I would argue that they also tend to give BCS schools better seeds than non-BCS schools, all else being equal.
They also seem to put too much emphasis on the sheer volume of games played against the RPI top 50/100 which favors power conferences and penalizes non-BCS schools – in essence requiring a near perfect season to get an at-large berth.
The Coleman article does a great job of illustrating exactly what the selection committee is looking at and how those factors favor the BCS conferences:
It’s a shame but nothing will change until the non-BCS schools realize they have the power (outnumbering the power schools by 4-1) and stop getting pushed around.
I guess I would say the conclusion of the study is that the committee does a decent job of not penalizing a team for being either from one of the major conferences or not, at least by one metric.
I’m also a bit mystified by your exclusion of any discussion of losses. The losses to Yale and Princeton do a lot to offset the wins against Boston College and Colorado; USC had a home loss to Oregon, but a road win at Washington to end the regular season. If Harvard had been better than 2-3 against the top teams, that would have helped, but so would have been a better record than 21-3 against the rest of the schedule.
Not, of course, to say that Harvard isn’t better than USC. I would have picked Harvard. But Harvard’s schedule in and of itself didn’t hurt them as much as the fact that they didn’t do quite as well with it as they had to. (Of course, if they didn’t have that loss to Yale, they wouldn’t have had an at-large bid because they would have had an automatic bid. Supposing Princeton didn’t have its loss to Brown and 13-1 Princeton beat 13-1 Harvard 63-62, I think Harvard would have been invited.)
“It’s a shame but nothing will change until the non-BCS schools realize they have the power (outnumbering the power schools by 4-1) and stop getting pushed around.”
The non-BCS schools have the power to get the BCS schools to withdraw from the NCAA and set up their own organization.
I would argue that if it appears that BCS are not worse than non-BCS, it means the committee is doing an efficient job. If non-bcs schools outperformed the BCS, they would have a case that they were underseeded. Since they aren’t, it means that they are either being overrated themselves or both groups are treated fairly.
Ken’s interpretation is the correct one.
When doing studies like these, you are actually looking for bias in the selection process. If the mid-majors were statistically out-performing their major conference bretheren, this would be evidence that the selection committee was over-valuing conference affiliation- giving a school such as Harvard a stronger argument that it should have been selected. Since this is not the case (moreover the point estimate indicates the opposite trend, even if not statistically significant), this piece of the selection process appears to be working (ie. the 12-16 seeds win about as often as they should, regardless of conference affiliation). This would indicate that bubble teams, whether they are selected or not, really have nothing to complain about. There is no case here that mid-major teams are not selected often enough on the bubble.
However, VCU aside, most bubble teams have little impact on the tournament as a whole. The more interesting analysis would be to see if automatic bid mid-majors systematically outperform their seeds. As a top seeding can lead to a significantly easier path to the final, teams who win mid-majors and therefore may have a claim to these seeds are the ones who should be concerned.