1) Introduction

The NFL Draft has long been a subject of scrutiny for football fans and prospects alike. As a primary channel for player acquisition, draft behavior plays an essential role in roster construction and shapes the future composition of NFL franchises. While the selection process is largely secret, the outcomes can provide us some insight into the kinds of attributes teams value and aid us in evaluating the performance of different teams and general managers.

In 2005, Cade Massey and Richard Thaler published Overconfidence vs Market Efficiency in the National Football League, a seminal paper examining the decision making of NFL teams in the annual draft from 1988 to 2004. Massey and Thaler found significant evidence that NFL general managers fall prey to a number of biases in the player selection process, most notably overconfidence and the winner’s curse. Overconfidence, a term popularized by Kahneman and Tversky, refers to situations in which there exists a positive difference between a person’s subjective accuracy perception of their judgments and the true accuracy of those judgments. The winner’s curse is a phenomenon in auction theory suggesting that the winner of the auction is likely to end up paying more than the true asset value. Both of these biases are associated with one particular feature of team behavior over the years: overvaluation picks in early rounds of the draft and undervaluation picks in later rounds of the draft.

A likely explanation for these trends in pick valuation lies in a set of guidelines posited by former Dallas Cowboys head coach Jimmy Johnson. In the 1980s, Johnson developed a valuation system for draft picks based on previous trades, making explicit the values implied by prior transactions. This became known as “The Chart” and was quickly adopted by a number of teams as a general framework for constructing and assessing trade proposals. While teams adhered closely to this outline, The Chart was constructed as a descriptive tool, not a prescriptive one – that is, it was built to reflect the preferences that teams had demonstrated in the past, not necessarily optimal behavior.

Now, as teams become increasingly analytically driven, we could potentially expect biases to be addressed and behavioral tendencies to change. In a more recent edition of Massey and Thaler’s research, The Loser’s Curse: Decision Making and Market Efficiency in the National Football League Draft (2013), they find that the biases persist, but the past eight years have seen significant developments in team drafting approaches and attitudes toward analytics.

Even since the most recent Massey-Thaler paper, a number of other public analyses have been released, including FiveThirtyEight’s examination of team and GM consistency in beating the market (2014), Brian Burke’s update to the Massey-Thaler analysis in response to the new CBA (2016), and PFF’s exploration of surplus value by the PFF WAR metric (2020). We use a similar approach to derive pick value but expand on these analyses by digging deeper into behavioral tendencies and their associated results.

With variation in draft pick valuation as the backdrop, we attempt to formalize both macro- and micro-level team trends and outcomes. Each NFL franchise has a unique approach to constructing its roster via the draft, and in this report, we offer insights into what those strategies look like and how successful they have – or haven’t – been in the last decade. We include sections on general drafting success, player performance over/under expectation, high- vs. low-variance approaches, positional capital allocation, pick trading tendencies, and much more. The backdrop of these analyses is an expected pick value model we built using Pro Football Reference’s Approximate Value metric, which is defined by Pro Football Reference as “an attempt to put a single number on the seasonal value of a player at any position from any year.” We hope you enjoy it.

2) Methodology

In order to generate a model of predicted draft pick value, we use two measures of value, both based on Pro Football Reference’s Approximate Value (AV): AV per season since draft (regardless of whether they played in a given year), and AV per game played. We split into two definitions of value because the former takes longevity into account by dividing total career production by number of potential seasons, and the latter looks exclusively at how well a player produced while he was on the field. Additionally, AV per season allows us to project total career production by multiplying the number of years since the draft times the projected AV per season. We find each of these definitions to have merit depending on the context of the analysis, and we will use them intermittently throughout this report. Importantly, we do not limit our player valuation to the first four years of a player’s career. While this approach has its limitations, we work under the simplifying assumption that each team/GM is making drafting decisions by total player value.

Pick value per game:

Pick value per season:

Looking at the above plots of average pick value vs. pick number (with accompanying LOESS lines), there appears to be a logarithmic relationship, and we confirm this fact in the following plots. An important methodological note is that in order to avoid dropping observations that would include a ln(0), we use the average value at each pick number (as the plots display).

Regressing log AV per season on pick number gives us the following result, which suggests that each successive pick declines in per-season value by approximately 0.96%.

Regressing log AV per game on pick number gives the following result, which suggests that each successive pick declines in per-game value by approximately 0.74%.

These pick values label lower picks as more valuable than other popular public models (and hence higher picks as less valuable); by our construction, approximately 25-30% of total value is gone by the end of the first round. Most other models have that figure around 35-40% (OverTheCap, PFF). The reason for this is likely our use of average pick value in the regressions, which allows one or two right-tail outcomes to buoy average late-round draft values (you can observe that even average pick values become significantly more dispersed as the draft gets later). With this in mind, we assume that our predictions for pick value have wider confidence intervals in later rounds.

Before jumping into any analysis, we feel it necessary to reiterate that these findings are driven by Pro Football Reference’s system of player valuation, and any judgment of the quality of that valuation is beyond the scope of this report. As such, we are aware that results may be affected by biases and/or inaccuracies in the Approximate Value metric, but we believe strongly in the validity of these high-level insights.

As a methodological note, whenever we reference general manager performance, we isolate each GM’s tenure to one team (e.g., Dave Gettleman’s time with the Panthers and Giants are treated separately).

3) General Insights

3.1) Team Performance

We then assign a per-game, per-season, and total (per season times number of seasons since draft) expected value to each draft pick from the last 10 years. Summing the total expected and actual values (based on actual career production), we get a cursory look at who’s had the most draft capital and how those picks have materialized – or not:

NFL Team Draft Performance Over Expected
2011-2020 | Expected approximate value
Tm Expected AV Actual AV AV Over Expected
SEA 924 1,206 352
BAL 989 1,122 159
KC 887 1,015 147
NO 758 887 143
DAL 876 992 130
CAR 816 913 121
GB 928 1,028 119
NE 933 979 92
ATL 749 828 83
HOU 925 960 64
WAS 929 970 61
PIT 860 864 47
MIA 921 951 42
BUF 952 969 33
MIN 1,014 995 25
CHI 800 798 17
IND 848 860 16
PHI 909 909 12
TB 887 861 −9
LV 884 839 −11
LAR 1,016 974 −11
DEN 918 883 −19
ARI 890 834 −28
DET 902 850 −36
LAC 860 802 −52
CIN 1,053 948 −77
JAX 925 837 −85
TEN 976 861 −110
NYJ 883 700 −143
NYG 870 703 −154
SF 1,046 860 −161
CLE 1,162 952 −187
Table: HSAC | Data: Football Reference

Unsurprisingly, teams that have consistently finished at the bottom of the league have also accrued the most draft capital, and inability to cash in on these resources seems to perpetuate the cycle (the top four teams in expected AV are also in the bottom 12 in AV over expected). The Ravens’ and Patriots’ ability to maintain high expected – and realized – returns despite winning consistently over this period speaks to their organizational intelligence in draft pick trading and acquiring compensatory picks through player development (more on this later). As you will see in the next figure, Seattle’s massive performance over expectation is almost entirely driven by three players (Russell Wilson, Bobby Wagner, and Richard Sherman). Finally, drafting over expectation does seem to separate the elite teams from the pack; the seven franchises with at least a .600 win percentage in the last decade are all in the top 12 in terms of drafting over expectation.

Below are some additional visualizations of expected versus actual draft performance.

By total AV:

By AV per game:

One thing that sticks out here is the exceptional performance of the Ravens in both per-game and overall production. Given the franchise’s open embrace of analytics, perhaps it should be no surprise that Baltimore has drafted well, but it’s definitely something to watch as Eric DeCosta enters his third season as Ravens GM after taking over for the extremely successful Ozzie Newsome. It’s also interesting to note the teams that have drafted over expectation in one metric but not another; for example, a fair number of teams (Chicago, Indianapolis, Buffalo, Jacksonville) have been pretty average overall but have done quite well on a per-game basis, which suggests a high proportion of players who were good while on the field but didn’t play many games.

Also, it’s vital to note that this chart – along with others, but especially this one – measures outcomes. Unfortunately, we don’t get to observe team big boards or explicit draft philosophies. With that said, the Law of Large Numbers suggests that given enough sampled observations, we can get a decent idea of the true underlying tendencies.

3.2) Individual Picks

We can also look at which individual players have most over- and under-performed relative to expectation, from both a per-game and career production standpoint.

By total AV:

By AV per game:

Most of the names you see are the usual suspects, and the per-game measure seems to have a degree of recency bias – this makes sense given that performance tails off later in your career (unless your name is Tom Brady). As mentioned previously, the Seahawks are well represented here, as the top three players in AV over expected are Russell Wilson (+95.3), Richard Sherman (+77.2), and Bobby Wagner (+70.7). Chart No. 2 is an obligatory reminder that Patrick Mahomes is a wunderkind, and he’d probably show up on the total production over expected leaderboard if not for his red-shirt rookie NFL season.

3.3) Importance of Drafting Success

While the draft is indisputably important as the primary channel for acquisition of young, affordable talent, it is not clear precisely how much effective drafting contributes to longer term team success. Here, we compare AV over expected with team winning percentage, and it doesn’t take more than a glance to observe the strong, positive association. Teams that realize greater outcomes than their pre-draft expectation experience high win percentages, most likely because of those successful picks. Note that AV has some measures of team success baked in (team offensive and defensive performance), so some of that correlation comes from the direct relationship between good team offense/defense and the metric we use to measure production.

3.4) Consistency

While certain GMs have experienced greater success than others, the public skill perceptions derived from a few observations may not accurately reflect true ability. In 2011, Cade Massey explored this phenomenon, trying to assess true manager skill in his paper Flipping Coins in the War Room: Skill and Chance in the NFL Draft. Massey’s findings suggest that teams misallocate resources toward improving their selection skill but neglect the role of uncertainty.

KEY INSIGHT: One big takeaway here is that it’s really hard to draft well consistently. Aside from John Schneider and Ozzie Newsome, the really good drafters have all made very few selections (Brandon Beane, John Lynch, John Dorsey), and the larger bubbles tend to cluster towards the middle (à la regression to the mean).

The Packers’ Brian Gutenkunst fares particularly poorly here, in large part because last year’s first-round pick (Jordan Love) hasn’t stepped on the field and Gutenkunst has only been around since 2018. Mike Brown, owner and GM of the Bengals, has consistently underperformed, as has Jacksonville GM Trent Baalke (hello, Trevor Lawrence).

Now, we graph success rate (the percentage of picks with positive AV per game over expected), versus the total AV per game over expected. Analogous to the differentiation between EPA per play and percentage of plays with positive EPA (success rate), this approach allows us to determine whether success for specific observations is driven by consistent performance above expectation or a small number of successful picks.

There is definitely a positive relationship between success rate and AV over expected, but for any given AV over expected, there are a wide range of success rates that could lead to that particular value. Perhaps the greatest indicator of the randomness in the process is Dave Gettleman, who actually records the highest success rate over his time with the Panthers, but one of the lowest success rates with the Giants. We also see John Schneider pop up as someone who is average on a per-game basis and below average in pick success rate, adding to the narrative that a few big selections have buoyed his performance.

As one further measure of whether selection success is truly skill-based (and hence replicable), we examine the correlation between general managers’ success in year n and year n+1:

KEY INSIGHT: There is no observable stickiness in year over year performance of general managers. This supports FiveThirtyEight’s 2014 finding that it is extremely difficult to consistently beat the market. Given the difficulty of regularly beating the market in the selection process, trade skill is an increasingly important way for general managers to differentiate themselves and a key axis along which managers can create value for their teams.

4) Strategy

While success of outcomes gives us one view of teams’ selection processes, another angle is general strategy: do franchises tend to allocate their capital in distinct ways that provide insight into how they value particular positions or approaches? Here, we explore further.

4.1) Positional Allocation

One axis by which to measure drafting preferences is the level of investment in various position groups. We get the percentages in the table below by summing the expected AV per game of picks spent by each team on each position and dividing by the summed AV per game of all picks made by the team in that span.

NFL Team Draft Capital Allocation by Position
2011-2020 | Value-weighted percent of total draft capital
Tm QB RB WR TE OL DL LB DB ST Offense Defense
ARI 8% 9% 15% 5% 18% 17% 9% 19% 0% 55% 45%
ATL 1% 9% 10% 4% 18% 16% 15% 26% 1% 41% 58%
BAL 4% 9% 14% 7% 17% 20% 13% 16% 0% 51% 49%
BUF 8% 6% 15% 3% 15% 14% 17% 20% 2% 47% 51%
CAR 5% 7% 11% 4% 13% 26% 13% 20% 1% 40% 59%
CHI 5% 8% 12% 6% 17% 17% 14% 20% 1% 48% 51%
CIN 7% 10% 14% 7% 17% 13% 15% 17% 1% 55% 44%
CLE 9% 8% 10% 6% 15% 21% 11% 20% 1% 48% 51%
DAL 2% 9% 11% 5% 16% 19% 16% 22% 0% 43% 57%
DEN 7% 8% 14% 8% 17% 19% 11% 16% 1% 54% 45%
DET 1% 12% 8% 7% 20% 19% 12% 19% 2% 48% 50%
GB 4% 9% 11% 6% 14% 17% 14% 23% 1% 45% 54%
HOU 5% 5% 15% 6% 20% 23% 7% 17% 1% 51% 48%
IND 4% 8% 12% 4% 23% 16% 14% 18% 0% 52% 48%
JAX 8% 7% 17% 3% 14% 14% 14% 22% 2% 48% 50%
KC 6% 10% 13% 3% 17% 19% 13% 19% 0% 48% 52%
LAC 4% 7% 9% 4% 17% 18% 19% 20% 1% 42% 58%
LAR 5% 12% 17% 7% 16% 14% 9% 19% 1% 56% 42%
LV 4% 5% 13% 5% 17% 20% 10% 25% 1% 44% 55%
MIA 6% 9% 11% 8% 22% 13% 12% 18% 2% 55% 43%
MIN 5% 5% 13% 6% 19% 15% 12% 22% 3% 48% 49%
NE 6% 8% 8% 5% 19% 18% 13% 20% 2% 47% 51%
NO 3% 7% 10% 2% 18% 18% 18% 25% 0% 39% 61%
NYG 7% 9% 10% 4% 20% 22% 6% 23% 0% 50% 50%
NYJ 11% 5% 14% 5% 12% 20% 11% 20% 1% 47% 52%
PHI 9% 6% 15% 4% 17% 16% 11% 20% 1% 51% 48%
PIT 4% 7% 16% 4% 13% 12% 21% 23% 1% 44% 56%
SEA 2% 10% 12% 5% 18% 26% 11% 15% 1% 48% 51%
SF 4% 8% 15% 5% 14% 23% 9% 20% 2% 46% 52%
TB 5% 12% 9% 7% 11% 16% 12% 25% 3% 44% 53%
TEN 7% 9% 14% 2% 18% 16% 16% 16% 0% 51% 49%
WAS 7% 12% 12% 3% 17% 21% 10% 17% 1% 52% 48%
Avg 5% 8% 12% 5% 17% 18% 13% 20% 1% 48% 51%
Table: HSAC | Data: Football Reference

The average allocation by position gives us an idea of which positions teams tend to target in the draft. The heaviest spending is on DBs, which makes sense because 1) a struggling secondary is a very identifiable and pressing weakness, while other weaknesses (e.g., receiving corps) can often be attributed to other effects (poor QB performance, play design, etc.), and 2) the defensive backfield has five starters and often employs players who have special teams roles, requiring more players to fill roster spots than most other positions. Teams tend to allocate the smallest amount of draft capital to special teams, which is reasonable given that special teams plays only make up a small portion of the plays in a game, so even very strong special teams players may not be worth a high investment. Additionally, the lack of pure special teamers drafted is an artifact of player classification, where only long-snappers, punters, and kickers are drafted as such.

The Jets lead the league in draft capital allocated to quarterbacks, which isn’t a shocker given that a predominant storyline of the past decade has been their never-ending search for a signal-caller. In the past 10 seasons, they have started 11 different quarterbacks and will be starting a 12th going into the 2021 season after trading 2018 first-round pick Sam Darnold to the Panthers.

For the majority of teams, we see a fairly even split between offensive and defensive spending. New Orleans has the most unbalanced allocation, with 61% of draft capital going to defense and 39% going to offense. This likely resulted from a five-year stretch in which the Saints ranked in the bottom four teams of the league by points allowed in four out of five years. As expected, stretches of consistently poor performance tend to lead to increased targeting of associated weaknesses. On the other end of the spectrum from the Saints, we see the Los Angeles Rams leading the league with an offensive capital allocation of 56%. In the Steve Spagnuolo/Jeff Fisher era, the Rams never ranked above 21st in points scored, so rebuilding the offense was definitely a primary objective in the early drafts of the decade.

We also take a look at where teams have successfully/unsuccessfully invested by summing total AV over expected by position and team:

NFL Team Drafting Success by Position
2011-2020 | Total AV over expected
Tm QB RB WR TE OL DL LB DB ST Offense Defense
ARI −12 −18 −11 −52 13 3 −16 64 0 −79 51
ATL −4 45 59 0 8 −28 14 −16 5 108 −30
BAL 59 −23 −14 −8 122 95 −5 −66 0 134 24
BUF −19 −12 −9 −2 16 −7 59 2 5 −26 54
CAR 52 4 −28 −4 22 −51 80 28 18 46 57
CHI 5 26 −12 −31 45 −58 26 11 6 33 −21
CIN 37 25 40 −47 −4 −56 −41 −35 5 51 −133
CLE −54 −31 −49 −33 30 −18 17 −54 5 −137 −55
DAL 54 52 −5 −33 62 6 2 −7 0 128 1
DEN −32 −26 −20 −14 49 4 63 −49 6 −42 17
DET −4 −22 −45 −23 112 −47 12 −19 1 17 −55
GB −8 18 52 −33 70 −3 6 13 4 99 16
HOU 18 −9 −35 −12 74 53 33 −61 4 35 25
IND 16 10 24 −31 57 −50 38 −49 0 78 −61
JAX −15 −8 −46 −9 −8 −5 44 −36 −2 −87 4
KC 6 8 −3 44 84 30 1 −23 0 139 8
LAC 0 3 11 −9 4 19 −34 −45 −0 9 −60
LAR 4 −18 −82 −25 −15 88 29 −5 13 −136 112
LV 27 27 8 −29 16 19 29 −108 −1 50 −60
MIA 29 33 10 −30 −19 −12 15 8 9 22 11
MIN −19 26 13 −12 8 −11 56 −41 5 16 4
NE −9 16 −49 −13 124 1 86 −65 1 69 22
NO −17 58 79 −0 78 75 −77 −52 0 198 −55
NYG −15 −38 −5 −18 2 −22 13 −70 0 −75 −80
NYJ −33 −2 −81 −29 14 4 33 −57 7 −130 −21
PHI 15 26 −36 10 46 21 −15 −45 −10 61 −39
PIT −21 33 −15 −4 52 60 20 −76 −3 46 4
SEA 95 12 −11 −5 15 24 145 73 4 106 242
SF 3 −35 −65 3 8 −26 −10 −42 2 −85 −78
TB 11 −35 29 −47 55 −43 81 −46 −14 13 −8
TEN −33 −30 −37 −10 −13 9 24 −19 0 −124 14
WAS 31 6 −15 −3 24 36 1 −15 −4 43 22
Avg 5 4 −11 −16 36 3 23 −28 2 18 −2
Table: HSAC | Data: Football Reference

KEY INSIGHT: The Rams and Saints have noticeably poor draft performance in the categories where they were spending heavily. While teams clearly draft to address existing weaknesses, the extremes likely result from a dual effect: teams who draft poorly witness poor performance, and this perpetuates the cycle of preexisting weaknesses.

Zeroing in on team success at the position level, we see that the Patriots lead the league in average value over expected for the offensive line, likely an area they paid close attention to with Brady under center – and one they thrived at developing under now-retired Dante Scarnecchia.

Inspired by the Saints and Rams, we were curious whether failure at drafting a particular position leads to greater investment in those players because there are holes to fill.

Investment vs. payoff (each dot represents one team and position group):

There appears to be little correlation between draft capital allocation and performance over expected. As investment increases, we observe greater variance in payoff, which makes sense given that larger expected AV is typically associated with higher variance individual decisions (early picks) as well as more picks in general, which allow for greater variance overall.

While capital allocation describes a general distribution by position, it fails to provide us information at the player level. Ideally, we would distinguish between a team that spent 5% of its draft capital on a single quarterback and a team that spent 5% of its draft capital on three quarterbacks. Here, we’ve displayed the percentage of capital allocated per pick spent at each position (e.g., the Panthers spent 2.32% of their total draft capital per pick spent on QBs). This chart diverges from the previous positional allocation graphic in that it distinguishes between teams that spend a lot of low-value picks on a position group and those that spend a few high-value ones (and vice versa).

NFL Team Draft Capital Allocation per Pick by Position
2011-2020 | Value-weighted percent of total draft capital per pick
Tm QB RB WR TE OL DL LB DB ST Offense Defense
ARI 1.9% 1.1% 1.3% 1.0% 1.2% 1.3% 1.8% 1.5% 0.0% 1.3% 1.5%
ATL 0.5% 1.3% 1.6% 1.3% 1.8% 1.5% 1.5% 1.6% 0.7% 1.5% 1.5%
BAL 0.9% 1.0% 1.0% 1.4% 1.0% 1.0% 1.6% 1.0% 0.0% 1.0% 1.1%
BUF 1.6% 1.2% 1.3% 0.8% 1.4% 1.8% 1.3% 1.3% 0.7% 1.3% 1.4%
CAR 2.3% 1.2% 1.8% 1.4% 1.5% 2.0% 1.4% 1.4% 0.6% 1.5% 1.6%
CHI 1.7% 1.3% 1.5% 2.1% 1.5% 1.9% 1.8% 1.3% 0.8% 1.6% 1.6%
CIN 1.3% 1.0% 1.1% 1.2% 1.2% 1.1% 1.1% 1.0% 0.8% 1.1% 1.1%
CLE 1.8% 1.3% 1.0% 0.9% 1.2% 1.4% 1.0% 1.0% 0.5% 1.2% 1.1%
DAL 0.8% 1.1% 1.2% 1.0% 1.6% 1.2% 1.3% 1.2% 0.0% 1.2% 1.2%
DEN 1.2% 1.1% 1.4% 1.2% 1.3% 1.6% 1.0% 1.2% 0.5% 1.3% 1.3%
DET 0.6% 1.2% 1.2% 1.5% 1.5% 1.2% 1.2% 1.4% 0.6% 1.3% 1.2%
GB 1.2% 1.0% 0.9% 1.0% 1.0% 1.2% 1.0% 1.5% 0.6% 1.0% 1.2%
HOU 1.6% 0.9% 1.5% 1.2% 1.4% 1.4% 1.5% 1.2% 0.9% 1.3% 1.3%
IND 1.4% 1.0% 1.3% 1.4% 1.3% 1.2% 1.0% 1.4% 0.0% 1.3% 1.2%
JAX 1.3% 1.3% 1.4% 0.9% 1.6% 1.2% 1.4% 1.2% 1.0% 1.4% 1.3%
KC 1.5% 1.2% 1.3% 1.4% 1.6% 1.4% 1.5% 1.3% 0.0% 1.4% 1.4%
LAC 1.5% 1.0% 1.3% 1.9% 1.4% 1.7% 1.6% 1.7% 0.8% 1.3% 1.6%
LAR 1.5% 1.3% 1.4% 1.3% 1.1% 1.1% 0.8% 1.1% 0.6% 1.3% 1.0%
LV 1.5% 1.1% 1.1% 0.9% 1.3% 1.2% 1.0% 1.4% 0.7% 1.2% 1.2%
MIA 1.9% 1.0% 1.1% 1.1% 1.7% 1.2% 1.3% 1.3% 0.7% 1.3% 1.3%
MIN 1.5% 1.3% 1.0% 1.0% 1.0% 0.9% 0.8% 1.1% 0.6% 1.0% 0.9%
NE 1.3% 1.6% 1.1% 0.9% 1.1% 1.4% 1.0% 1.2% 0.8% 1.2% 1.2%
NO 1.3% 1.4% 2.0% 1.1% 1.8% 1.8% 1.6% 1.8% 0.0% 1.6% 1.7%
NYG 1.7% 1.3% 1.6% 1.4% 1.4% 1.4% 0.7% 1.4% 0.0% 1.5% 1.3%
NYJ 1.5% 0.8% 1.2% 1.3% 1.2% 1.7% 1.4% 1.3% 0.6% 1.2% 1.5%
PHI 1.8% 1.0% 1.5% 2.2% 1.3% 1.2% 1.2% 1.2% 1.2% 1.4% 1.2%
PIT 1.4% 1.2% 1.4% 0.8% 1.3% 1.0% 1.5% 1.3% 0.6% 1.2% 1.3%
SEA 1.0% 0.9% 1.0% 0.9% 1.1% 1.1% 1.1% 0.9% 0.9% 1.0% 1.0%
SF 1.0% 1.0% 1.2% 0.9% 0.9% 1.3% 1.1% 1.0% 0.9% 1.0% 1.1%
TB 2.3% 1.1% 1.1% 1.4% 1.6% 1.5% 1.3% 1.7% 1.4% 1.3% 1.5%
TEN 1.5% 1.3% 1.8% 1.2% 1.5% 1.4% 1.3% 1.1% 0.0% 1.5% 1.2%
WAS 1.8% 1.0% 1.0% 1.0% 1.2% 1.5% 1.2% 0.9% 0.5% 1.1% 1.1%
Avg 1.4% 1.1% 1.3% 1.2% 1.3% 1.4% 1.3% 1.3% 0.6% 1.3% 1.3%
Table: HSAC | Data: Football Reference

A couple standouts here are the Panthers, leading the league in QB investment since Cam Newton was the No. 1 pick in 2011, the Eagles, leading the league in TE investment, most likely because of the selections of Zach Ertz (2013) and Dallas Goedert (2018) – both in the second round. Interestingly, New England leads the league in capital allocated toward RBs. While the “running backs don’t matter” narrative has pervaded the draft discourse in recent years, the Patriots selected Sony Michel in the first round in 2018 and Damien Harris in the third in 2019.

KEY INSIGHT: The league average per-player investment is quite similar across positions. Of course, quarterback leads and special teams is significantly lower than the rest, but the general uniformity suggests that teams are typically drafting to fill holes, and not spending capital according to a theoretical belief about which positions warrant more draft spending.

Finally, we look at generic positional payoff to get a rough idea of the difficulty of identifying talent and how that varies by position. However, we should be wary of drawing strong conclusions because positions differ in their total distribution of AV relative to expected.

NFL Drafting Payoff by Position
2011-2020 | Approximate value per game over expected
Pos Predicted AV/G Actual AV/G AV/G Over Expected
QB 0.228 0.311 0.083
OL 0.203 0.254 0.052
RB 0.176 0.211 0.035
LB 0.190 0.217 0.027
DL 0.206 0.203 −0.003
WR 0.197 0.185 −0.012
ST 0.118 0.103 −0.015
DB 0.195 0.173 −0.023
TE 0.184 0.109 −0.075
Table: HSAC | Data: Football Reference

Unsurprisingly, quarterback is the position which corresponds to the greatest average value over expected, because quarterback is the position that can take on the highest average values. Interestingly, the average value over expected is significantly negative for tight ends. This could potentially be explained by the fact that even starting tight ends may see low usage numbers in many schemes, which also makes talent identification more difficult at the collegiate level.

A trend we see here and continue to observe throughout the report is that putting a lower bound on AV (at 0) means that using AV over expected limits measured underachievement without capping overachievement.

4.2) Low vs. High Variance Approach

One way to analyze drafting tendencies is through variance in outcomes above expected: if there is high variance in performance over expected, the team is drafting many players who either significantly over- or underproduce (i.e., a “boom-or-bust” strategy). Alternatively, a low variance strategy suggests safer draft picks that are less likely to boom or bust.

Here, we plot production over expected (per season) versus variance in production over expected:

Interestingly, we see a strong, positive relationship between the variance and outcome, which has two possible explanations. For one, negative payoff is bounded because the least that a player can produce is zero, so the distribution of possible outcomes is positively skewed. As such, high variance simply indicates high positive variance, so it should come as no surprise that higher variance corresponds to better outcomes on average. The other competing – and more interesting, although perhaps less likely – explanation is that choosing high-variance players is an optimal draft strategy that results in higher payoffs overall. A more rigorous analysis is likely necessary to disentangle these two possible effects, and we plan on investigating this further in the coming weeks. In both plots, the Saints, Seahawks, Chiefs, Ravens, and Colts have higher variance in outcomes, whereas the Giants, Browns, Bengals, Raiders, Lions, and Broncos tend to perform more consistently (and poorly).

5) Draft Pick Trading

One of the most interesting aspects of the draft is that it allows teams to move assets between the present and future. In this section, we unpack team draft pick trading trends: who trades up vs. back, gives up future picks for opportunities today, and effectively generates surplus through draft pick trading? Importantly, we look exclusively at pure draft pick trades (no players involved), and we assign expected pick values based on where they actually fall, even if the exact position of the pick was unknown at the time of the trade. These caveats mean we are looking at process over results and separating draft pick trading from actual player selection, which provides distinct – but complementing – insights to the previous sections on draft selections. Through examination of surplus generated by trades, we can identify teams that are improving their expected Approximate Value. We acknowledge that many trades are made circumstantially to target specific players, but this method gives us a measure of the expected value generated from the trade itself.

5.1) Performance

Draft pick trading performance by team:

NFL Team Draft Pick Trading Success
2011-2020 | All draft pick trades | Predicted AV per game
Tm Trades Given Received Surplus
CLE 28 10.58 13.13 2.55
SEA 28 9.41 11.75 2.34
MIN 37 9.68 11.41 1.73
IND 17 5.27 6.72 1.45
LV 22 6.26 7.57 1.31
NE 37 11.02 12.24 1.22
LAR 25 9.45 10.61 1.16
SF 28 9.82 10.61 0.79
BAL 18 6.43 7.12 0.68
WAS 20 6.96 7.57 0.61
TEN 18 6.22 6.76 0.53
DEN 17 5.50 5.90 0.39
CIN 8 2.52 2.82 0.30
TB 18 6.39 6.50 0.11
ARI 9 3.51 3.21 −0.29
DAL 10 2.77 2.45 −0.32
DET 15 4.82 4.42 −0.40
GB 19 6.57 6.13 −0.44
NYJ 17 5.25 4.77 −0.48
MIA 23 7.43 6.93 −0.50
KC 12 4.57 3.99 −0.58
PHI 24 8.29 7.67 −0.62
PIT 4 1.76 1.04 −0.72
NYG 4 1.70 0.98 −0.72
HOU 13 4.57 3.79 −0.78
JAX 11 4.32 3.49 −0.84
CHI 15 6.06 5.17 −0.89
LAC 5 2.63 1.64 −0.99
CAR 11 4.03 2.81 −1.22
BUF 12 6.85 5.32 −1.53
ATL 14 5.59 3.98 −1.61
NO 13 5.74 3.50 −2.24
Table: HSAC | Data: Football Reference, Lee Sharpe

Cleveland ranks first in surplus generated from trades despite the poor outcomes we observed in the draft performance charts. This means Cleveland has made a number of effective trades, but not used the draft picks they received in those trades to select players who materialized – an indicator that poor fortune has played a heavy role in the Browns’ struggles (pretty hard to go 0-16 without some degree of bad luck).

Seattle ranks second in expected surplus generated; as we saw earlier, Seattle’s drafting success is heavily influenced by a few selections, but trading back is widely recognized as a key part of their process (see next section). Just like Seattle, New England and Minnesota are also very active players in the trade market.

Draft pick trading performance by GM (min. 10 trades over that span):

NFL GM Draft Pick Trading Success
2011-2020 | All draft pick trades | Predicted AV per game
gm Tm Trades Given Received Surplus
John Schneider SEA 28 9.41 11.75 2.34
Rick Spielman MIN 37 9.68 11.41 1.73
Chris Ballard IND 11 3.36 4.91 1.55
Bill Belichick NE 37 11.02 12.24 1.22
Les Snead LAR 24 9.30 10.39 1.09
Reggie McKenzie LV 14 3.73 4.80 1.07
Ozzie Newsome BAL 14 4.99 5.61 0.62
Trent Baalke SF 19 6.17 6.66 0.49
Jon Robinson TEN 12 4.37 4.78 0.41
John Elway DEN 17 5.50 5.90 0.39
Jason Licht TB 12 4.11 4.48 0.37
Ted Thompson GB 14 3.88 3.86 −0.01
Chris Grier MIA 11 3.12 2.92 −0.20
Jerry Jones DAL 10 2.77 2.45 −0.32
Howie Roseman PHI 24 8.29 7.67 −0.62
Ryan Pace CHI 11 4.74 4.12 −0.62
Mike Maccagnan NYJ 10 3.16 2.45 −0.71
Rick Smith HOU 10 4.01 3.22 −0.79
Thomas Dimitroff ATL 14 5.59 3.98 −1.61
Mickey Loomis NO 13 5.74 3.50 −2.24
Table: HSAC | Data: Football Reference, Lee Sharpe

Steelers’ GM Kevin Colbert is a glaring omission from this table because he doesn’t meet the 10-trade threshold, which is pretty incredible given that he’s been there for a decade. In the five trades they’ve made since Ted Thompson’s departure, perhaps the most notable of which was moving up from 30 to 26 to pick Jordan Love, the Packers have lost a fair amount of expected surplus (from -0.01 AV/G per trade with Thompson to -0.44 today).

Inspired by anecdotal evidence from Seattle, New England, and Minnesota, we take a look at how trade frequency relates to trade success, and there appears to be a strong positive relationship.

KEY INSIGHT: Given that teams who trade more often have accumulated more surplus from trades, we pose two potential explanations: 1) teams with skills that provide them an advantage in trade markets are likely to engage more heavily in those markets, and 2) teams who are active in the trade discussions amass a better understanding of the markets and thus are more easily able to identify and capitalize on available opportunities. In either (or both) of these cases, a pattern of stronger knowledgeable teams preying on less informed, less active market participants is likely to arise.

Our results support this hypothesis: all teams that have engaged in 15 or fewer trades (except the Bengals) have a negative total trade surplus. In particular, the Giants and Steelers, notorious for inactivity in the trade markets, have pretty poor trades surplus even after only engaging in four trades each.

5.2) Directional Tendencies

Here, we track the average pick differential between the highest pick on each side of the trade and see which teams and GMs prefer to trade back (negative average pick differential) vs. forward (positive average pick differential). We use the highest pick on each side of the trade in order to avoid late-round picks from diluting the principal assets moved in the trade.

At the team level:

Which Teams Like to Trade Up/Down?
2011-2020 | All draft pick trades | Avg. highest pick on both sides of trade
Tm Trades Given Received Net
NO 13 97.3 87.0 10.3
ATL 14 105.1 96.1 9.0
LAC 5 46.8 39.8 7.0
NYG 4 100.0 93.2 6.8
KC 12 94.4 88.3 6.1
JAX 11 75.1 69.2 5.9
TEN 18 115.6 110.1 5.5
SF 28 79.1 73.8 5.4
CAR 11 104.7 99.8 4.9
BUF 12 50.1 45.5 4.6
GB 19 95.8 91.6 4.2
CLE 28 73.4 71.3 2.0
MIA 23 109.2 107.3 1.8
TB 18 84.8 83.3 1.5
CIN 8 93.6 92.4 1.2
CHI 15 83.1 82.0 1.1
HOU 13 99.8 99.8 −0.1
NE 37 105.4 106.2 −0.8
ARI 9 78.2 79.3 −1.1
MIN 37 113.6 114.7 −1.1
PIT 4 75.2 76.5 −1.2
LAR 25 91.2 93.1 −1.8
DEN 17 100.6 103.5 −2.9
PHI 24 106.8 109.8 −3.1
BAL 18 74.4 78.1 −3.6
SEA 28 74.0 78.0 −4.0
IND 17 84.3 88.6 −4.4
WAS 20 90.4 95.2 −4.8
LV 22 97.8 103.5 −5.8
DET 15 98.0 103.9 −5.9
NYJ 17 110.1 117.8 −7.7
DAL 10 122.4 132.3 −9.9
Table: HSAC | Data: Football Reference, Lee Sharpe

The Saints and Falcons like to move up, whereas the Cowboys and Jets prefer to fall back. An interesting follow-up (that would be admittedly difficult to measure) would be to look at whether teams tend to trade up in order to target a specific player or because they believe they are improving their general expected surplus. Conventional knowledge says it’s the former, but perhaps there are strategic trade-ups that are not necessarily made to target a particular player but rather to increase draft capital, as is often the case with trading back.

Draft pick trading tendencies by GM (min. 10 trades over that span):

Which GMs Like to Trade Up/Down?
2011-2020 | All draft pick trades | Avg. highest pick on both sides of trade
gm Tm Trades Given Received Net
Mickey Loomis NO 13 97.3 87.0 10.3
Thomas Dimitroff ATL 14 105.1 96.1 9.0
Jon Robinson TEN 12 120.7 113.9 6.8
Trent Baalke SF 19 87.1 81.8 5.2
Ted Thompson GB 14 115.6 111.5 4.1
Chris Grier MIA 11 135.8 132.9 2.9
Jason Licht TB 12 84.2 84.2 0.0
Ryan Pace CHI 11 76.2 76.2 0.0
Bill Belichick NE 37 105.4 106.2 −0.8
Rick Spielman MIN 37 113.6 114.7 −1.1
Les Snead LAR 24 89.0 90.4 −1.4
Rick Smith HOU 10 81.0 82.4 −1.4
Ozzie Newsome BAL 14 70.6 72.5 −1.9
John Elway DEN 17 100.6 103.5 −2.9
Howie Roseman PHI 24 106.8 109.8 −3.1
John Schneider SEA 28 74.0 78.0 −4.0
Reggie McKenzie LV 14 101.1 106.0 −4.9
Chris Ballard IND 11 80.2 86.7 −6.5
Mike Maccagnan NYJ 10 120.1 129.4 −9.3
Jerry Jones DAL 10 122.4 132.3 −9.9
Table: HSAC | Data: Football Reference, Lee Sharpe

Many GMs here match up with the team table given that they served in the position for the duration of the time period in question, but it’s interesting to observe “part-time” GMs that have stronger tendencies than their franchise did before/after their arrival. For example, Chris Ballard conducted 11 of Indianapolis’s 17 pick trades in the last decade, but his strong tendencies have clearly influenced the Colts organization to trade back more frequently than they did prior to his arrival.

5.3) Time Preference

Next, we look at which teams and GMs like to give up picks today for picks in future years (or vice versa). In order to get a proxy for these time-based trends, we average the number of years into the future of the picks on each side of the deal and subtract the picks given away from those received; a positive net value indicates a preference for trading into the future, whereas a negative one suggests the team/GM isn’t afraid to give up tomorrow’s assets for more chances today.

Which Teams Like to Give up Future Picks?
2011-2020 | All draft pick trades | Avg. number of years into future
Tm Trades Given Received Net
PIT 4 0.33 0.00 0.33
NO 13 0.32 0.00 0.32
DAL 10 0.25 0.00 0.25
CHI 15 0.17 0.04 0.13
KC 12 0.11 0.00 0.11
BAL 18 0.10 0.00 0.10
DET 15 0.10 0.00 0.10
SEA 28 0.09 0.00 0.09
TB 18 0.08 0.00 0.08
LAC 5 0.07 0.00 0.07
TEN 18 0.12 0.07 0.05
CAR 11 0.05 0.00 0.05
IND 17 0.09 0.04 0.04
ARI 9 0.03 0.00 0.03
LAR 25 0.05 0.03 0.02
LV 22 0.02 0.00 0.02
BUF 12 0.08 0.07 0.01
CIN 8 0.00 0.00 0.00
HOU 13 0.08 0.08 0.00
NYG 4 0.00 0.00 0.00
NYJ 17 0.07 0.08 −0.00
GB 19 0.00 0.02 −0.02
PHI 24 0.05 0.08 −0.03
MIA 23 0.04 0.08 −0.04
ATL 14 0.01 0.07 −0.06
DEN 17 0.02 0.08 −0.06
WAS 20 0.06 0.12 −0.06
JAX 11 0.00 0.09 −0.09
NE 37 0.01 0.10 −0.09
MIN 37 0.00 0.10 −0.10
SF 28 0.00 0.15 −0.15
CLE 28 0.02 0.21 −0.19
Table: HSAC | Data: Football Reference, Lee Sharpe

Interestingly, when we examine specific behavioral tendencies, some of the extreme performers also exhibit strong tendencies. New Orleans, for example, has the strongest tendency to trade forward and the second strongest tendency to give away future picks and has recorded the worst expected trade surplus of any team over the past decade. As we explored previously, Seattle and Cleveland have some of the highest trade surplus values, with Seattle’s heavily driven by a tendency to trade back, and here we observe that Cleveland has the strongest tendency to trade for future picks.

Which GMs Like to Give up Future Picks?
2011-2020 | All draft pick trades | Avg. number of years into future
gm Tm Trades Given Received Net
Mickey Loomis NO 13 0.32 0.00 0.32
Jerry Jones DAL 10 0.25 0.00 0.25
Ryan Pace CHI 11 0.19 0.05 0.14
Rick Smith HOU 10 0.10 0.00 0.10
Ozzie Newsome BAL 14 0.10 0.00 0.10
John Schneider SEA 28 0.09 0.00 0.09
Jason Licht TB 12 0.08 0.00 0.08
Mike Maccagnan NYJ 10 0.12 0.10 0.02
Les Snead LAR 24 0.06 0.03 0.02
Reggie McKenzie LV 14 0.00 0.00 0.00
Ted Thompson GB 14 0.00 0.00 0.00
Howie Roseman PHI 24 0.05 0.08 −0.03
Jon Robinson TEN 12 0.07 0.11 −0.04
Thomas Dimitroff ATL 14 0.01 0.07 −0.06
John Elway DEN 17 0.02 0.08 −0.06
Chris Ballard IND 11 0.00 0.07 −0.07
Bill Belichick NE 37 0.01 0.10 −0.09
Rick Spielman MIN 37 0.00 0.10 −0.10
Chris Grier MIA 11 0.06 0.17 −0.11
Trent Baalke SF 19 0.00 0.18 −0.18
Table: HSAC | Data: Football Reference, Lee Sharpe

Examining trade tendencies at the GM level, we consistently see John Schneider, Rick Spielman, and Bill Belichick at the top in both trade count and surplus generated. It makes sense that those at the top have made more trades because general managers who are implementing effective processes are likely to be around longer, and thus have more time to make trades.

Mickey Loomis leads the GMs in both net average pick differential and net average year differential, indicating a tendency to trade forward and to give up future picks. As we saw previously in the Saints’ surplus value, these results don’t seem to have yielded great results. Jerry Jones also lies on the extremes of our two axes, with the highest tendency to trade back and the second highest tendency to give up future picks. He falls in the middle of the pack surplus-wise, though.

5.4) Summaries

While a few GMs (most prominently John Schneider) stand out for their strong performance across both draft trade behavior and realized outcomes, there doesn’t appear to be a strong association between skill across the two categories. Many GMs improve their expectation through draft trades, but do not select well (Trent Baalke). Others make poor trade decisions, reducing their expectation, but hit on some selections, thus boosting their realized surplus (Mickey Loomis). Granted, stronger associations may emerge with more data.

Again, we see Mickey Loomis’s tendencies deviating strongly from the pack, and his negative surplus values failing to justify that behavior. With consistently poor trade surplus, he has likely been riding on the performance of a few successful selections – notably Cam Jordan (+40.3 AVOE), Alvin Kamara (+40.2), and Michael Thomas (+40.1).

KEY INSIGHT: While it is difficult to identify a strong relationship between trade tendencies and surplus, at the team level we do observe a clustering of higher surplus values on the bottom left frontier of the scatterplot (see lighter blue shaded dots), indicating that generally, those who trade back and into the future extract more surplus from trading.

6) Team/GM Asset Management Summary

A few teams distinguish themselves along both axes. The teams in the top right quadrant have effectively recorded high surplus numbers in both trades and the draft and have also witnessed success on the field. In fact, the seven teams in the top right quadrant collectively account for 11 of the 20 Super Bowl appearances in the past decade. Even excluding the Patriots, the remaining six teams are overrepresented in recent Super Bowls relative to what randomness would dictate.

Some interesting pairs of teams stand out here. Consider Baltimore and Washington, which are almost identical in trade surplus and draft surplus but saw vastly different results over the past decade, with Baltimore consistently performing in the top tier of teams and Washington consistently ranking among the bottom teams in the league. This could point to differences in a number of other factors including coaching, free agency acquisitions, and in-game decision making. Another key pair is Cleveland and Seattle: both accumulate a large quantity of trade surplus, but the big difference in draft surplus is likely responsible for the massive gap in win rate.

Interestingly, there doesn’t appear to be a strong link between draft surplus (success of selections) and trade surplus (expected value added through trades). One theory would suggest that identification of talent and understanding of draft markets and pick valuation are separate skills that are not necessarily likely to co-occur. Again, we err on the side of caution in drawing strong conclusions here because we are working with a relatively narrow time frame.

Many of the well-known veteran GMs fall in the top right quadrant, recording high draft surplus and trade surplus. In the bottom left quadrant, there is a notable lack of GMs who perform poorly along both axes – Mike Maccagnan, who was with the Jets from 2015 to 2019, is the only GM to record negative surplus in both categories. This effect, along with positive averages in both categories, is likely due to the 10 trade cutoff, which requires that a GM has spent considerable time with one team and hence has likely experienced some degree of success.

8) Conclusions & Areas for Further Research

With the 2021 NFL Draft around the corner, we are eager to observe whether the trends we’ve identified continue. We will be gathering additional data from a number of newer general managers, which will help improve our confidence in their demonstrated tendencies. In addition, seven teams have hired new general managers this year, so we’ll be witnessing the first draft for many of them.

We’ve already seen significant trade action throughout the offseason, with 2021 picks involved in a number of major blockbusters, including the Sam Darnold trade, the Jared Goff/Matthew Stafford trade, and the Carson Wentz trade. As we enter the draft, we will likely see more picks change hands and are interested in examining whether the behavior we witness is consistent with the tendencies we’ve identified.

While this analysis is certainly rigorous, it is by no means objective truth and ought to be regarded as a set of high-level, descriptive trends based on one way of evaluating draft picks. A few areas of concern that warrant further research include but are not limited to:
- the reliability of Approximate Value as a true measure of player contributions
- examining entire career production as opposed to isolating the first four or five years
- the relatively small size of the dataset, particularly as it relates to GMs

We look forward to your feedback and seeing how these behavioral tendencies, both successful and not, pan out in the coming season.

9) Acknowledgements

There are far too many people to thank for helping this project come to fruition, but here are a few of the most important:
- Kevin Meers and Prof. Christopher Avery for advising and helping work out the kinks
- Lee Sharpe for his NFL trade data repository
- Ben Baldwin and everyone who works on nflfastR for their team colors/logos repository
- The folks at Pro-Football Reference for their draft records, Approximate Value model, and general manager information