by Chris Bruce
With fantasy basketball nearing playoff time and fantasy baseball drafts in full swing, a thought occurred to our club: how much do these fantasy stats that people meticulously pore over actually relate to the real value a player brings to their team? As it turns out, the answer to this question varies widely by sport, fantasy format and player position.
For each sport I’ve calculated fantasy value that is attributable to each player during the last complete season. For football this was straightforward, calculating the number of fantasy points that a player accumulated over the season using the standard scoring values (25 pass yards/point, 10 rush or rec yards/point, 6 points per rush/rec/pass TD and -2 points/int).
Baseball and basketball required more involved calculations since these are usually done in a rotisserie or head-to-head accumulation format. For both of these sports I determined the fantasy value for each player by calculating the portion of each statistical category that each player represented from the pool of stats above replacement level. For example, if Player X had 10 home runs while the replacement level player had 5, and the entire league had a cumulative total of 100 home runs above replacement level, then Player X would have 5% [ (10-5) / 100 ] of the pool of home runs above replacement level. In these accumulated stat formats, the portion of the statistical category that a player represents is essentially the fantasy value of that player in that category, and each category is worth an equal amount. Therefore, a player’s overall value is determined by summing that player’s values in each stat category.
For fantasy teams, replacement level is the level of player that one could pick up from free agency or the waiver wire at any time, so it is determined by the number of teams and the number of players per team. For each sport I’ve assumed standard team sizes and a 12-team league, but changing these assumptions won’t change the conclusions significantly. Correlations were also only done for the number of players that would end up on a team under these team size and number assumptions.
Baseball is the sport with the longest history of fantasy and the greatest volume of statistical research. Despite that, most leagues still hinge on traditional stats that the sabremetric guys disregarded long ago. When comparing a player’s fantasy value in a traditional 5×5 rotisserie league (R, HR, RBI, SB and BA for hitters; W, ERA, K, WHIP and SV for pitchers) to their wins above replacement (WAR) there is a R2 of 0.60 for hitters and 0.75 for pitchers. That is to say, for pitchers only 75% of the variation in WAR, a measure of overall value to the team, is accounted for in the fantasy representation of their value in a 5×5 rotisserie league. The difference between pitchers and hitters is driven by the lack of defensive categories in the 5×5 categories – when looking at only offensive WAR the R2 for hitters goes up to 0.71. Additionally, when you replace batting average with on base percentage, a statistic more representative of the value a player is adding, the R2climbs to 0.75.
The extra 25% that is not explained by fantasy value is from three sources: 1) the use of statistics that aren’t good representations of player value or are context-dependent (e.g., ERA instead of FIP), 2) statistics that are missing from the fantasy game (e.g., slugging or number of bases) and 3) the incorrect importance given to stat categories by weighting them all equally. These three reasons will apply to all of the sports analyzed here.
Basketball is fairly similar, though the fantasy value has even less correlation to real value on the court. Using a 9 category scoring system (FG%, FT%, 3P%, PTS, AST, REB, STL, BLK, TO), the R2 between fantasy value and win shares (WS) is 0.66, significantly lower than baseball. If you’re in a traditional 5-category league (PTS, AST, REB, STL, BLK) then the value drops even further to 0.57.
Football, however, has the least connection between fantasy value and real value on the field – when comparing fantasy points to expected points added (EPA), the R2 is a dismal 0.47, and a 0.53 if you’re in a points per reception league. This is largely due to the lack of an accurate, comprehensive statistic for player value across positions. I’ve used EPA, but this is still an approximation and is dependent on field position. Another reason is the popular use of a points system which isn’t aligned with the real value that yards bring at each position. Interestingly, the correlation between fantasy value and EPA varies widely by position – 0.80 for QBs, 0.79 for TEs, 0.44 for WRs and 0.03 for RBs. There is a huge disconnect for RBs because they are rewarded handsomely for gaining yards in fantasy, even though in the game these yards may not be adding much value (if they come far from the end zone, or come on 3rd down when there isn’t a conversion, etc)
Overall, no fantasy sports replicate their real sport perfectly, though baseball seems to be doing the best job. Additionally, depending on what you’re looking for in your fantasy league, you may consider tweaking the stats you use or the scoring format to make the league more or less representative of the real thing.
Interesting. You mention this, but it’s also probably worth reiterating that EPA calculations are a worthwhile yet still worthless (practically speaking) attempt at describing a football player’s value/contribution.