What types of fantasy analysis is overused?

Some popular ways people talk about fantasy football that aren't particularly useful

It’s been awhile! I’ve been getting some questions about my plans for July and August, and the short answer is I apologize I haven’t laid this out yet, and I’m behind. But yes, I’m definitely going to be writing. It’s been a busy offseason for me with a lot of interest and discussions around the industry, and I’ve also taken some time over the past couple months to get out of the house and enjoy some summer activities after a long year-and-a-half.

But judging by my interactions, you want to know if I’m going to be providing projections, rankings, draft strategy, and the like. Absolutely I am. I’m a volume writer, and when the content hose turns on, you’ll have plenty to chew on. I just checked and last year at CBS, I wrote 25 articles in the month of August, so nearly one per day. I’m not going to promise anything on that scale, but I’m also going to make sure anyone who wants my advice and wants to read my take on any player, can find that information. As I always have.

I’m still working out the details of how that will all look, and I’m going out of town for one last long weekend next Thursday so I’m not sure if I’ll have it all in place for another week or more. But the season is nearly here! The Hall of Fame game that marks the beginning of the NFL preseason is fewer than 20 days away. For those of you that have been so gracious as to support the newsletter and have wondered when to turn those monthly subscriptions back on, the time is definitely coming. I’ll have more information on that soon.

But I just wanted to get something written here today, and it’s a piece of content I’ve wanted to write for a couple of months. Everyone in the fantasy football industry tends to be incredibly supportive of one another, and I think that’s great for the overall health of the industry. But one result is people tend to shy away from criticism.

One problem with that is to understand what you should be considering as you approach your fantasy drafts, it probably makes a lot of sense to be willing to discuss what types of analysis aren’t particularly helpful. I’ve been in this industry for more than a half decade now, and a lot of that has been spent on the content planning side. And one thing I can tell you about that is there are pieces of content that do very well in terms of clicks and traffic numbers that more or less have to be written. I don’t mean to say analysts are writing things they don’t believe — a lot of it is either good shorthand or is at least interesting information to know with appropriate context — but you’ll sometimes see particularly blunt analysts say right in some of these types of pieces to take the information with a grain of salt, because it isn’t particularly predictive.

I want to run through a couple of these types of analyses that get a lot of play in our industry but probably aren’t actually helpful. And frankly, because they are written about and discussed everywhere, are probably actively unhelpful. It’s a delicate topic and I’m not trying to call out any specific person, people, or site, but below are some analyses that simply by their prevalence around the industry can start to carry more weight than they probably should.

It’s important to understand that these analyses do have utility, but that what I’m saying is people rely on and discuss them too much. Some of these things continue to get discussed because sometimes they do point us in the right direction, and as you read the examples you might think of past instances where you’ve used that type of analysis and it’s worked for you. What’s important to recognize is that if something changes your opinion on 10 situations and is right on five of those takeaways and wrong on the other five, it didn’t actually help you.

Plus, these ideas don’t just change an individual’s idea on 10 things; these ideas can change the market, and impact ADP. And that’s why I want to discuss them, because understanding that they aren’t that valuable can actually create opportunity.

Before we jump into those though, I want to quickly promote a couple new podcast projects I’ve been working on. I mentioned I’ve been lucky enough to have had interest in my work from around the industry this offseason, and one of the very cool perks of that is I’ve more or less been able to choose projects I want to work on. I say this not to brag, but to emphasize that my co-hosts on these two projects were probably the two people I most wanted to do more work with this year.

The first is a projections series over at Establish the Run with Mike Leone, who is renowned for his excellent DFS projections on a weekly basis. I’ve been lucky enough to have had lengthy conversations with Leone about his process, and I can’t imagine anyone does it better. I mean I guess I haven’t reviewed every set of projections in the industry, but Leone goes about it the right way and some of the really intelligent extra elements he incorporates are things I haven’t seen anywhere else and are there to help balance some of the biggest flaws and most fragile elements of projections generally.

If you’ve been a reader of mine for some time, you know I’m pretty skeptical of projections, which makes the topic of this limited-run podcast series a funny one. We’re going division by division and discussing our projections for all 32 teams in depth, but a big part of that is actively discussing fragile elements of our projections that can be exploitable in drafts. The episodes are based around projections in theory, but they are really more about us discussing each team, player, and situation, and finding draft-day targets. We’ve finished the first half of these episodes, covering the entire AFC. You can find them by searching for “Establish the Edge” in your podcast app. Here’s an Apple link for the first episode covering the AFC South.

The other podcast I’ve recently launched will be a mainstay hopefully for years to come, meaning it will continue through August, into the season, and beyond. That podcast is one I’m hosting with Shawn Siegele, my personal favorite fantasy analyst. Shawn is something of a mysterious figure in that he doesn’t have a social media presence and people tend to talk about him in hushed tones, but he’s actually a super humble and down-to-earth guy that is in no way responsible for the aura around him. He’s also incredibly good at fantasy football; he’s the rare instance where the reality surpasses the considerable hype, and you should be reading everything he writes over at RotoViz.

That podcast is called Stealing Bananas, which is just a mashup of “Stealing Signals” and Shawn’s original site “Money in the Banana Stand.” You can find it on any of your major podcast services (here’s the Apple link for that one). Shawn and I are doing three episodes per week, all on the same topic, with one focusing on a higher-level, more evergreen discussion, one focused on player-specific takes, and a third typically featuring a guest. So far we’ve had some pretty legendary guests on with Evan Silva, JJ Zachariason, and Drew Dinkmeyer joining our first three weeks. You’ll want to be subscribed to this podcast, and as with any podcast it’s super helpful for us to grow it if you rate and review it in your favorite podcast app.

Now let’s jump into the topic of the day. Before we start really digging into what matters for 2021 fantasy football in the coming weeks, let’s first talk about a few things not to look at.

Vacated opportunity

The first popular fantasy metric I pay almost no attention to is vacated opportunity, which is to say the discussions that add up the total number of targets, carries, and other stats for players who are no longer on a team’s roster. Like all of the stats I’m going to discuss today, vacated opportunity makes intuitive sense, because we’re just looking at which teams have the most and least available opportunity for new faces or young guys already on the roster to step into.

The issue is that its utility stops almost immediately. There’s value in the very simple knowledge of which teams experienced a lot of turnover, but once you’re aware of roster changes, there’s nothing else added here. And while putting specific numbers on this vacated opportunity is a great shorthand way of emphasizing which teams have lost more 2020 contributors than others, those specific numbers also lead to emphasis on which teams have the most or least in this category, which isn’t relevant.

There are several reasons it’s not relevant, but the simplest way to say it is that when a team has a new roster, its trends change. This is true on the player level, but I want to talk about it more on the team level, because we’re pretty terrible at projecting volume there. I’ve mentioned a piece I wrote at CBS on a couple of podcasts this offseason, and the topic wasn’t particularly catchy but it was one that I think was probably among the best things I’ve written over the past few years. It was called, “Where projections missed in 2019, and what that can teach us for 2020,” and in it I reviewed all of my team-level play volume, run, and pass projections to see how accurate I was. I figured there were going to be instances where I was way off, but the result shocked even me. Here’s a quote:

My projection was within 10 total plays of the actual 2019 play volume for eight of 32 teams, which seems solid. But I was also at least 50 plays away from the observed total for nine teams. In terms of run/pass, I was off by at least 50 pass plays for 11 teams, and at least 50 rush plays off for seven. 

For exactly half the teams, my projection was at least 50 plays, passes or runs away from the actual observed outcome.

I went on to discuss the circumstances that led to me being so wrong on so many teams, and keep in mind this was a review of the 2019 season, so it wasn’t pandemic-related. You can see the results for yourself, but the thing I took away from it was just how many of those situations were completely unforeseeable things that not just I missed, but the overall fantasy football landscape at large was also very off on. Even some of the things where we were right, like Arizona playing faster, it was hard to be right on the degree of the shift. I might just be terrible at projections, but in all fairness that probably wasn’t the sole cause. Most of these results were either simply not projectable, or it would have taken some luck to get it really dialed in, because NFL seasons are chaotic and unpredictable.

And to put into perspective what being 50 plays off means, the gap between the team who ran the fewest plays in 2019 (Washington, 885) and the team that ran the most (Philadelphia, 1,104) was only 219 plays. And that’s with Washington being an outlier; Pittsburgh ran the second-fewest plays at 937 in 2019, so almost all the teams were within 200 plays of each other (which was also the case in 2020 where the gap from top to bottom was 187). Given that context, being off by 50 plays is a huge amount of volume. You can think of it like the difference between being a 50th percentile team and being below the 25th or above 75th percentile.

And that’s actually a really interesting way to think about it. The league average in 2019 was 1,016 plays; if I had projected every team for exactly that amount, I would have been off by more than 50 plays for teams who ran fewer than 966 or more than 1,066 plays. There were only 11 teams who fell outside that range in 2019, and again, I was off by 50 or more plays on nine teams in my projections. Similarly, if I had projected every team for league average play volume, I would have been within 10 plays on 11 teams; as I noted in that piece last year, I was actually within 10 plays (or extremely accurate) on just eight. I could have projected every team for exactly league average volume and been more or less just as accurate across the entirety of the league as I actually was. And I promise you, most of my actual projections followed conventional wisdom fairly closely; they weren’t crazy and out of line with the thinking about all these offenses that was driving ADP (I cover many of the specific instances in the article I linked to above).

This is relevant to the vacated opportunity discussion because we really just have no idea what most of these offenses will look like. Projections at least take into account personnel changes, quarterback changes, and the types of context that people discussing vacated opportunity are trying to account for. But even after accounting for the changes of rosters, we’re still not any more accurate. Just taking last year’s team-level volume and assuming it will stay the same — which is sort of a necessary underlying idea behind vacated opportunity — is obviously not going to be any better. And this is before I’ve even begun to discuss whether we understand any volume shifts on the player level when we talk about vacated opportunity.

Again, there’s value in understanding which rosters have changed, and that’s what most vacated opportunity analysis is trying to say. It’s trying to tell people who have been disengaged all offseason that for certain teams, there are new names to know because the guys you remember putting up stats for them in 2020 aren’t there in 2021.

But taking it any further than that and really diving into which teams have the most vacated opportunity, or any other specific takeaway, really isn’t helpful. The reality is a lot of teams that were very good for fantasy potential in 2020 are going to be very bad for it in 2021, and vice versa, and we’re just not going to be very accurate when we try to determine which of those teams will be which.

And to the extent we might be able to identify some of those teams, it would be the teams that have a bunch of either personnel or coaching turnover! These are, of course, the teams with a lot of vacated opportunity, which are inherently those rosters that featured the most turnover. What we need to be doing with those teams is viewing them with huge error bars as if they are a completely clean slate. We can make our best guess, but looking at the specific amount of vacated opportunity from a prior roster that is different from the one we’ll see in 2021 is just as likely to be way wrong as it is to help you figure out who to draft.

In short, please don’t draft Russell Gage just because Julio Jones is gone. That guy is not going to help you win your league.

Relating ADP with some positional rank to define value

This is another useful shorthand, but it’s not at all helpful in determining how to actually win. It can go two ways.

First, you see the analyses that relate end-of-season finish to preseason ADP. “I took him WR30 and he was WR25,” so he helped your team. Second, you see it with projections. “He’s going WR30 and he projects as my WR25,” so he’s a good pick.

There is just so much more that goes into this, the most obvious of which is opportunity cost. You don’t win fantasy leagues by stacking small wins against ADP; you win fantasy leagues by stacking league-winners. That’s why we call them that. And if you took a guy who was a slight win against ADP — or worse projects as a small win against ADP — you’re ignoring the profiles of all the other players you could draft at that position. My aforementioned podcast partner Mike Leone has called certain running backs who fit this discussion “silent killers” to your roster. If you take a running back in the fourth round at RB25 and he finishes RB20, I can promise you he is a massively negative presence to your team in any kind of win rate or related analysis, both because running backs bust at a high rate so a RB20 finish isn’t that impressive, and also because it’s a near lock that other players who you could have reasonable taken there were much better and were disproportionately on the rosters that did win.

I can make that same argument for a running back taken RB25 who finishes RB15 or even probably RB10, which sort of drives home the point. The difference in end-of-season finish is relative to how the other players at that position did during the season — which both doesn’t account for the relative strength of the positions (and no one forced you to take a running back) and doesn’t consider the distribution of that position. In the running back example, typically the difference between RB10 and RB20 really isn’t that significant; it’s often just a matter of games played among players that weren’t particularly useful if you used a high pick on them.

What we’re really trying to talk about is whether a player moved the needle, and if they finished five spots below or above their ADP doesn’t change that. That gets back to the bigger issue I mentioned, which is that you don’t win leagues like this, by stacking small wins, and transitions us into the issue with relating projections to ADP. There are a lot of players every year that look like very good bets to beat their ADPs in the end-of-season results, but that doesn’t mean the market is just bored of them. Think of all the Jarvis Landry tweets about how he always beats his positional ADP — the reason he continued to go later than he should have was not because people didn’t know that, it’s because they didn’t believe he would be a difference-maker on their rosters. That’s not to say there’s never a time and place to draft a guy like Landry — I’ve been writing about how players with high-reception potential are woefully undervalued in PPR in favor of more prototypical alpha No. 1s for a long time — so Landry is kind of a terrible example. But he is one a popular one, and it’s easy to point to him as an example of the phenomenon where ADP isn’t wrong as much as it’s telling us something different than whether that player projects to finish higher than where he’s going. Expected upside and downside is typically baked into ADP.

I could go on for days about how to properly value cost-adjusted risk and upside and what that means for a variety of players, but the simple point is weighing those things is how you actually build a winning fantasy football roster, not trying to find guys who look like small gains against ADP.

You have to understand that your roster is a portfolio of bets, and your goal is to generate the highest return of 12 people with similar portfolios. If all your bets are on what look like stable wins with low ROIs at best, you’re probably going to do OK, but you’re not likely to win your league. Meanwhile, with one pick who becomes what we refer to as a league-winner, you massively increase the overall portfolio of your roster, because those are players who who are overwhelmingly on the best rosters across all the leagues out there. The RotoViz Best Ball Win Rates tool says outside the top-20 picks, those guys last year were players like Stefon Diggs, James Robinson, Justin Jefferson, and Logan Thomas, and the case to draft basically none of those guys amounted to “He projects well against ADP.”

Running back projections generally

Look, I will bash season-long projections any chance I get. Projections in 2021 make most fantasy players worse. They overuse them. They take them far too literally when they are a snapshot of what we expect going into Week 1 in a league that is Absolutely. Guaranteed. to change our expectations immediately.

And running back is the worst. By the end of September, tons of season-long projections are useless. They are simply a launching point. I brought up James Robinson just now and your thought might have been that if only we’d projected him to be the lead back in Jacksonville, we might have been more accurate, but that’s the point. Even after Leonard Fournette was cut, no one was projecting an undrafted rookie free agent to take more than 85% of the running back rush attempts in his backfield. We would have been a lot more open to that after a few games, obviously.

Obviously there are some players that we feel much more confident about, and that’s why they typically go in Rounds 1 and 2. Projections give an end result for every player in the league, but they are not apples to apples comparisons because the error bars on some player (and team!) projections are far wider than others.

Alright, that’s all I have for today. Hopefully I don’t go two months until I write again. Until next time!