Stealing Signals, Week 8, Part 1
Football data and how we use it, plus TNF and early games
I told myself with Halloween landing on Tuesday this week, I’d be quicker with the intros. I even went out of my way to write some extra conceptual stuff up last week to scratch that itch. So of course, I have more ideas floating around than ever.
This is a great reminder for me to mention something. Christmas this year falls on a Monday, and I’m not planning to write up Stealing Signals that week. Last year, when it fell on Sunday, most of the games were Saturday, and despite both my wife and I having big families which has traditionally meant full days on Christmas Eve as well as Christmas, it worked out decently enough. Since I’m west coast, I watched the early window of games Saturday morning at 10 a.m., then was available for family time the rest of Christmas Eve and Christmas Sunday, before writing up the early slate Monday the 26th and then finding some time to watch back some of the stuff I’d missed for Tuesday’s writeup.
That said, it wasn’t my best work, and Stealing Signals isn’t even that relevant with one week left in the season when the focus is more on start/sits. And this year, the schedule would be much more difficult — it’s another full day of games on Christmas Eve, and I will probably be able to watch in the morning, but then Monday — when I need to be writing — is Christmas, and there are three games that day as well, spaced out like Thanksgiving. There’s just no way for the schedule to come together without telling my family to enjoy the holidays without me. And I know almost all of you will understand and will if anything give me crap for explaining this more than was necessary, but I still wanted to! I’ll find time later that week to focus more efforts on Input Volatility and give ideas for how I’d play start/sits for championship week.
Alright, now that I’ve written the introduction to the introduction, here are some assorted introductory thoughts. To try to cut down on how deep I go, I’ll just do that thing where I throw this divider out there whenever I want to switch gears.
I wrote recently about how the increase in schedule length has seemed to minimize the importance of each game in the NFL season, and I have an additional take with that. I think even on a scheme level, teams are trying to hide stuff on film, and not necessarily approaching every game with a 100% desire to go all in to win that game.
That’s maybe not even that shocking, and has probably gone on for a long time. But when you think about an NFL where the defensive focus has been on eliminating explosive plays, and it’s a bend-don’t-break philosophy where red zone efficiency is emphasized, it does hit home when you consider that unique play designs and looks can be such a key to converting those short-area touchdowns.
I’m specifically thinking back to the Chiefs holding back that jet motion return play for the Super Bowl, something they’d set up basically all year with jet motion looks, which led to two walk-in scores in the red zone on the biggest stage imaginable. Converting touchdowns on those drives — both of which came early in the fourth quarter, and helped turn a 27-21 deficit into a 35-27 lead — could be argued as one of the keys to the Chiefs’ win.
Maybe the idea for that play design only came to Andy Reid and co. because of the extra week of prep time before the Super Bowl, and seeing the way the Eagles rolled their safeties on that type of motion made them especially vulnerable. But even if that specific example doesn’t hold, I’ve referenced other examples in the past, like Lane Johnson at the Super Bowl last year saying:
“Over the course of a season we try to change. We monitor what we do on offense. We’re trying to see if we’re being too predictable in what we’re doing. But that’s what we have analytics for. I feel like that branch of the NFL is definitely taking off over the past few years, helping teams gameplan.”
One of my big takeaways in that section of my 2022 recap was “week-to-week variability … might only get wilder in the future, and it felt like that was especially true over the first couple months of the season, as teams felt each other out a bit, whereas the good teams did in some cases seem to start to get a little more consistent and predictable later in the year as they narrowed in on their strongest approaches.”
That’s probably too narrow of an explanation because for any reasoning like this in the NFL, there are always going to be great examples and then also counterexamples. It’s variable; teams are doing different stuff. But I did argue teams might be specifically tailoring gameplan to opponent more than ever, as well, and that “trying to predict approach based on matchup … on a weekly basis” is “more OK than ever … despite the potential pitfalls of that.” (That particular type of analysis will never be my forte, but it remains potentially useful advice.)
Anyway, I thought about these ideas again this week while watching some flat performances from heavily favored teams, where their gameplans felt too vanilla. As I started this section with, I’ve felt like teams are willing to pack it up in a given game a little quicker, or at least from a health perspective pull guys who aren’t right even if they are just “hurt not injured,” as that old saying goes. It all remains fluid, and I’m sure I’ll have more thoughts on this.
One of the biggest things I wanted to write about this week is the dichotomy between watching games versus just looking at stats. I used to mock film bros — and I frankly still do behind closed doors, because so much of what they are trying to apply doesn’t clear the necessary analytical bar to be useful, even if their film points are valid (they often overapply how they can be actionable) — but at this point, I do strongly believe that actually watching the games is a huge advantage in fantasy football.
I’ve written against some of the newer stats that get referenced in fantasy, and one of my big points about all stats is they are going to seem useful in some cases (and not useful in others), and whether you find value in them will be a little bit dependent on how the outcomes go and what you focus on/are biased toward, but that we should be looking for things that are better than 50/50 when we lean on them. And it’s just so, so common in fantasy analysis for some new-age analysis to have some high-profile examples of nailing stuff for a limited sample, and people to really buy in, and then like a year later no one is talking about it at all, presumably because once people chased that line of thinking it dried up and there were some major misses. It was never better than 50/50, the coin just came up heads a few times in a row and you convinced yourself that was a trend that mattered.
It happens constantly. And some of the higher-profile “advanced” stuff that sticks, largely sticks because it can’t be easily disproven. It’s a black box.
I don’t mean to paint with too broad of a brush here, because that would be silly. And I do recognize how some of my recent arguments on this topic have sounded dangerously close to those of a decade ago from people who just didn’t understand basic football stats, and didn’t want to try (e.g. “It’s all a small sample,” “There are too many things happening on every play,” et al).
But things have shifted since then, data is more ubiquitous, and I’m essentially saying that now using those arguments may actually be valid within the realm of acknowledging there’s a ton of good football data can do, and that we’re not trying to shift all the way to the outdated claim that only grinding the all-22 will tell you the real truth about the sport. That’s obviously not it, and not where I am intending to go when I invoke these arguments.
We do still have to acknowledge there are limitations to basically all football statistics, simply by nature of the way the sport is played. The ESPN WR model got a lot of publicity this week, and it’s a black box. People with full understanding of what we’re going for questioned the results. It’s not an inability to understand the data or the model — or a general phobia of numbers or analytics — that drove all those criticisms. And part of it simply is that for WRs, it’s often like a half-dozen to a dozen moments per game that define their week. Watching truly is where you get an idea of who is capable of doing more or less than what the numbers say, particularly when you have a respect for and understanding of the numbers and are seeing in real time what the data is probably going to say (I could probably predict most of the league’s TPRRs in a given weekend, without looking at a box score but only at the games, to within a certain degree of accuracy) and how it was compiled. That’s fundamentally different than just looking up the data and trying to use it to make a point.
For the ESPN WR model, Tyreek Hill’s YAC score came under scrutiny, and then there were a ton of speculations about what in the model was driving his low rank when he had the most raw yards after the catch of any WR. And ultimately, honestly, it’s probably that YAC itself is a bit fluky, because typically speaking huge chunks of it are gained on a couple plays. There are a lot of football metrics where stats are split into success rate and explosive play rate — the success rate side is a central part of sports betting models that acknowledge explosive play rate is flukier, and could create misleading numbers on stuff like yards per play, which is essentially just success rate and explosive rate combined.
Well, YAC is one of those stats where explosive rate is even more of the equation. So we’re talking about only a few plays! I saw D.J. Moore rated highly, and I thought of the play in the Washington game where he elevated for a high throw that the defender tried to undercut and tip away, and when he landed he was just gone. That probably had a low YAC expectation because the defender was so near to him!
But it’s also probably a huge chunk of Moore’s total YAC for the year. Moore’s great at YAC and already has 314 yards, but that was a 56-yard TD that was maybe like 40 yards of YAC, which would be like one-eighth of his entire season so far on that one play. I don’t know if those numbers are exact, but the point remains that one play can be like one-eighth of the sample for a stat like that, and then you get to the specifics of the play and consider what the model is trying to grasp, and that with player tracking a central part of it is probably the defender’s proximity, but that it also probably can’t differentiate between the defender going for the ball and whiffing vis-a-vis the WR making the catch through contract and tossing the defender aside (or whatever other impressive YAC thing might be more predictive), and you just understand that there’s so much that’s uncertain about such a small subset of plays influencing a model like that.
Does it mean we can’t trust that model? No, that’s not what I’m saying. But one of the other things that came to mind as I chewed on this topic was how the NFL’s Next Gen Stats have been around for a few years now, and they were awesome when they were released, and I still see them used during games a lot, and rightfully so. They are a unique thing in the space.
But it’s funny how they haven’t evolved really at all. Or at least, the way the data gets to us — we can’t even get full league data, just qualifying players. There are cool box scores that break down stats like routes in real time, but those disappear right after the game and as far as I know aren’t accessible after that (the game center after the games are over is different and very thin on useful data). We can’t get the raw stats either, just obscure models that are more black boxes. That’s one of the really cool things about PFF and the other sites doing similar like what Fantasy Points has spun up this year where they offer a ton of data, sortable and downloadable, so you can dig into it how you want. Oh, and NFLfastR for the coders. That’s been indispensable for the industry and is where I get most of the data for this column, with help from the awesome Sam Hoppen.
But we’re deep into the life cycle of Next Gen Stats, by data terms, and it’s still in like a beta testing mode on the NFL’s page. There are like 10-15 stats for each position, and again you can’t even search for players or find full data when you’re looking for something. The player needs to have qualified that week, or for the season.
And even among what they have, so much of it is just interesting more than useful. When I dig into new stats out of curiosity, I’m almost always underwhelmed. The latest was for RBs, I saw this “time behind the line of scrimmage” stat that I thought was something else at first, then I thought it might be a fun barometer of explosiveness, and then I looked at the results and quickly saw it was worthless. That was probably the intention of the stat, but you can look at the names at the top and bottom and realize quickly you’re getting a good mix of both types of backs.
For example, among the lowest times behind the line of scrimmage, we get backs like Latavius Murray who does run hard downhill and that makes sense, but then also some stuck-in-the-mud types who nonetheless do carry the ball up the middle a lot like A.J. Dillon. And it strikes me that when a guy like that runs straight up behind his linemen, stops, and then gets taken down a second later, the ball never crosses the line of scrimmage but he also wasn’t behind it for very long.
On the flip side, we have some of the most explosive backs in the league, because of the effectiveness of the outside zone scheme, and how they are running stretch plays designed to keep them behind the line of scrimmage moving somewhat laterally until they spot the right hole, plant their foot, and make their cut upward. Jerome Ford is slowest in the league, Kenneth Walker is third, Raheem Mostert and Bijan Robinson are on the slow end — what they have in common is they are all in the top 15 of total Rush Yards Over Expected, but so too is Chuba Hubbard, the back who spends the least amount of time in the backfield on average. The back who spends the third least amount of time, Rhamondre Stevenson, is second from the bottom of RYOE with only Rachaad White (a middle of the pack TLOS guy) behind him. There are also, as you might expect, high TLOS guys at the bottom RYOE, like Tyler Allgeier (who helps clarify that it’s Atlanta’s rushing system that leads to the high TLOS marks for both their backs, and the RYOE is probably better at explaining something useful about the two backs).
The point is for this ostensibly interesting stat, we quickly get back to the 50/50 issue, where yes there are probably insights that are captured, but if we can’t figure out which are which, what’s the use of the model? I’ll actually answer that rhetorical. The model is not doing anything for me; I’m making the determinations of what to believe based on what I have seen in the games, and pairing that with the traditional stats like the most-lambasted of all, yards per carry (which I promise is a better stat than this TLOS “Next Generation” stuff).
And to be clear, I’m not criticizing data people for trying. We absolutely should be trying to build new stats that are useful! But there’s a dynamic where people use the data wrong, and also when it’s not great and there’s no real way to roll it out any further, you sometimes wind up in a situation like the Next Gen Stats or ESPN WR model stuff where we’re only getting a sneak peak and we don’t really know anything about what’s going into it or what is being reported to us. Suddenly, instead of it being a tool, we have to think critically about whether we’re being duped (similar to how I had to parse what might be happening with that TLOS data set, which I think I probably have right in that there’s a ton of influence from the gap the run is designed to go through, and probably there’s no way to account for that because stuff like outside zone is all about patience until the moment to explode upfield, so you can’t even build in run gap into some model of efficiency, or I suppose you could but now we’re just working toward RYOE, which by the way is a black box dataset I do like and find useful, largely because it was devised during the Big Data Bowl while looking for ways to predict rushing success based on player tracking, and it was found that the RB didn’t influence this model, which meant that it was based purely on the blocking scheme, players in the box, distance to the ballcarrier, etc., and the RB’s performance could be evaluated outside that, although even that has been criticized as faulty in a great tweet thread I can’t find at the moment but which I know I’ve written about and basically argued you can’t apply the whole remainder of unexplained variance from a model to one factor, in this case the RB, and call it “over expectation,” which over expectation stuff is some of our best stats and still that’s a great point that they are probably wrong and making too much of an assumption that the “expectation” model itself captured all relevant factors other than the one they are assigning the remainder to).
(I’m sorry that was such a ridiculous parenthetical aside, but it’s also the exact point. This is why I am making the same claims about how difficult this stuff is that misguided film bros resort to when defending their bad takes.)
Broadly, if you’ve been around this space for any length of time, my question to you would be how many more models and new data sets have you been introduced to as groundbreaking than ones that actually stuck? (And some of the ones that stuck, by the way, are being frequently tweaked.) And the answer to that is obviously that there are overwhelmingly more that have been introduced than those that were maintained and have been able to make great predictions over time, which does not say that those models do not exist, but that they are difficult to find in a diluted and noisy part of this space where an extremely high percentage of unique data or models is always in that “beta mode” and never actually gets out of it.
(To find data actually worth listening to, you should focus more on the model builder and the process probably than anything else. Like I implicitly trust everything Mike Leone does, because I know from first-hand experience he’s rigorous and focused on accuracy and correctness above notoriety or those things. But even focusing on the model-builder can be tricky, because in cases like the ESPN WR model, I would presume the people in charge of that, who I have a ton of respect for, would discuss it differently than we’ve seen them discuss it publicly this week on social media.)
The other thing heavily emphasized here is whether you trust the data/models or the analyst interpreting it. And that’s the other way to get this right, is to find analysts who understand what they are actually looking at, or at least use the data with enough hesitancy and respect for uncertainty that it can be effective.
For example, my buddy Pat Kerrane is incredible at weighing all the relevant data points. It’s why I respect him so much for his prospect takes. It’s why when he won $2 million last year I told anyone who would listen how deserving he was, and I kept finding myself calling him a “grinder,” but then sort of regretting that because it sounds almost derogatory to imply he just does grunt work to get ahead or something. Pat’s superpower is an ability to pull from all the tools on the analytical toolbelt (to steal that analogy from another great at this skill, Rich Hribar). So when he dives into a new stat like first-read targets, which I’ve come out against around here, I actually care what he has to say about it, because he isn’t making that new stat the sole focus. He’s comparing it to every other stat he already uses, and probably drawing in some useful stat from five years ago I have to think back to our RotoViz days together to remember because I’ve long since forgotten it, and then because he’s grounding that stat he’s figuring out how to use it. Pat’s been using first-read targets in his great Walkthrough since last year, and he very clearly has a grasp on what that dataset can mean when he looks at unique profiles where it has something to offer to his analytical process. Crucially, he’s not relying on it for every single piece of analysis.
That’s a really unique skill. For a lot of analysts, including myself, there’s a process you develop and you go back to. You kind of have a lane. I try to do what Pat does, but I’m definitely a little too reliant on my core stats (HVT for RBs, TPRR-based stuff for WRs/TEs). In fairness, I landed on those because I felt like they were adequately simple and also useful, such that they were reliable launching off points for the profile. So I would defend my own process as being pretty strong in its own right, particularly how I’ve used TPRR over the past few years to — I believe — really great success identifying WR ranges of outcomes with more precision than I ever had. (That doesn’t mean I always play them perfectly when considering cost, and more on A.J. Brown as an example there below.) Despite feeling like I can focus on the key points of the profile well enough to make stronger predictions, I do somewhat frequently wish I’d have incorporated some fringe analysis in a specific profile, and that’s the type of thing guys like Pat and Rich don’t miss.
Anyway, I say all of that because I truly do care about the first-read target data set when Pat uses it, and yet I feel like I’ve come around to a strong understanding of what my deal with that dataset is, and I more or less think everyone (not just Pat) is mistakenly trying to claim it is something that it is not. I’ve written this before, but something like three-quarters of all WR/TE targets fall into this “first read” data (with a significant difference on that specific ratio between PFF and Fantasy Points, which gets to part of the point that this stuff is difficult to quantify, but the higher number, from Fantasy Points, was up at a whopping 88%).
That link goes to a FP article about first-read targets, which has a ton of good clarifications about what is being charted, and is generally a really good piece from a young writer named Ryan Heath who I’ve been impressed with and don’t want to criticize too much here, and I’ve frankly hesitated to even go into this over time. But I will just add I also think in addition to a ton of great stuff, there are also some misunderstandings of the dataset and why it’s useful in that piece (even including a claim this was a never-before-used stat when someone as prominent as Kerrane was using it in an article at NBC last year).
As an aside here, a really tough thing about the fantasy industry is you even mildly criticize and it can become a huge thing, which is ridiculous. There’s a broader issue with data and models where we expect the person who built the model to understand every cause and effect within their model, and what the algorithm is doing and all those things. Ryan writes a whole piece here where he gets a ton of stuff right and then there are some things I want to politely disagree with, and yet I know this industry well enough to know that’s going to be seen as “coming at” someone. That’s just an impossible bar to be holding Ryan to, first of all, and it creates a lack of growth and iteration in the space because people can’t take criticism since everyone needs to be an “expert” in a field that is riddled with uncertainty anyway, or else suddenly you have no value because there are 500 other analysts waiting in the wings who will claim to be experts and you gotta keep up with the Joneses. I digress.
As Ryan writes, this is the quarterback’s first read (because if you think about charting, that’s all you can really know). But to get back to the idea that watching the games and understanding the sport needs to be a huge part of data analysis, when we start to split up target data, we have to keep in mind there are going to be a lot of buckets. It strains credulity to argue 88% of targets are first reads, at least in terms of an assumption that’s how the play was drawn up.
This is where I cite some prior points from the last time I wrote about this about there being combo reads/half-field reads, presnap reads, option routes based on defensive leverage, etc. I am also not the person to explain all of this in the finest detail. But any claim on the pass-catcher’s side that this huge subset of targets is fully indicative of offensive intent at the play-calling level is a bridge too far. Yes, there’s probably some of that within the data set. This data will capture true first-read plays where the play was drawn up for a guy, called at the right time where it got the right defense, and executed such that it went to that guy.
But it will also capture plays that were drawn up for one guy where a safety rolled over the top and the quarterback made a presnap adjustment to his progression. Or plays where the whole point is the presnap read and your progression will be different based on alignment. We are just massively ignoring the quarterback’s influence on this dataset when we claim three-quarters or more targets are dictated at the play design level. That’s how this stat keeps getting talked about, and that’s crazy.
To cut to the chase, this “strong understanding” I’ve come to is: This dataset isn’t useful because of what’s in it, but that sliver of targets it removes. One of the things Pat has done research on (and talked to me about, which helped me here) is that the first-read dataset is more predictive than overall target share in smaller samples, but in larger samples that shifts back to where you’d just prefer target share. (This is based on his findings based on the parameters he used to research that, but there are people who might argue the opposite based on different parameters.)
That’s because the 12% of targets that are removed when we get to an 88% dataset that we’re calling “first reads” are the clearly noisiest of all targets. It’s the longest extended plays, scramble drills, and clear third and fourth option in progressions, when a QB is coming back across the field from his half-field read to throw the backside dig or whatever.
One of the things Ryan discusses in his piece is the QBs who have a higher or lower percentage of first-read throws than others. And among those who are highest, there are both some young/bad QBs like Zach Wilson and then veteran/good QBs like Tom Brady. He gets to what he calls a dual hypothesis of QBs ranking high in first-read throw rate if they either have trouble processing or are capable of leading a well-schemed offense. But then he notes year-to-year QB rates aren’t particularly sticky.
And I think this is just great evidence of how this dataset is pulling in a bunch of different things. The Bradys of the world are ranking highly in that stat not necessarily because they are in a well-schemed offense, but because of their own ability to process starting at the presnap level. Part of the difficulty of presnap reads is you have 40 seconds max, right? This is why Aaron Rodgers loves to drain the play clock. These vets are getting the call, relaying it, approaching the line with thoughts already formulating, then making reads, calling out hot routes, and deciding right at the line of scrimmage what their “first read” is going to be. It might be the offense; it’s far more likely to me that it’s the quarterback.
On the other side of the equation, guys like Zach Wilson probably do rate highly for the reasons Ryan says, which is to say that they can’t do much more than announce the play to the team, get to the line, take the snap, and throw the ball in the direction the play said was the first read. That may still be a “high-low” half-field combination read, but they aren’t getting to their backside receiver.
But the fact that the year-to-year rates aren’t sticky probably just goes further to the point that this isn’t one stat but rather a big subset of all targets, with the noisiest among those removed. It also probably gets to the difficulty of charting these plays, which I noted before different sources are finding very different rates of the number of plays that are truly first reads.
And I guess that’s a key point I should emphasize — I don’t think this is mis-charted, we’re just never going to know what the called play was, what the route progression was supposed to be, and what checks the QB made. Now obviously people charting this do understand schemes and are going to be able to make the most educated guesses imaginable. Former NFL QB J.T. O’Sullivan does great QB breakdowns on his YouTube channel The QB School; I wish I had time to watch these a lot more, but I remembered a moment in one of the first plays from a video way back in Week 3 that also noted you have an “alert” route on some plays. Here’s a screenshot where he’s drawn where the primary routes are going, and he says, “You’d love to read this…” and goes from “alert” to “1” to “2”.
The points I’m making are 1) even O’Sullivan, who clearly has a better grasp on this than me, says, “You’d love to read this…” which acknowledges that your read might change in some circumstances based on defensive alignment or other factors, and 2) the alert should only get the ball here under specific criteria, but for charting purposes this is one of those combo reads where either the alert or the first read is going to be charted as a first read.
It’s even possible that the guy marked “2” gets a first-read target on this play, as I’m pretty sure it was Peyton Manning on a recent Manningcast breaking down a presnap read where he determined the backside slant was getting the ball based on presnap alignment. Remember, the QB knows the play! When he gets to the line, it’s possible he might just throw out his frontside reads (the alert and “1”) and lock into the “2” (I’m not saying that would be likely, especially not on this particular play where the backside route is a slower-developing deep in route).
The above example is actually a pretty simple route. Consider now the bunch concepts where you have three routes on the same side of the field, and trying to guess at what the true progression was, and specifically based on how the defense aligned, which probably changes the first read out of a bunch just based on leverage. The point is the charter is not in the huddle, and more importantly not in the QB’s brain. He doesn’t know what the QB has been taught, and what he’s doing with that information from the coaches as he reacts on the field. (Ryan does a great job of laying out all of this context in his piece, by the way.)
There is a ton of gray area; the majority of NFL plays look like the one I just highlighted from O’Sullivan above. It is not the case that playcallers are designing throws to specific first reads during the week, then calling them on Sundays, and then they are thrown to those players more than two-thirds of the time, and it’s all that simple of a chain of events that signifies intent the way the concept “first-read targets” implies.
What the “first-read” dataset winds up being is a high percentage of all throws, to the extent that it’s really just a focused version of target share which removes the small subset of the highest-variance extended plays and clear “progression” plays where we see the QB move from the frontside of the play to the backside, but a point I’m making here is a quarterback’s progression is rarely as obvious as those examples (e.g. he’s going through multiple reads on the same side, not adjusting his head or body, and it will look like a first read).
That’s at least as far as I see it. And from that perspective, yeah it makes sense that “first-read” data is better than full target share in small samples, because we’re wiping some of the highest-variance stuff out of that. But we’re still leaving behind a huge percentage of the whole dataset, not just plays drawn up for a specific WR as the first read.
If you can take a dataset and find a way to remove the noisiest 12% of that data, you’re probably going to find that it improves predictiveness. You might not if you remove 50% of the data, because the sample size might get to small. We know splits data is often worse than a larger dataset. But if it’s just 12% and you’re focusing in on the best 88% of the data? Feels like the sweet spot.
And probably the reason Kerrane found that in larger sample sizes the overall target share matters more is that the “12%” sliver of targets normalizes to where good players who earn a lot of targets also wind up earning the targets on extended plays when that sample gets large enough.
What I’m arguing is there’s a ton of gray area between true first-read targets and broken plays (tons of plays like the one I highlighted from O’Sullivan above), and this dataset claims to be true first-read targets, but is actually first-read targets plus all the gray area and is really just removing the broken plays. And since we’re talking about broken plays, there is a little chaos and noise there, but even that stabilizes to where you’d just prefer to know who earns targets, full stop, in a big enough sample.
To be clear, this doesn’t actually matter re: the dataset being useful. People like Kerrane, who is so good at centering a piece of data like first-read targets alongside actual target share and everything else, is able to implicitly identify when that 12% (or whatever number) is due for regression when he emphasizes that a player’s first-read rate is way too far out of line with his actual target share. He is finding actionable conclusions in specific instances.
But these points do matter in the context of how we build up datasets to be things they are simply not. First-read targets can be used effectively, in just the ways I discussed, by sharp analysts like Kerrane. Otherwise, it’s probably just telling you what target share already does, and it’s not some new superdrug.
But because everyone in this industry is always on the hunt for the next big stat without stopping to think about how football is actually played, we wind up in these weird spots with data that is useful but is also being massively overemphasized because someone found some higher r-squared but doesn’t understand what the data he’s looking at even is. And to be clear, I have switched at this point to talking about data generally; I am not criticizing anyone specific in this situation, and I specifically chose first-read data to make this point because it’s one of the more complicated things that’s been introduced to the space in a long time.
But I mean if I did want to make a commentary, I might point out how we assume that the person who is talking the most or the loudest about something must have done the legwork and know the most. But that’s another unfair assumption! Fantasy Points for example is doing great work on first-read targets. Me arguing their commentary that “behind most targets is some amount of intention on the team’s part” overstates the “first-read” name of the dataset doesn’t change that. Hell, it might even be the case that all that needs to be changed there is to say “team or quarterback’s part,” to account for stuff like presnap reads, because ultimately, yes, within the gray area I mentioned above, there is some amount of intent.
In fact, I saw some really cool stuff this week on Twitter, and I’m sure this will get talked about more and I’ll have thoughts on it, where team No. 1s seemed to do very well against single-high defensive looks, whereas targets seemed to balance more against the two-high stuff. And I thought that was super interesting — maybe the two-high stuff forces QBs to throw to a specific place the coverage has dictated more, but in single-high they can target their best player. How that layers into this whole conversation about the first-read dataset is fascinating, because obviously the implication on some level is the play design and route didn’t even matter — it was the defensive alignment that dictated.
[Another implication here, which I’m not going to get to today, is that my whole spiel over the years not to think of it like there’s a No. 1 or a No. 2 WR on certain teams, because some No. 2s are really sort of just another No. 1 in their own right — that concept certainly gets challenged by data that argues a QB might lock onto an elite No. 1 (like an A.J. Brown) against a specific coverage look. There’s also an argument the RPO stuff I wrote about this summer plays in here, as Brown and Tyreek Hill were two of the most productive No. 1s in those specific single-high situations if I’m remembering the data correctly (and those are two heavy RPO offenses).]
Anyway, “first-read” targets aren’t the first dataset where the way we talked about it — the way we tried to conceptualize why it’s useful, to make sure we’re not being duped — maybe isn’t right. And I keep writing about it because the name “first-read” does seem to influence the way everyone talks about this stat.
There’s a related concept that led me down this road today, which I had in my notes as “pay attention to the analysis, not necessarily the conclusions.” I gotta be quick at this point because that last section was ridiculous, but there is a ton of good fantasy analysis out there where people don’t necessarily come to the right conclusions.