Analytics vs. Heuristics-Decision Making in the NFL

[D. L. Hot-Flamethrower]

For years and years, NFL coaches made decisions based almost completely on heuristics. For those unsure of what a heuristic is https://en.wikipedia.org/wiki/Heuristic. Essentially, it is a rule of thumb, or trial and error aiding in decision making. For example, you don't go for it from your own 18 on 4th down, early, and in a close game.

 

Analytics came along and it including looking at your chances of winning a game if you went for it on 4th down much more. Initially, this challenge to the old line approach proved to be correct because coaches had evolved into punting way to often in opponents territory (Think Doug Marrone punting from the 32).

 

I would argue that analytics has been taken too far recently and is being misused by some coaches. I would also argue that some of the current decisions (think Brandon Staley) are just recklessly applied analytics. For one thing, the downside of this type of overly aggressive decision-making, the rewards (keeping the ball) vs. the downside risk (giving the opponents points) cannot possibly be worth it.

 

My current opinion on the situation, which I welcome all of your opinions, is that there is something to be learned from certain analytics. Yes coaches had become too conservative and needing to be pushed to go for it more. But, there is much more to it for consideration by the decision maker. I also do not believe some of these situations coaches go for it(Staley); are using data that accurately represents THIS situation. I question how many times in history some of these situations have occurred to have been studied properly. Apologize for the length. What say you?

[prissythecat]

Not sure how analytics played a role in Staley’s decision to call time out ? Seemed more like a gut feel decision. 

[BillnutinHouston]

I think the answer depends on what you are trying to decide.

 

Mike Schopp, an analytics devotee, would never have selected Josh Allen because the analytics argued (nay, screamed) against it.  The problem is, analytics can't measure or predict heart, commitment and the impact of vastly improved coaching.

 

The outcomes of in-game decisions, I think, are greatly improved by knowing the numbers/odds of doing certain things in certain situations.

 

In short, both analytics and gut have a place in practically every decision, but both have a place at the table.

Edited January 11, 2022 by BillnutinHouston

[GunnerBill]

My big issue with it is that the use of prior success rates creates an artificial slant towards aggression in the models. 

 

Teams with good offenses go for more 4th downs. And because they go for more 4th downs with their good offenses they convert them at a decent rate. That factors into the model going forward. 

 

If every team, including those with bad offenses, went for every 4th and let's say 3 or less, the conversion rate of 4th downs would fall and it would affect what the model says about whether to go for it. 

[D. L. Hot-Flamethrower]

Just now, prissythecat said:

Not sure how analytics played a role in Staley’s decision to call time out ? Seemed more like a gut feel decision. 

Yeah I wasn't thinking about that, more the fourth down from the 18. I believe he said he wanted to get his top run defenders on the field and that was why he called the ill-fated time out.

1 minute ago, GunnerBill said:

My big issue with it is that the use of prior success rates creates an artificial slant towards aggression in the models. 

 

Teams with good offenses go for more 4th downs. And because they go for more 4th downs with their good offenses they convert them at a decent rate. That factors into the model going forward. 

 

If every team, including those with bad offenses, went for every 4th and let's say 3 or less, the conversion rate of 4th downs would fall and it would affect what the model says about whether to go for it. 

I'd also add that if your defense is as soft as a team like San Diego that must be considered.

 

I also don't think that just looking at win probability of a given situation should be the default for a decision.

[Big Turk]

21 minutes ago, D. L. Hot-Flamethrower said:

For years and years, NFL coaches made decisions based almost completely on heuristics. For those unsure of what a heuristic is https://en.wikipedia.org/wiki/Heuristic. Essentially, it is a rule of thumb, or trial and error aiding in decision making. For example, you don't go for it from your own 18 on 4th down, early, and in a close game.

 

Analytics came along and it including looking at your chances of winning a game if you went for it on 4th down much more. Initially, this challenge to the old line approach proved to be correct because coaches had evolved into punting way to often in opponents territory (Think Doug Marrone punting from the 32).

 

I would argue that analytics has been taken too far recently and is being misused by some coaches. I would also argue that some of the current decisions (think Brandon Staley) are just recklessly applied analytics. For one thing, the downside of this type of overly aggressive decision-making, the rewards (keeping the ball) vs. the downside risk (giving the opponents points) cannot possibly be worth it.

 

My current opinion on the situation, which I welcome all of your opinions, is that there is something to be learned from certain analytics. Yes coaches had become too conservative and needing to be pushed to go for it more. But, there is much more to it for consideration by the decision maker. I also do not believe some of these situations coaches go for it(Staley); are using data that accurately represents THIS situation. I question how many times in history some of these situations have occurred to have been studied properly. Apologize for the length. What say you?

 

The problem with only using analytics is the sample size for a team in a given season.  For instance, it MAY statistically be the right move to go for it on 4th down many times, but that is over a huge sample size.  During an individual season, the sample size is extremely small meaning there is a chance it won't work out to those percentages, similar to if you flipped a coin 30 times.  Sometimes you might get 20 heads or 25 heads instead of 15/15.  Over a huge sample size of like 10K flips, it would be very close to 50/50, but over a smaller sample size there is a lot more variability. With teams only going for it on 4th down maybe 20-40 times a year, the variability is much more pronounced. So while it could be the right decision analytically to go for it, it could also cost a coach his job if he has a string of bad luck/variability due to small sample size and the gamble doesn't pay off.

Edited January 11, 2022 by Big Turk

[D. L. Hot-Flamethrower]

https://www.forbes.com/sites/liamfox/2021/08/12/how-the-nfl-uses-analytics-according-to-the-lead-analyst-of-a-super-bowl-champion/?sh=759e4b5424e6

 

To be fair, analytics means much more than just going for it on fourth down of course. This article talks about Doug Pedersen and how his decisions to go for it on 4th down and 2 pointers likely helped their SB run. I think being receptive to new ways of thinking (innovation) can give you a short -term advantage. I also think not giving adequate weight to experience can be deadly.

2 minutes ago, Big Turk said:

 

The problem with only using analytics is the sample size for a team in a given season.  For instance, it MAY statistically be the right move to go for it on 4th down many times, but that is over a huge sample size.  During an individual season, the sample size is extremely small meaning there is a chance it won't work out to those percentages, similar to if you flopped a coin 30 times.  Sometimes you might get 20 heads or 25 heads instead of 15/15.  Over a huge sample size of like 10K flips, it would be very close to 50/50, but over a smaller sample size there is a lot more variability. With teams only going for it on 4th down maybe 20-40 times a year, the variability is much more pronounced. So while it could be the right decision analytically to go for it, it could also cost a coach his job if he has a string of bad luck/variability due to small sample size and the gamble doesn't pay off.

Excellent post! To have a sample size large enough for some of these situations they'd have to include ALL games and all teams otherwise as you say they'd be too small. And, as Gunner pointed out, the teams in question and they're abilities must be considered as well which reduces sample sizes.

[Evian]

The great coaches use instinct. The situations in the NFL are too different in each game to be tied to a certain way of decision making. 

[dave mcbride]

26 minutes ago, D. L. Hot-Flamethrower said:

For years and years, NFL coaches made decisions based almost completely on heuristics. For those unsure of what a heuristic is https://en.wikipedia.org/wiki/Heuristic. Essentially, it is a rule of thumb, or trial and error aiding in decision making. For example, you don't go for it from your own 18 on 4th down, early, and in a close game.

 

Analytics came along and it including looking at your chances of winning a game if you went for it on 4th down much more. Initially, this challenge to the old line approach proved to be correct because coaches had evolved into punting way to often in opponents territory (Think Doug Marrone punting from the 32).

 

I would argue that analytics has been taken too far recently and is being misused by some coaches. I would also argue that some of the current decisions (think Brandon Staley) are just recklessly applied analytics. For one thing, the downside of this type of overly aggressive decision-making, the rewards (keeping the ball) vs. the downside risk (giving the opponents points) cannot possibly be worth it.

 

My current opinion on the situation, which I welcome all of your opinions, is that there is something to be learned from certain analytics. Yes coaches had become too conservative and needing to be pushed to go for it more. But, there is much more to it for consideration by the decision maker. I also do not believe some of these situations coaches go for it(Staley); are using data that accurately represents THIS situation. I question how many times in history some of these situations have occurred to have been studied properly. Apologize for the length. What say you?

It was Gregg Williams who punted from the 32, not Marrone. Just an FYI. Atlanta basically punted from the 32 a couple of weeks ago too, although they took a five-yard penalty to move it back five yards. And of course, they kicked it into the end zone, netting 12 yards.

Edited January 11, 2022 by dave mcbride

[D. L. Hot-Flamethrower]

https://theathletic.com/3030201/2021/12/23/hate-nfl-analytics-based-decisions-the-math-adds-up-more-often-than-you-might-think/

 

Some examples where decisions that were analytics-based may have helped this season.

 

I'm not a pro-analytics or pro-old school. But, I do get a kick out of guys like Bradshaw and Long, everything is black and white with these tools.

3 minutes ago, dave mcbride said:

It was Gregg Williams who punted from the 32, not Marrone. Just an FYI.

Doug must have been close at least, especially in volume.

6 minutes ago, Evian said:

The great coaches use instinct. The situations in the NFL are too different in each game to be tied to a certain way of decision making. 

Yes I agree. I remember Belichick going  for it against Peyton Manning and the Colts years ago. If I remember correctly the score was tied and it was fourth down from about his own 25 and he went and got stopped. His reasoning? We weren't going to stop Peyton tonight!

[BuffaloRebound]

You still gotta go with your gut, but the analytics should still be used in coming to every decision.  The analytics tell you the baseline case, and then the coach has to factor in what’s happening on the field right now.  It’s like tweaking a model.  You need both.  

[dave mcbride]

10 minutes ago, D. L. Hot-Flamethrower said:

https://theathletic.com/3030201/2021/12/23/hate-nfl-analytics-based-decisions-the-math-adds-up-more-often-than-you-might-think/

 

Some examples where decisions that were analytics-based may have helped this season.

 

I'm not a pro-analytics or pro-old school. But, I do get a kick out of guys like Bradshaw and Long, everything is black and white with these tools.

Doug must have been close at least, especially in volume.

Yes I agree. I remember Belichick going  for it against Peyton Manning and the Colts years ago. If I remember correctly the score was tied and it was fourth down from about his own 25 and he went and got stopped. His reasoning? We weren't going to stop Peyton tonight!

The Pats player actually got the first (it was clear on replay) but got a terrible spot from the ref. Belichick couldn't challenge because they were out of timeouts. That was what was famous about that one.

[Matt_In_NH]

34 minutes ago, prissythecat said:

Not sure how analytics played a role in Staley’s decision to call time out ? Seemed more like a gut feel decision. 

4 seconds on the play clock is what I heard...that was a tough situation where both sides were wondering what the other was thinking.   I have not watched it in the real time flow but not sure he deserves all the criticism.

[mannc]

28 minutes ago, Big Turk said:

 

The problem with only using analytics is the sample size for a team in a given season.  For instance, it MAY statistically be the right move to go for it on 4th down many times, but that is over a huge sample size.  During an individual season, the sample size is extremely small meaning there is a chance it won't work out to those percentages, similar to if you flopped a coin 30 times.  Sometimes you might get 20 heads or 25 heads instead of 15/15.  Over a huge sample size of like 10K flips, it would be very close to 50/50, but over a smaller sample size there is a lot more variability. With teams only going for it on 4th down maybe 20-40 times a year, the variability is much more pronounced. So while it could be the right decision analytically to go for it, it could also cost a coach his job if he has a string of bad luck/variability due to small sample size and the gamble doesn't pay off.

This is the real drawback with football analytics: How reliable are numbers upon which you are basing your decisions? And as many have pointed out, every situation is different.  If your defense has been unable to stop a Pat Mahomes, then you’re probably going to be more inclined to go for it on 4th down to keep the ball away from him late in the game.
 

In my view, the best thing about analytics is that it has caused coaches to think about every decision in a fresh light, instead of automatically doing what every other coach has done since time immemorial.  

Here’s one I’d love to see more often: You score a touchdown and on the scoring play or the extra point, the other side gets a 15-yard personal foul that is enforced on the kickoff.  Why don’t you go for a surprise onside kick?  You’re kicking from midfield so the worst that happens is the other team gets it on their own 35…and the chances of recovering a surprise onside kick are very good.  But no one ever does it.

[GoBills808]

I think what you’re calling ‘analytics’ is just the process by which NFL coaches are attempting to capture as much EV (or EPA, W% etc) in a given situation.

 

It’s a well defined mathematical set by which one informs his or her decision making. It really can’t be taken ‘too far’.

[aristocrat]

Staley got reckless in that third quarter 4 down on their 20. That's a punt situation. 

[Mojo44]

There is definitely an over reliance on analytics in football. Several posters of already noted the small sample size in most of the statistics which significantly limits any predictive value. I would also like to see standard deviation measures in football analytics. I’m guessing the standard deviation will be quite high making it even more difficult to predict in a specific situation. A nomothetic approach doesn’t work with a small sample size. In this case an idiographic or situational approach is better. 
 

The only place in sports where analytics seem to work well is in baseball regarding the infield shifts. And this is only because most hitters are unable or unwilling to change how they hit. 
 

also, keep in mind that when a coach makes a decision based on the situation and not on analytics this is not just using his “gut”.  These decisions are usually based on accumulated empirical information gained from experience.
 

 

[Albany,n.y.]

52 minutes ago, BillnutinHouston said:

I think the answer depends on what you are trying to decide.

 

Mike Schopp, an analytics devotee, would never have selected Josh Allen because the analytics argued (nay, screamed) against it.  The problem is, analytics can't measure or predict heart, commitment and the impact of vastly improved coaching.

 

The outcomes of in-game decisions, I think, are greatly improved by knowing the numbers/odds of doing certain things in certain situations.

 

In short, both analytics and gut have a place in practically every decision, but both have a place at the table.

I thought of the draft too when I saw the OP.  The problem with people using analytics in player evaluation is they're applying statistics to performances that are much better understood by looking at film.  The analytics people in the media predicted Josh's future with a bunch of numbers & maybe watching a few clips or remember watching some games.  The Bills watched every play on film, were at at least 3 workouts (Combine, in Wyoming, in Orchard Park), had multiple meetings with him and talked to his coaches.  By watching film or going to a game, they were able to put his numbers in the proper context.  One of the things Beane mentioned after drafting Josh was a game played in horrible weather where Josh's statistics looked bad on paper, but on the field he willed the team to victory.  Analytics without a visual review of the games is not a viable way to scout players.  Unfortunately, the so called draft experts in the media who criticize a high pick without doing the due diligence that the drafting team does will never really get it.  It took Schopp years to figure Josh was the real deal.   

[mannc]

3 minutes ago, aristocrat said:

Staley got reckless in that third quarter 4 down on their 20. That's a punt situation. 

True, but I’m not sure that was an analytics-based decision…

[aristocrat]

4 minutes ago, mannc said:

True, but I’m not sure that was an analytics-based decision…

 

You have an 80 percent chance of converting the fourth down I believe is what I heard from the nerds. 

[stuvian]

owners are using analytics to justify firing coaches. The Eagles Jeff Lurid (no sarc) questioned Doug Pederson's play calling all during Carson Wentz' meltdown on the way out the door. I can see the day coming when these alpha male owners decide to use algorithms to call the plays themselves. If Jerry Jones can be a GM why not coach too? After all, rich people know everything, right?

 

The Browns have gone all in on Moneyball. Maybe they can win the Superbowl on Madden. 

 

Analytics cannot account for the human factor and hidden intel on injuries etc. It can tell us part of the story but not the whole story.

[Albany,n.y.]

5 minutes ago, aristocrat said:

Staley got reckless in that third quarter 4 down on their 20. That's a punt situation. 

I had futures on the Chargers this year (pushed the over 9, lost the over 10.5) and I watched more Chargers games than what I'd usually see.  Staley has made some pretty bizarre decisions this season, like going for 4th down deep in his own territory, going for it on 4th down instead of kicking field goals etc.  I understood early in the season when he had a kicker he didn't trust, but when they improved the position he still did the same things.  It was very frustrating watching him cost a game by being so unconventional.  By the time they played the Raiders I was pretty convinced I couldn't trust him to get me a 10th win & I almost expected him to blow it, which I believe he did with the crazy 4th from their own 20.  .  

[aristocrat]

3 minutes ago, Albany,n.y. said:

I had futures on the Chargers this year (pushed the over 9, lost the over 10.5) and I watched more Chargers games than what I'd usually see.  Staley has made some pretty bizarre decisions this season, like going for 4th down deep in his own territory, going for it on 4th down instead of kicking field goals etc.  I understood early in the season when he had a kicker he didn't trust, but when they improved the position he still did the same things.  It was very frustrating watching him cost a game by being so unconventional.  By the time they played the Raiders I was pretty convinced I couldn't trust him to get me a 10th win & I almost expected him to blow it, which I believe he did with the crazy 4th from their own 20.  .  

 

I could understand on your own 40 but they were on the 18.  That's a free 3 points and maybe a td.  Free points when you're already down.  

[hondo in seattle]

I think analytics are great.  But analytics themselves are a bit of a fancy heuristic.  Analytics are an attempt to take something infinitely complex and boil it down to simple numbers.

 

And you can't do that.  You can't quantify everything.  

 

For example, not all 4th and 1s are equal.  Maybe the right guard has a sore hammy.  Maybe your OL really needs a confidence builder.  Maybe that part of the field is slippery.  Show me a 4th and 1 chart that accounts for all the human, situational, and environmental variables?  There isn't one.  So the coach needs to make the final decision.   Analytics can help him make a better decision but shouldn't make the decision for him.

[Sweats]

Hey, if it were up to me, there'd be zero punts, no FG's and every team would go for it on 4th down.....regardless of score or field position.

It just makes the games more tolerable to watch sometimes.....sitting on the edge of your seat waiting to see if the team makes it on 4th and 22 is spectacular in my books, man.

[Mat68]

Anything that is absolute or not challengeable is a problem.  Staley going for it just because thats what they do is oppositely as stupid as always punting or kicking.  Nuance, is the key to coaching.  Every situation is different.  You cannot act like they are all the same in a vacuum.  I think Mcdermott isnt a perfect example but pretty close to what you would want.  In 17 with a pedestrian offense he was very conservative.  That resulted in a miracle playoff appearance.  Last season Buffalo was one of the more aggressive 4th down teams in football.  Different teams, different gambles.  For every try improving the odds of victory the failed attempt improves the odds of defeat.  Yes LAC were 4/5 on 4th down.  The 1 miss was when the outcome was much closer to even. The last 4 they made,  the team had no choice but to go for it because the game would be all but over.    

[GoBills808]

A good (mathematically sound) decision is a good decision regardless of outcome. 

[Rochesterfan]

2 hours ago, GunnerBill said:

My big issue with it is that the use of prior success rates creates an artificial slant towards aggression in the models. 

 

Teams with good offenses go for more 4th downs. And because they go for more 4th downs with their good offenses they convert them at a decent rate. That factors into the model going forward. 

 

If every team, including those with bad offenses, went for every 4th and let's say 3 or less, the conversion rate of 4th downs would fall and it would affect what the model says about whether to go for it. 

 

I would add things like situation sneed to be factored in many scenarios. 
 

For example - the 2 point conversion - many times the 2 point conversion is attempted by a team down multiple scores - late and the defending team don’t put up the same defense as what the Ravens saw when trying to win with the 2 point conversion and failing twice.

 

So overall from an analytical point of view - the 2 point conversion is >50%, but the numbers that occur late in specific situations is lower.

 

The same thing happens with 4th downs.  The numbers get impacted by situations like later game up by multiple scores and you are slowly letting a team move because time is more important than score/yards.

 

I think a conservative coach - like Fangio was - costs his team wins, but an overly aggressive coach can do the same thing.  Doug Pedersen used aggression to win a Super Bowl, but then that same approach cost him multiple games in the next couple of years when the offense was not as good.

 

 

Edited January 11, 2022 by Rochesterfan

[Rochesterfan]

1 hour ago, aristocrat said:

 

I could understand on your own 40 but they were on the 18.  That's a free 3 points and maybe a td.  Free points when you're already down.  

 

That is exactly the thing - even if the conversion rate is 80% - you are not in scoring range.  From the 20 - you still have a below average score rate.  
 

The risk of giving up immediate points - versus maybe getting 5 more first downs to score yourself has to be terrible.

 

The only way it makes sense is if you have a weather impact - going into the wind and want to maintain possession and not punt that direction.

 

 

[GunnerBill]

1 hour ago, hondo in seattle said:

I think analytics are great.  But analytics themselves are a bit of a fancy heuristic.  Analytics are an attempt to take something infinitely complex and boil it down to simple numbers.

 

And you can't do that.  You can't quantify everything.  

 

For example, not all 4th and 1s are equal.  Maybe the right guard has a sore hammy.  Maybe your OL really needs a confidence builder.  Maybe that part of the field is slippery.  Show me a 4th and 1 chart that accounts for all the human, situational, and environmental variables?  There isn't one.  So the coach needs to make the final decision.   Analytics can help him make a better decision but shouldn't make the decision for him.

 

And as of the present time it is impossible to build a model that factors in the human and environmental variables. You can account for situational variables, and indeed the models already do. They account for field position, they account for scoreboard, they account for time on the clock. The advanced analytical models really are amazing and you would be mad to discount them in the way the modern game works. However, the general models that networks etc use do have that inbuilt bias where they rely on past success rates that I outlined earlier. Indeed most NFL teams now have their own models for exactly that reason. Because when say Matt Nagy is deciding whether to go for a 4th a 3 with Andy Dalton the overall success rate of 4th and 3 plays from your own 48 when 7 points down with 8 minutes remaining that has been built in data from Tom Brady and Aaron Rodgers converting it a ton is of little use to him. Similarly if you have Justin Herbert converting 4th downs for fun your model probably doesn't care that Andy Dalton only gets 4th and 3 on 2 of 10 tries. So when you hear folks say things like "the number say you go for it there" those people have to be aware that teams are getting even more granular than a general model ever can. 

Edited January 11, 2022 by GunnerBill

[Cash]

3 hours ago, GoBills808 said:

A good (mathematically sound) decision is a good decision regardless of outcome. 

That’s technically true. But if your decisions keep leading to bad outcomes, how can you be sure they were good/sound decisions in the first place?

 

Not trying to accuse you of anything, but a lot of the analytics folks I’ve seen are deeply arrogant in a way that makes me question all of their conclusions. Their attitude seems to roughly be, “I’ve completely figured it out, therefore anyone who disagrees with my model is an idiot.” And every time they’re wrong, they come back with logic similar to the quote above. 
 

I have 2 main problems with these folks, plus a bonus third problem:

 

1.) How do they know their model is accurate? As GB & others pointed out above, the models are based on historical data. But as teams change their behavior, there’s no reason to expect that the models will continue to hold, or be predictive at a useful level. 
 

2.) Many of these folks seem to have no understanding of the concept of error. It’s maddening. I saw an article a few weeks ago, blasting a coach for a 4th down decision. Per the model quotes, the coach’s actual decision resulted in about 46.2% win probability, but the “correct” decision resulted in about 46.5% win probability. But there was no mention of the model’s margin of error. Off the top of my head, I can’t recall any NFL analytics people ever providing a margin of error. This is nuts. 
 

3.) Bonus: Arrogance. As I mentioned above, there’s a lot of “Everyone is dumber than me” energy coming from the current crop of analysts. This rankles me on a personal level, so I’ll admit some bias here. But I also think this is a legitimate issue. It ties in to both of the issues above: if you think you’re the smartest guy in the room, it’s often hard to self-evaluate. Much easier to make fun of “stupid” coaching decisions and tell your detractors that only process matters; never results. Plus, these guys are so high on their own farts that they can’t always interpret what their own models are saying. YardsPerPass pointed out an example on Twitter a while back. If you look at the Football Outsiders predraft QB write up in 2018, they lambaste Josh Allen and glow over Sam Darnold. (Nailed it.) But their own model only gave Allen about a 5% greater chance of busting than Darnold, and I think a pretty similar disparity in likelihood to break out. Based on the writeups, I would’ve expected Allen’s bust chance to be like 80-90%, compared to like 20% for Darnold. 

[Nextmanup]

Note that OP is using heuristics, effectively, in his arguments...and not math or logic.

 

Which is kind of funny.

 

It's this simple: as you approach a decision in a game, your choices have win probability associated with them.   We can use the benefit of thousands of games having been played before with similar situations to calculate what is the better approach, i.e., what has the highest probability of success. 

 

And better is better.  Period.

I think some fans confuse "better" with "guaranteed."

 

Analytics is NOT about calculating the winning play, it's about giving yourself the highest probability of success with a given play.  It can still go badly though.  

 

A coach can CORRECTLY choose to go for it on 4th down 10 times in a row, and not get the first down EVERY TIME and still be correct in his decision making (if the circumstances support that)....and he still had a better shot of winning the games by going for it all those times on 4th down than he would have by punting, even though they ALL went against him.  

 

Before arguing about that, think about it for a bit.

 

 

 

 

4 hours ago, Sweats said:

Hey, if it were up to me, there'd be zero punts, no FG's and every team would go for it on 4th down.....regardless of score or field position.

It just makes the games more tolerable to watch sometimes.....sitting on the edge of your seat waiting to see if the team makes it on 4th and 22 is spectacular in my books, man.

You just gave me a thought.  I wonder if, in 10 years, "punters" will no longer exist.

 

The idea being that punting is so rare, teams can't justify a roster spot for one.

Have the place kicker do it the best he can and move on.

 

Hell, that might be a good idea NOW.  Make better use of the roster spot.

 

 

[reddogblitz]

7 hours ago, Albany,n.y. said:

I had futures on the Chargers this year (pushed the over 9, lost the over 10.5) and I watched more Chargers games than what I'd usually see.  Staley has made some pretty bizarre decisions this season, like going for 4th down deep in his own territory, going for it on 4th down instead of kicking field goals etc.  I understood early in the season when he had a kicker he didn't trust, but when they improved the position he still did the same things.  It was very frustrating watching him cost a game by being so unconventional.  By the time they played the Raiders I was pretty convinced I couldn't trust him to get me a 10th win & I almost expected him to blow it, which I believe he did with the crazy 4th from their own 20.  .  

 

I was surprised he didn't go for it instead of kicking the FG to tie.

 

Its crazy. He's got a talented team but blows games with bizarre coaching decisions.

 

After they didn't get the 4th down from their own 18, the team fell off a cliff and muddled around for a quarter before getting too little too late at the end.

[GunnerBill]

2 hours ago, Nextmanup said:

Note that OP is using heuristics, effectively, in his arguments...and not math or logic.

 

Which is kind of funny.

 

It's this simple: as you approach a decision in a game, your choices have win probability associated with them.   We can use the benefit of thousands of games having been played before with similar situations to calculate what is the better approach, i.e., what has the highest probability of success. 

 

And better is better.  Period.

I think some fans confuse "better" with "guaranteed."

 

Analytics is NOT about calculating the winning play, it's about giving yourself the highest probability of success with a given play.  It can still go badly though.  

 

A coach can CORRECTLY choose to go for it on 4th down 10 times in a row, and not get the first down EVERY TIME and still be correct in his decision making (if the circumstances support that)....and he still had a better shot of winning the games by going for it all those times on 4th down than he would have by punting, even though they ALL went against him.  

 

Before arguing about that, think about it for a bit.

 

 

I agree with this but as I point out above teams are building their own models that balance the data over thousands of games with their players. Because the problem with the thousands of games argument is that there is an inbuilt bias. Until the last maybe 2 or 3 seasons the only teams who regularly went for 4th downs beyond 4th and inches were ones with good offenses and top quarterbacks and so they converted at a higher rate. That is the one thing clever teams are trying to balance against. If you have Justin Herbert or Josh Allen or Patrick Mahomes everything past your own 45 is potential 4 down territory (especially if it is 4th and less than 5). If you have Andy Dalton or Jared Goff that math is not the same. They will not convert at the same rate. 

[GoBills808]

3 hours ago, Cash said:

That’s technically true. But if your decisions keep leading to bad outcomes, how can you be sure they were good/sound decisions in the first place?

 

Not trying to accuse you of anything, but a lot of the analytics folks I’ve seen are deeply arrogant in a way that makes me question all of their conclusions. Their attitude seems to roughly be, “I’ve completely figured it out, therefore anyone who disagrees with my model is an idiot.” And every time they’re wrong, they come back with logic similar to the quote above. 
 

I have 2 main problems with these folks, plus a bonus third problem:

 

1.) How do they know their model is accurate? As GB & others pointed out above, the models are based on historical data. But as teams change their behavior, there’s no reason to expect that the models will continue to hold, or be predictive at a useful level. 
 

2.) Many of these folks seem to have no understanding of the concept of error. It’s maddening. I saw an article a few weeks ago, blasting a coach for a 4th down decision. Per the model quotes, the coach’s actual decision resulted in about 46.2% win probability, but the “correct” decision resulted in about 46.5% win probability. But there was no mention of the model’s margin of error. Off the top of my head, I can’t recall any NFL analytics people ever providing a margin of error. This is nuts. 
 

3.) Bonus: Arrogance. As I mentioned above, there’s a lot of “Everyone is dumber than me” energy coming from the current crop of analysts. This rankles me on a personal level, so I’ll admit some bias here. But I also think this is a legitimate issue. It ties in to both of the issues above: if you think you’re the smartest guy in the room, it’s often hard to self-evaluate. Much easier to make fun of “stupid” coaching decisions and tell your detractors that only process matters; never results. Plus, these guys are so high on their own farts that they can’t always interpret what their own models are saying. YardsPerPass pointed out an example on Twitter a while back. If you look at the Football Outsiders predraft QB write up in 2018, they lambaste Josh Allen and glow over Sam Darnold. (Nailed it.) But their own model only gave Allen about a 5% greater chance of busting than Darnold, and I think a pretty similar disparity in likelihood to break out. Based on the writeups, I would’ve expected Allen’s bust chance to be like 80-90%, compared to like 20% for Darnold. 

Calculating things that teams rely upon when making decisions in-game, like win% or EPA, shouldn't have a 'right' or a 'wrong' attached to it. It's just a formula that  informs your decision making. If you want to build a circular swimming pool and you calculate the diameter but then decide you'd prefer a rectangle, your initial measurement isn't wrong. You just didn't use it.

[LeGOATski]

"Kick it early, go for it late." Tried and true. 🙂

 

I think it certainly has gone too far with the amount of coaches going for 4th early in the game. It's really about whether or not you have the momentum. For instance, if your team just easily drove down the field in 10 plays and now it's 4th and goal, then go for it. You also have to consider how good your red zone offense is. However, if your team has struggled to get down the field and now it's 4th and short at the 35, take the field goal or try to pin them deep. If you wanna go for it, how good is your OL and RB? 

 

These decisions are pretty easy if you have your finger on the pulse of the game and your team, but too often we're seeing coaches go for it despite the factors I mentioned being unfavorable at the time. And I assume it's simply because "analytics said so."

[Mikie2times]

If I said you have a 65% chance of doubling your money and a 35% of losing it all, one outcome, like a coin flip, would you say yes? Maybe when your 20, probably not when your 60. That's the root of the problem with some of these guys. Staley looked at the decision as a +3% win probability in a binary way. Where another way to look at it, still from a statistical perspective, is you're electing to create a 17% swing in win% based on one play by going for it. Is that something worth doing midway through the 3rd quarter in a 3 point game? Further, is that worth doing vs a team that really wasn't doing much offensively without the refs help? Last part bringing more of the old school football mind into it. I really would like to think right answer exists in the middle of it all.    

[Generic_Bills_Fan]

Using analytics can’t really be evaluated on an individual play basis…it’s based on the probability of a play being successful but obviously the alternative could still happen.  Over the long term following the analytics should theoretically pick you up a win or two but obviously in a 60/40 should I go for this 4th down type situation you’re still gonna get it wrong a big chunk of the time and fans are gonna second guess you 😂

[GoBills808]

2 minutes ago, KzooMike said:

If I said you have a 65% chance of doubling your money and a 35% of losing it all, one outcome, like a coin flip, would you say yes? Maybe when your 20, probably not when your 60. That's the root of the problem with some of these guys. Staley looked at the decision as a +3% win probability in a binary way. Where another way to look at it, still from a statistical perspective, is you're electing to create a 17% swing in win% based on one play by going for it. Is that something worth doing midway through the 3rd quarter in a 3 point game? Further, is that worth doing vs a team that really wasn't doing much offensively without the refs help? Last part bringing more of the old school football mind into it. I really would like to think right answer exists in the middle of it all.    

So basically capturing 30% EV doesn''t convince you, but you're going to allow yourself to be informed by a difference of 17%? Where is the logic in that?

[Mikie2times]

4 minutes ago, GoBills808 said:

So basically capturing 30% EV doesn''t convince you, but you're going to allow yourself to be informed by a difference of 17%? Where is the logic in that?

In the example I stated, under a singular trial, I don't have enough trials to risk my life savings even with a 30+EV at the age of 60. That would be mathematically the correct decision, but in real life, I'm pretty certain even most statisticians would not take that proposal. In the situation Staley faced, I would rather extend the trials/plays I have with the Raiders because I feel I have the better team. Last thing I would want to do is introduce a huge amount of variance into the equation by doing things like going for it on my own 18 yard line even if it does come with a 3% EV.