Jump to content

Interesting chart showing Bills as 6th most intelligent 4th down decision-making team in 2018


Logic

Recommended Posts

15 hours ago, Logic said:

Stumbled across this on Twitter today.

In a nutshell, the chart shows how often teams made the decision on 4th down that matched what the New York Times 4th Down bot (which is built based on what decisions analytics suggest give a team the best statistical chance at winning) suggested. The higher you are on the chart, the more often you made a "good" choice on 4th down. This chart is just for the 2018 season. It shows the Bills as the 6th "smartest" or "most frequent makers of correct decisions" on 4th down.

I find it interesting because so many people claim that McDermott is an awful 4th down decision maker. At least one source, the one mentioned here, disagrees.

 

 

 

There it is again... Bill and Panthers...  

Link to comment
Share on other sites

15 hours ago, Logic said:

Stumbled across this on Twitter today.

In a nutshell, the chart shows how often teams made the decision on 4th down that matched what the New York Times 4th Down bot (which is built based on what decisions analytics suggest give a team the best statistical chance at winning) suggested. The higher you are on the chart, the more often you made a "good" choice on 4th down. This chart is just for the 2018 season. It shows the Bills as the 6th "smartest" or "most frequent makers of correct decisions" on 4th down.

I find it interesting because so many people claim that McDermott is an awful 4th down decision maker. At least one source, the one mentioned here, disagrees.

 

We ranked 31st (of 32 that would be) in converting 4th downs.  We were ahead of Rosen and Arizona.  

 

I'm not sure what "good decisions" means, but in terms off converting 4th downs, we sucked, royally.  

 

In other unfavorable news, we ranked 30th in Scoring %, that is the percentage of drives resulting in offensive scores. 

 

We ranked 29th in drives ending in offensive TOs.  

 

28th in average number of plays/drive. 

 

30th in average number of plays/drive. 

 

26th in average time/drive

 

30th in average number of points/drive 

 

 

1 hour ago, buffalobloodfloridahome said:

I think we as fans should realize last year was a time to test things out and see what works. Go for broke, because we knew we were outmatched on our oline and needed every opportunity to run game clock and keep the other offense off the field. This year we will see a much more normal play calling and risk taking as long as our offense has improved as dramatically as it should. We played for broke last year with low expectations and a rookie QB

 

Let's hope so, but it's still a very hefty climb to merely average.  

Link to comment
Share on other sites

this analysis is problematic. all it actually shows is how the team's decisions lined up with an algorithm. that's it. you cannot draw any more conclusions than that.

 

there is nothing that shows the bot is correct. maybe the bot is wrong 100% of the time. that would mean we are one of the worst decision makers (for example).

 

analysis is fun, but far too many people draw conclusions from it when there is no basis for it. decision making is far more complicated than making a decision tree and following it.

  • Like (+1) 2
Link to comment
Share on other sites

7 hours ago, MJS said:

Sure, but it's a pretty small sample size. Teams don't go for it on 4th down all that often. Differences between them are murky from the small sample size.

 

It's better to make the right decision in the long run. If your team isn't good enough, well that sucks and you'll lose games regardless of whether or not you punt it on 4th down.

 

That is correct. 

 

The range for 4th-down attempts runs from a high of 25 (Denver) to a low of 8 (Chargers).  

 

29 teams rank between 23 and 12 attempts/game.  That's on the magnitude of approximately 1/game.  

 

We had 18, converted 7.  

Link to comment
Share on other sites

9 minutes ago, dickleyjones said:

this analysis is problematic. all it actually shows is how the team's decisions lined up with an algorithm. that's it. you cannot draw any more conclusions than that.

 

there is nothing that shows the bot is correct. maybe the bot is wrong 100% of the time. that would mean we are one of the worst decision makers (for example).

 

analysis is fun, but far too many people draw conclusions from it when there is no basis for it. decision making is far more complicated than making a decision tree and following it.

 

Good points.

 

On analysis however, one has to know which analyses are relevant and which are not.  That's the problem with these metrics departments on teams these days, they overthink things and use irrelevant, at least relatlively speaking, metrics/data/analyses.  

 

Link to comment
Share on other sites

32 minutes ago, dickleyjones said:

this analysis is problematic. all it actually shows is how the team's decisions lined up with an algorithm. that's it. you cannot draw any more conclusions than that.

 

there is nothing that shows the bot is correct. maybe the bot is wrong 100% of the time. that would mean we are one of the worst decision makers (for example).

 

analysis is fun, but far too many people draw conclusions from it when there is no basis for it. decision making is far more complicated than making a decision tree and following it.

Correct - how do we know the algorithm is correct?  There are always potential bias built into the logic.  People assume the algorithm is right which is not true.

Link to comment
Share on other sites

47 minutes ago, Ronin said:

 

We ranked 31st (of 32 that would be) in converting 4th downs.  We were ahead of Rosen and Arizona.  

 

I'm not sure what "good decisions" means, but in terms off converting 4th downs, we sucked, royally.  

 

In other unfavorable news, we ranked 30th in Scoring %, that is the percentage of drives resulting in offensive scores. 

 

We ranked 29th in drives ending in offensive TOs.  

 

28th in average number of plays/drive. 

 

30th in average number of plays/drive. 

 

26th in average time/drive

 

30th in average number of points/drive 

 

 

 

Just the percentage of converting 4th downs doesn't tell you too much. You'd also need to know how difficult the situations were. Not converting a 4th and one with reasonable consistency is an indicator of inability. Not converting a 4th and 18 with consistency is more an indicator of how often you were so far behind in a game that you would be trying to convert a 4th and 18.

 

And yes, our offense was bad last year. Which has a lot to do with how very few resources we put into improving the offense the last two years (outside of bringing in Josh Allen). The unit we put a lot of resources into performed quite well last year. This year we had a ton of money under the cap thanks to their hard and painful work in clearing up the cap. And they put a lot of resources into the offense. I, personally, expect an improved offense this year, with the improvements to the OL in particular and the offense in general.

Link to comment
Share on other sites

31 minutes ago, Just Joshin' said:

Correct - how do we know the algorithm is correct?  There are always potential bias built into the logic.  People assume the algorithm is right which is not true.

 

 

It's very likely a smart algorithm indeed. It's been tried with great success. Yes, it was in high school, but the team using it have had terrific success. Look up Pulaski Academy. Their coach doesn't never punt, but almost never. He punted in his most famous game (there's a documentaray about it, against Highland Park). But it was 4th and 30 from their own 16. The math said punt, so he punted.

 

https://www.si.com/college-football/2015/09/09/why-college-football-coach-should-adopt-no-punt-always-onside-kick-philosophy

 

I wouldn't do what Kelley does, but I'd sure take more risks than most NFL coaches are willing to assume. It makes sense. NFL coaches are too risk-averse on this issue and a few others as well.

Edited by Thurman#1
Link to comment
Share on other sites

16 hours ago, TPS said:

The Bills ranked 31st in their 4th down conversion rate in 2018.  I think what this states is they were good at making the right decision on 4th and 5 or less yards.  

I think one of the reasons McD and Beane re-did the O-line is they did not win the line of scrimmage on most short yardage runs, with the exception of Allen and his sneaks.  I expect them to be much better with some of the beasts they brought in....

 

https://www.teamrankings.com/nfl/stat/fourth-down-conversion-pct

 

I think a lot of guys arent getting that this doesnt mean we were good at converting on fourth.  

 

 

 

  • Like (+1) 1
Link to comment
Share on other sites

4 minutes ago, Thurman#1 said:

 

 

It's very likely a smart algorithm indeed. It's been tried with great success. Yes, it was in high school, but the team using it have had terrific success. Look up Pulaski Academy. Their coach doesn't never punt, but almost never. He punted in his most famous game (there's a documentaray about it, against Highland Park). But it was 4th and 30 from their own 16. The math said punt, so he punted.

 

https://www.si.com/college-football/2015/09/09/why-college-football-coach-should-adopt-no-punt-always-onside-kick-philosophy

 

I wouldn't do what Kelley does, but I'd sure take more risks than most NFL coaches are willing to assume. It makes sense.

Patriots are middle of the pack.  I havent looked but I bet they are among the highest to convert.  I think whether you think your O has the ability to convert is the biggest factor in going for it.

Link to comment
Share on other sites

17 hours ago, Inigo Montoya said:

Hopefully with an upgraded offensive line to go with our athletic 6'5" -- 240 lb quarterback, we will have even more success on 4th and short.  It's also easier to go for it on 4th down when you know you have a great defense to help bail you out if you don't get it.

 

   QB keepers are all well and good.

   Having an o line that can line up on 4th and short and everyone knows the RB is getting it and you still win most of those situations.....?

    That is preferable.

 

Link to comment
Share on other sites

17 hours ago, Albany,n.y. said:

The main reason the Bills rank so high is because Allen on a QB sneak is automatic on 4th & a yard or less. 

 

They rank high because their decisions tended to match analytics optimums.   But those assume average offense against average defense based on the score and time remaining in the game or half.  Every team would then augment those baseline analytics with coaching decisions in cases when you have an all-pro punter or kicker, or if your short yardage offense is notably good (Allen 1 yard sneaks), game-time injuries (like to a kicker), weather, or other facts that make the particular circumstance deviate from "averages".

 

I wonder if there is any correlation to making close to "optimal" decisions based only on analytics and outcome.  Notably, Pats* were in the middle of the road.

Edited by Rock'em Sock'em
Link to comment
Share on other sites

On 6/21/2019 at 5:45 PM, Mr. WEO said:

 

 

Nope.  They were actually awful on 4th down conversions, even if the robot told them to "go for it".  

 

Next to last in conversions (an anemic 38.9%).  At some point, a HC has to say "we're not really very good at this, even if it seems like the right call"...and punt.

 

You can't just use 4th down conversion percentage and said they should just punt even the "right call" by the bot is to go for it. Let's look at each 4th down attempt in the season where Bills converted 7 out of 18 attempts (38.89%).

 

(1) Among the 18 attempts, half of them were when trailing near the end of the game or trailing by large score. You probably can still use punting argument in some of these plays, but not most of them when considering the score and situation.

  • 4th&8 at own 39, 14:02 3rd, tailing 0-26, failure (peterman incomplete pass)
  • 4ht&4 at opp 20, 10:59 4th, tailing 5-31, failure (Anderson 3 yard pass)
  • 4th&2 at opp 32, 3:52 4th, tailing 6-25, failure (Anderson incomplete pass)
  • 4th&18 at opp 43, 0:48 4th, tailing 6-25, failure (Peterman 9 yard pass)
  • 4th&1 at opp 33, 5:13 3rd, tailing 3-31, success (Peterman 1 yard run)
  • 4th&9 at opp 31, 2:36 3rd, tailing 3-31, failure (Peterman incomplete pass)
  • 4th&16 at opp 31, 2:00 4th, tailing 9-41, failure (Peterman sacked)
  • 4th&11 at opp 30, 1:05 4th, tailing 17-21, failure (Allen incomplete pass)
  • 4th&3 at Opp 25, 2:23 2nd, tailing 0-14, failure (Allen incomplete pass)

 

(2) Of the remaining attempts, 6 can be categorized as 4th & short as following. Bills converted 5 of them. Using overall 38.9% 4th down conversion to claim they should have punted in these situations is simply wrong consider their 4th&short conversion rate is actually high.

  • 4th&1 at opp 1, 12:14 2nd, leading 17-0, success (Allen 1 yard run)
  • 4th&1 at opp 38, 13:13 1st, 0-0, success (Ivory 3 yard run)
  • 4th&1 at opp 28, 7:40 2nd, leading 7-3, success (Ivory 2 yard run)
  • 4th&2 at opp 13, 3:19 3rd, leading 7-13, failure (Allen 1 yard run, fumble)
  • 4th&1 at opp 41, 1:50 4th, leading 14-13, success (Allen 2 yard run)
  • 4th&1 at Opp 8, 10:46 3rd, 14-14 success (Allen 3 yard run)

 

(3) For the remaining 4th down attempts,  one is the famous Bojoquez botched play in TEN game when he thought it's a fake. Coach didn't call for 4th down attempt and actually called for FG. Another one is a trick play where Logan Thomas executed a fake punt and converted the 4th down. Maybe they should have called for punting in that situation, but that's what trick play does with the surprise element.

 

The last 4th down attempt is Allen took a kneel in the end of the last game when leading 42-17. There is really no point to argue whether they should have punted there.

  • 4th&7 at opp 12, 5:03 2nd, leading 7-3, failure (Bojorquez incomplete pass, FG formation, TEN game, miscommunication)
  • 4th&6 at own 47, 7:26 1st, leading 7-0, success (L. Thomas 15 yard pass, punt formation)
  • 4th&4 at Opp 24, 1:21 4th, leading 42-17 failure (Allen -1 kneel)
Edited by syhuang
  • Like (+1) 2
Link to comment
Share on other sites

 

13 hours ago, formerlyofCtown said:

Patriots are middle of the pack.  I havent looked but I bet they are among the highest to convert.  I think whether you think your O has the ability to convert is the biggest factor in going for it.

 

 

I disagree. Ability to convert should be a factor, but IMO the second or third-most important.

 

Situation is most important. 

 

4th down conversion rates are statistically all over the place, due to the small sample size and the difference in likelihood of conversion in different situations, such as a 4th and 1 being far easier than a 4th and 18.

 

3rd down conversion rates are far more indicative of ability to convert, due to larger sample. But situation is still a huge factor. And 3rd down conversion rates go from 29% to 47% last year. There are none in single digits and nobody over 75%. They're gathered in a reasonably small range. Not to be too bad of a bore, but how much of the difference might be due to ... for example, only ... say, the Colts having an average 3rd down of 3rd and three while the Cards might have an average 3rd down of 3rd and five. Made up those numbers ... if anyone can be bothered to find the real ones, that'd be great. 

 

Point is, though, situation is by far the most important variable. Oh, and the Pats were 9th in 3rd down conversions. But 25th in 4th down conversions, at 50%, probably because they are a bit more situationally aggressive than most teams, and for good reason. Fewer times to need to make desperate conversion attempts, but more willing to try in non-desperate situations where the distance is a bit longer than most coaches would be comfortable with.

 

 

 

Nice post, Syhuang!!!

Edited by Thurman#1
Link to comment
Share on other sites

9 hours ago, syhuang said:

 

You can't just use 4th down conversion percentage and said they should just punt even the "right call" by the bot is to go for it. Let's look at each 4th down attempt in the season where Bills converted 7 out of 18 attempts (38.89%).

 

(1) Among the 18 attempts, half of them were when trailing near the end of the game or trailing by large score. You probably can still use punting argument in some of these plays, but not most of them when considering the score and situation.

  • 4th&8 at own 39, 14:02 3rd, tailing 0-26, failure (peterman incomplete pass)
  • 4ht&4 at opp 20, 10:59 4th, tailing 5-31, failure (Anderson 3 yard pass)
  • 4th&2 at opp 32, 3:52 4th, tailing 6-25, failure (Anderson incomplete pass)
  • 4th&18 at opp 43, 0:48 4th, tailing 6-25, failure (Peterman 9 yard pass)
  • 4th&1 at opp 33, 5:13 3rd, tailing 3-31, success (Peterman 1 yard run)
  • 4th&9 at opp 31, 2:36 3rd, tailing 3-31, failure (Peterman incomplete pass)
  • 4th&16 at opp 31, 2:00 4th, tailing 9-41, failure (Peterman sacked)
  • 4th&11 at opp 30, 1:05 4th, tailing 17-21, failure (Allen incomplete pass)
  • 4th&3 at Opp 25, 2:23 2nd, tailing 0-14, failure (Allen incomplete pass)

 

(2) Of the remaining attempts, 6 can be categorized as 4th & short as following. Bills converted 5 of them. Using overall 38.9% 4th down conversion to claim they should have punted in these situations is simply wrong consider their 4th&short conversion rate is actually high.

  • 4th&1 at opp 1, 12:14 2nd, leading 17-0, success (Allen 1 yard run)
  • 4th&1 at opp 38, 13:13 1st, 0-0, success (Ivory 3 yard run)
  • 4th&1 at opp 28, 7:40 2nd, leading 7-3, success (Ivory 2 yard run)
  • 4th&2 at opp 13, 3:19 3rd, leading 7-13, failure (Allen 1 yard run, fumble)
  • 4th&1 at opp 41, 1:50 4th, leading 14-13, success (Allen 2 yard run)
  • 4th&1 at Opp 8, 10:46 3rd, 14-14 success (Allen 3 yard run)

 

(3) For the remaining 4th down attempts,  one is the famous Bojoquez botched play in TEN game when he thought it's a fake. Coach didn't call for 4th down attempt and actually called for FG. Another one is a trick play where Logan Thomas executed a fake punt and converted the 4th down. Maybe they should have called for punting in that situation, but that's what trick play does with the surprise element.

 

The last 4th down attempt is Allen took a kneel in the end of the last game when leading 42-17. There is really no point to argue whether they should have punted there.

  • 4th&7 at opp 12, 5:03 2nd, leading 7-3, failure (Bojorquez incomplete pass, FG formation, TEN game, miscommunication)
  • 4th&6 at own 47, 7:26 1st, leading 7-0, success (L. Thomas 15 yard pass, punt formation)
  • 4th&4 at Opp 24, 1:21 4th, leading 42-17 failure (Allen -1 kneel)

Really good post

Link to comment
Share on other sites

×
×
  • Create New...