QB Metrics and Advanced Analytics--Good info

[Big Turk]

Piggybacking on my last thread regarding Career draft data, I wanted to see how QB's since 1999 looked like based on AV/Game Played so I could have a normalized look at all players without regard to how long they have been in the league. However, when I ran it per game played, I quickly realized something was amiss...weighted AV punishes players who play longer based on the formula it uses. There is no way in any metric that Aaron Brooks should be rated ahead of Drew Brees and Tom Brady.

 

So I reversed the weighted AV back to Career AV, then divided it by the number of games played and multipled by 16 to give an average "seasonal" AV rating which came out much better, although there are a few things I didn't feel are right---namely Derek Carr being rated in the "Below Average" QB grouping(I assume this has to be an issue with the AV calculation on PFR)

 

QB's with ties to the Bills are highlighted in red. 7 QB's from the "Terrible" and "How Did He make it to the NFL??" List, 3 from the "Below Average List", and 2 from the "Average" List. And we wonder why we have sucked for so long...

 

Even more interesting is that it settles the debate of some posters who claim Flacco is such an elite QB in comparison to Tyrod. The analytics say this is not only a false statement and that they are essentially the same player in terms of per game value to their team(with Flacco having a slight edge), but that Flacco is clearly in the "Average" QB category and not the "Elite" or even "Very Good" category.

 

The "Elite QB's"(Over 17):

Aaron Rodgers 21.0

Tom Brady 19.4

(Peyton Manning) 19.2 <---Not originally included as he missed the cutoff by a year, but I went and re-ran it for him also

Daunte Culpepper 19.0

Drew Brees 17.9

Phillip Rivers 17.6

Russell Wilson 17.5

Cam Newton 17.3

Michael Vick 17.2(shocked to see him this far up)

 

The "Very Good QB's"(16.9-15.0):

Matt Ryan 16.9

Dak Prescott 16.0(based on one season of data)

Donovan McNabb 15.9

Andrew Luck 15.5

Carson Palmer 15.5

Robert Griffin III 15.3(still living off his rookie season)

David Garrard 15.3(shocked part II)

Ben Roethlisberger 15.1

 

The "Good QB's"(14.9-13.0):

Aaron Brooks 14.9

Chad Pennington 14.3

Matthew Stafford 14.3

Jay Cutler 14.0

Alex Smith 13.8

Eli Manning 13.4

Marc Bulger 13.3

Andy Dalton 13.2

Marcus Mariota 13.0

 

The "Average QB's"(12.9-11.0)

Joe Flacco 12.7

Colin Kaepernick 12.5

Tyrod Taylor 12.5

Byron Leftwich 12.3

Josh Freeman 12.3

Teddy Bridgewater 12.1

Ryan Fitzpatrick 12.1

Jameis Winston 12.0

Kirk Cousins 12.0

Ryan Tannehill 11.9

Matt Schaub 11.8

Jason Campbell 11.7

 

The "Below Average QB's"(10.9-9.0)

David Carr 10.9

Sam Bardford 10.8

Matt Cassell 10.6

Vince Young 10.6

Kyle Orton 10.6

Derek Carr 10.4(Huh????---The AV HAS to be wrong for him on Pro Football Reference...)

Josh McCown 10.3

Blake Bortles 10.2

Christian Ponder 10.2

Shaun King 10.2

Nick Foles 10.1

Carson Wentz 10.0

Trevor Siemian 9.6

Rex Grossman 9.5

Mike Glennon 9.2

Dennis Dixon 9.2

Mark Sanchez 9.2

Chad Henne 9.0

 

The "Terrible QB's"(8.9-7.0)

Tim Couch 8.9

Jake Locker 8.8

J.P Losman 8.8

Trent Edwards 8.6

John Beck 8.2

Matt Mauck 8.0

Tarvaris Jackson 8.0

Quincy Carter 7.9

Patrick Ramsey 7.6

Geno Smith 7.5

JT O'Sullivan 7.3

Matt Leinart 7.3

Brandon Weeden 7.2

Derek Anderson 7.2

Joey Harrington 7.1

Chris Redman 7.0

Chris Simms 7.0

 

The "How Did He Make It To The NFL?? QB's"(6.9-5.0)

Drew Stanton 6.8

Kevin Kolb 6.8

Colt McCoy 6.8

Tyler Thigpen 6.6

Sage Rosenfels 6.6

A.J. McCarron 6.3

EJ Manuel 6.3

Charlie Whitehurst 6.2

Tim Rattay 6.2

Chris Weinke 6.2

Dan Orlovsky 6.0

Jim Sorgi 6.0

Edited June 15, 2017 by matter2003

[Thurman#1]

I don't think looking at this one measure you've created and calculated "clearly settles" anything whatsoever, much less whether or not Flacco is better than Tyrod. It ain't "the analytics" saying "that they are essentially the same player in terms of per game value to their team(with Flacco having a slight edge), but that Flacco is clearly in the "Average" QB category and not the "Elite" or even "Very Good" category." It's your one particular way of looking at this one particular stat.

 

I guess you can look at it as some added data.

 

Plus, I'm not sure where you see that Aaron Brooks is higher at weighted AV than Brees or Rodgers. Looks to me like what they say is that Brooks has a 65 weighted AV, 755th overall since 1960 while Rodgers has a 124, 26th overall and Brees has a 153 7th overall. Weighted AV has them in fairly reasonable places. Could it be your new stat that has them strangely?

 

IMHO it's a bit questionable whether AV, a stat meant to be good at making sweeping generalizations about large amounts of data, a large groups of seasons together for instance, should usefully be divided up in an attempt to pretend that it is like a knife and can make small differentiations. That's not how it was designed. He (Doug from PFR) himself points out that there's a good reason he included the "Approximate" in Approximate Value.

 

 

 

In any case, it looks to have been an interesting project. I am impressed by your energy and interest.

Edited June 15, 2017 by Thurman#1

[transplantbillsfan]

What the hell is AV?

[jr1]

isn't Culpepper kind of high? He was pretty much done by 28

[Wayne Cubed]

What the hell is AV?

 

It's a metric that Pro Football Reference made up, it stands for Approximate Value. Here's some information on it:

 

"AV is not meant to be a be-all end-all metric. Football stat lines just do not come close to capturing all the contributions of a player the way they do in baseball and basketball. If one player is a 16 and another is a 14, we can't be very confident that the 16AV player actually had a better season than the 14AV player. But I am pretty confident that the collection of all players with 16AV played better, as an entire group, than the collection of all players with 14AV."

 

"Essentially, AV is a substitute for --- and a significant improvement upon, in my opinion --- metrics like 'number of seasons as a starter' or 'number of times making the pro bowl' or the like. You should think of it as being essentially like those two metrics, but with interpolation in between. That is, 'number of seasons as a starter' is a reasonable starting point if you're trying to measure, say, how good a particular draft class is, or what kind of player you can expect to get with the #13 pick in the draft. But obviously some starters are better than others. Starters on good teams are, as a group, better than starters on bad teams. Starting WRs who had lots of receiving yards are, as a group, better than starting WRs who did not have many receiving yards. Starters who made the pro bowl are, as a group, better than starters who didn't, and so on. And non-starters aren't worthless, so they get some points too."

 

http://www.pro-football-reference.com/blog/index37a8.html

[cd1]

What happened to Peyton Manning?

[atlbillsfan1975]

Average means you take the highs and lows and add them togethether and then divide by those amounts. If the highs occur in the playoffs and result in a SB win, who cares about 'AV'? This system, while overall does a decent job of ranking QB's, must also have an asterisk with SB winner. show the 'AV' of the quarterback during that SB season and playoffs.

I think this is nice but I also think this is what's degrading sports to some extent. Watch and enjoy. Sports are suppose to be entertainment. Now some treat it like a job. It's a nice hobby to do all this and not get paid .

[fansince88]

What!!! Jeff Tuel,didnt even make the list? He was the best ever! !

[Big Turk]

I don't think looking at this one measure you've created and calculated "clearly settles" anything whatsoever, much less whether or not Flacco is better than Tyrod. It ain't "the analytics" saying "that they are essentially the same player in terms of per game value to their team(with Flacco having a slight edge), but that Flacco is clearly in the "Average" QB category and not the "Elite" or even "Very Good" category." It's your one particular way of looking at this one particular stat.

 

I guess you can look at it as some added data.

 

Plus, I'm not sure where you see that Aaron Brooks is higher at weighted AV than Brees or Rodgers. Looks to me like what they say is that Brooks has a 65 weighted AV, 755th overall since 1960 while Rodgers has a 124, 26th overall and Brees has a 153 7th overall. Weighted AV has them in fairly reasonable places. Could it be your new stat that has them strangely?

 

IMHO it's a bit questionable whether AV, a stat meant to be good at making sweeping generalizations about large amounts of data, a large groups of seasons together for instance, should usefully be divided up in an attempt to pretend that it is like a knife and can make small differentiations. That's not how it was designed. He (Doug from PFR) himself points out that there's a good reason he included the "Approximate" in Approximate Value.

 

 

 

In any case, it looks to have been an interesting project. I am impressed by your energy and interest.

 

Brooks was higher when you took his weighted AV/Games played....I wanted to compare players on a per game basis to even things out and compare apples to apples. Weighted AV penalizes players who play more seasons by reducing their AV, but does so uniformly making it a fair comparison for players who played equal lengths of time or looking at how much a player was worth over his entire career, but isn't good for looking at it on a per game basis because players who played fewer years will be favored since their weighted AV years aren't being reduced by 5% each year as many times as older players.

isn't Culpepper kind of high? He was pretty much done by 28

 

This is purely based on a "per game" basis. It shows how much value the QB provides/provided to his team on a per game basis and then normalizes it for a 16 game season. This allows me to comapre QB's who have played different amounts of years and different amounts of games to look at how much value they provide on per game played basis.

What happened to Peyton Manning?

This only goes back to 1999(the first year of combine data, which was the original scope of the project), I believe Manning was drafted in 1998

What!!! Jeff Tuel,didnt even make the list? He was the best ever! !

He wasn't drafted, lol

[BuffaloHokie13]

What happened to Peyton Manning?

Drafted in 1998, OP used players since 1999.

 

In regards to Carr it's clearly due to his rookie year. <60% Completion, <80 RAT, <5 ANY/A

Actually very similar in those stats to EJ's rookie year. (Also <60%, <80, <5, and a 5 AV)

Edited June 15, 2017 by BuffaloHokie13

[Big Turk]

What happened to Peyton Manning?

 

Just for fun I went back and ran Manning, he would have scored a 19.2, which is directly behind Rodgers and Brady on a per game basis.

[racketmaster]

Interesting stuff. Always appreciate the time and effort behind running numbers. However, I am a little skeptical of any stats showing Culpepper and Vick in the same elite category as Rodgers and Brady. Or David Garrard having a higher number than Ben Roethlisberger. One more oddity that stands out is David Carr being higher than Derek Carr. Where is Peyton Manning scored?

[Kirby Jackson]

Good stuff!!

[Big Turk]

Interesting stuff. Always appreciate the time and effort behind running numbers. However, I am a little skeptical of any stats showing Culpepper and Vick in the same elite category as Rodgers and Brady. Or David Garrard having a higher number than Ben Roethlisberger. One more oddity that stands out is David Carr being higher than Derek Carr. Where is Peyton Manning scored?

See the above post...I went back and ran Mannking's numbers and he scored a 19.2

 

In regards to Roethlisberger, don't forget the first few years he was basically hand the ball off and throw when needed, so I'm sure that probably factored in as well...also he has posted some really bad road numbers throughout his career...most of his big games are at home...

[Royale with Cheese]

What the hell is AV?

It's a metric that you're not going to like because it doesn't put TT in a tier 1 category.

[Big Turk]

It's a metric that you're not going to like because it doesn't put TT in a tier 1 category.

I actually went back and re-ran things using the multiplier GamesPlayed/16 instead of YearsPlayed to remove players who got unfair score increases by simply playing a bunch of years, and it corrected a lot of the "eye-check" issues, although not all of them..Derek Carr was still ranked in the same category(although this time ahead of his brother in the same category), and Vick, Culpepper and Garrard all dropped down a rung...Taylor also dropped down a rung as well and the gap between him and Flacco went from 12.7 to 12.5 to a 13.0 to 11.1...

 

I'll post the newer data when I get home, I know it's not perfect but it's very interesting and I need a way to be able to look at players and create appropriate "career" curves for them along with a bunch of other things like draft position to career curves, etc, so getting something that is useful, even if not perfect, is important.

Edited June 15, 2017 by matter2003

[BuffaloHokie13]

I actually went back and re-ran things using the multiplier GamesPlayed/16 instead of YearsPlayed to remove players who got unfair score increases by simply playing a bunch of years, and it corrected a lot of the "eye-check" issues, although not all of them..Derek Carr was still ranked in the same category(although this time ahead of his brother in the same category), and Vick, Culpepper and Garrard all dropped down a rung...Taylor also dropped down a rung as well and the gap between him and Flacco went from 12.7 to 12.5 to a 13.0 to 11.1...

 

I'll post the newer data when I get home, I know it's not perfect but it's very interesting and I need a way to be able to look at players and create appropriate "career" curves for them along with a bunch of other things like draft position to career curves, etc, so getting something that is useful, even if not perfect, is important.

I am actually a little curious how you're getting the numbers you're getting.

 

Aaron Rodgers, for instance, has a career AV of 150 and 142 games played per PFR. So 150/142*16=16.9. I also wonder if it'd effect anything if you used games started vs. games played and only used AVs from seasons with starts. For Rodgers that'd change the numbers to 149 AV, 135 Starts, and a score of 17.7. Score went up because his 0, 0, and 1 AV from his first 3 years with no starts are factored out.

[Crusher]

There's no way one can measure what system a QB is in and what they are asked to do within that system. Are they asked to be caretakers of an offense as opposed to taking chances to make more plays for the team? This all matters and can make a guy who takes little risk appear better than a player who is asked to do more in the hopes of getting more production.

 

This is why many metrics for QB's just aren't very good when comparing players IMO.

[Big Turk]

There's no way one can measure what system a QB is in and what they are asked to do within that system. Are they asked to be caretakers of an offense as opposed to taking chances to make more plays for the team? This all matters and can make a guy who takes little risk appear better than a player who is asked to do more in the hopes of getting more production.

 

This is why many metrics for QB's just aren't very good when comparing players IMO.

To a point that is true, and by no means is it perfect. At the same time QBs that "aren't asked to do as much" is usually because they aren't good enough to be asked to do more.

[Kirby Jackson]

To a point that is true, and by no means is it perfect. At the same time QBs that "aren't asked to do as much" is usually because they aren't good enough to be asked to do more.

And they don't have the starts to register. If you draw a line of "with at least 10 starts" you'd probably eliminate every bum (besides EJ). Edited June 15, 2017 by Kirby Jackson