Jump to content

Case Study: 1,000 pass attempts determines a young QBs career path


Ramius

Recommended Posts

There was a guy on the local radio a few weeks back (i can't remember who) who stated that what projecting how well a young QB will develop, you can't look so much at years or starts, but rather pass attempts. He said that with young QBs he believed you really need to see 1,000 pass attempts before making a legitimate conclusion on how their career will turn out.

 

I wanted to see what kind of validity there was to the 1,000 attempt theory, so i put it to a small test. I took a list of recent QBs to reflect the "current" NFL landscape (going from 2001 to 2010, but including Peyton, Brady, and Warner) and checked their passer ratings in 3 categories: the season they hit 1,000 pass attempts, the season prior, and their career rating. I then checked the correlations between the ratings in those 3 categories. (i'm a scientist, so yes i understand the limits of a simple correlation)

 

Here's some of the methodology:

I used passer rating, because while not perfect, it's a pretty good reflection of the skill of a QB, especially over the course of a career. The season they hit 1,000 pass attempts was chosen for obvious reasons, and the season prior was chosen to see how big of a difference (if any) existed.

 

I considered QBs that started a good chunk of time consecutively, so we were dealing mostly with consistent starters, or guys projected to be long term starters. I also looked at lower drafted guys that had developed into starters. There were 28 QBs in my study.

 

The numbers:

16/28 QBs hit the 1,000 pass attempt plateau in their 3rd season. 1 QB hit it in his 2nd season, and the rest in year 4 or later.

 

I found that there was a 0.71 correlation between the passer rating in the season where a QB hit his 1,000th pass attempt and his career rating. There was only 0.46 correlation between the rating in the season prior to hitting 1,000 and career rating. Interestingly enough, there was one huge outlier, Ben Roethlisberger. He had his far and away worst season of his career in the seaosn he hit 1,000 attempts. When i remove that data point, the numbers become more stark.

 

The correlation in passer rating jumps to 0.77 in the 1,000 season, and drops to 0.43 for the season prior.

 

Conclusion:

With these numbers, i'm confident in saying that a QBs rating in the season where he hits 1,000 pass attempts is a good predictor of what his career rating will turn out to be.

 

Some food for thought:

1,000 attempts allows enough time for a QB to completely develop. It allows time to correct a "sophomore slump" but also will show if a good early season is a fluke (Josh Freeman). In that time, a QB will probably see most of everything that NFL defenses can throw at him, and defenses will have gotten enough film to adjust to said QB. It isn't reliant on starts, which can skew a passer's ability if they throw either a lot early on (Peyton), or not so much early on (Vick, Rivers).

 

The next group of QBs:

Since 2011, 5 drafted QBs have already hit 1,000 attempts: Newton, Dalton, Luck, Ponder, and Tannehill. Their ratings in the 1,000 season are, respectively: 86.2, 87.4, 87.0, 77.9, and 81.7. So the 1,000 theory predicts good things for each one save Ponder/Tannehill.

 

The following QBs are close to 1,000 and will hit that mark this season (if given enough PT in some cases): RGIII: 849 attempts, R. Wilson: 800 attempts, Weeden: 784 attempts, Gabbert: 777 attempts, Kaepernick: 639 attempts, Foles: 582 attempts. So whatever you see out of these guys this season has a good chance of being an accurate potrait of their careers going forward. Its especially interesting regarding RGIII and how his career has started,

 

EJ currently sits at 306 attempts. So chances are, no matter how good or bad EJ does in this upcoming year, it won't necessarily be an accurate reflection of how his career will turn out.

 

If anyone wants to know specific QBs and their numbers, fire away (i cant post from work, so it may be a bit before i respond)

Link to comment
Share on other sites

Good work. Some interesting analysis that I will file in the memory to use at a later date when thinking about QB's.

 

The only question I have is rather complex. If a QB like Chad Pennington gets his 1,000 passes in how does he compare to a guy like Carson Palmer who got 1,000 passes at probably the same rate? Or both of those guys compare to a Mark Sanchez? Philip Rivers?

 

I ask because I am curious how far after 1,000 passes did some of these "starters" last? By all accounts Pennington, Palmer and Sanchez lost out in their starting position earlier then many expected them to when they likely hit their 1,000 mark.

Link to comment
Share on other sites

There was a guy on the local radio a few weeks back (i can't remember who) who stated that what projecting how well a young QB will develop, you can't look so much at years or starts, but rather pass attempts. He said that with young QBs he believed you really need to see 1,000 pass attempts before making a legitimate conclusion on how their career will turn out.

 

I wanted to see what kind of validity there was to the 1,000 attempt theory, so i put it to a small test. I took a list of recent QBs to reflect the "current" NFL landscape (going from 2001 to 2010, but including Peyton, Brady, and Warner) and checked their passer ratings in 3 categories: the season they hit 1,000 pass attempts, the season prior, and their career rating. I then checked the correlations between the ratings in those 3 categories. (i'm a scientist, so yes i understand the limits of a simple correlation)

 

Here's some of the methodology:

I used passer rating, because while not perfect, it's a pretty good reflection of the skill of a QB, especially over the course of a career. The season they hit 1,000 pass attempts was chosen for obvious reasons, and the season prior was chosen to see how big of a difference (if any) existed.

 

I considered QBs that started a good chunk of time consecutively, so we were dealing mostly with consistent starters, or guys projected to be long term starters. I also looked at lower drafted guys that had developed into starters. There were 28 QBs in my study.

 

The numbers:

16/28 QBs hit the 1,000 pass attempt plateau in their 3rd season. 1 QB hit it in his 2nd season, and the rest in year 4 or later.

 

I found that there was a 0.71 correlation between the passer rating in the season where a QB hit his 1,000th pass attempt and his career rating. There was only 0.46 correlation between the rating in the season prior to hitting 1,000 and career rating. Interestingly enough, there was one huge outlier, Ben Roethlisberger. He had his far and away worst season of his career in the seaosn he hit 1,000 attempts. When i remove that data point, the numbers become more stark.

 

The correlation in passer rating jumps to 0.77 in the 1,000 season, and drops to 0.43 for the season prior.

 

Conclusion:

With these numbers, i'm confident in saying that a QBs rating in the season where he hits 1,000 pass attempts is a good predictor of what his career rating will turn out to be.

 

Some food for thought:

1,000 attempts allows enough time for a QB to completely develop. It allows time to correct a "sophomore slump" but also will show if a good early season is a fluke (Josh Freeman). In that time, a QB will probably see most of everything that NFL defenses can throw at him, and defenses will have gotten enough film to adjust to said QB. It isn't reliant on starts, which can skew a passer's ability if they throw either a lot early on (Peyton), or not so much early on (Vick, Rivers).

 

The next group of QBs:

Since 2011, 5 drafted QBs have already hit 1,000 attempts: Newton, Dalton, Luck, Ponder, and Tannehill. Their ratings in the 1,000 season are, respectively: 86.2, 87.4, 87.0, 77.9, and 81.7. So the 1,000 theory predicts good things for each one save Ponder/Tannehill.

 

The following QBs are close to 1,000 and will hit that mark this season (if given enough PT in some cases): RGIII: 849 attempts, R. Wilson: 800 attempts, Weeden: 784 attempts, Gabbert: 777 attempts, Kaepernick: 639 attempts, Foles: 582 attempts. So whatever you see out of these guys this season has a good chance of being an accurate potrait of their careers going forward. Its especially interesting regarding RGIII and how his career has started,

 

EJ currently sits at 306 attempts. So chances are, no matter how good or bad EJ does in this upcoming year, it won't necessarily be an accurate reflection of how his career will turn out.

 

If anyone wants to know specific QBs and their numbers, fire away (i cant post from work, so it may be a bit before i respond)

so at 9-10 games a year, we should finally know if he's any good by 2018 for sure!
Link to comment
Share on other sites

There was a guy on the local radio a few weeks back (i can't remember who) who stated that what projecting how well a young QB will develop, you can't look so much at years or starts, but rather pass attempts. He said that with young QBs he believed you really need to see 1,000 pass attempts before making a legitimate conclusion on how their career will turn out.

 

I wanted to see what kind of validity there was to the 1,000 attempt theory, so i put it to a small test. I took a list of recent QBs to reflect the "current" NFL landscape (going from 2001 to 2010, but including Peyton, Brady, and Warner) and checked their passer ratings in 3 categories: the season they hit 1,000 pass attempts, the season prior, and their career rating. I then checked the correlations between the ratings in those 3 categories. (i'm a scientist, so yes i understand the limits of a simple correlation)

 

Here's some of the methodology:

I used passer rating, because while not perfect, it's a pretty good reflection of the skill of a QB, especially over the course of a career. The season they hit 1,000 pass attempts was chosen for obvious reasons, and the season prior was chosen to see how big of a difference (if any) existed.

 

I considered QBs that started a good chunk of time consecutively, so we were dealing mostly with consistent starters, or guys projected to be long term starters. I also looked at lower drafted guys that had developed into starters. There were 28 QBs in my study.

 

The numbers:

16/28 QBs hit the 1,000 pass attempt plateau in their 3rd season. 1 QB hit it in his 2nd season, and the rest in year 4 or later.

 

I found that there was a 0.71 correlation between the passer rating in the season where a QB hit his 1,000th pass attempt and his career rating. There was only 0.46 correlation between the rating in the season prior to hitting 1,000 and career rating. Interestingly enough, there was one huge outlier, Ben Roethlisberger. He had his far and away worst season of his career in the seaosn he hit 1,000 attempts. When i remove that data point, the numbers become more stark.

 

The correlation in passer rating jumps to 0.77 in the 1,000 season, and drops to 0.43 for the season prior.

 

Conclusion:

With these numbers, i'm confident in saying that a QBs rating in the season where he hits 1,000 pass attempts is a good predictor of what his career rating will turn out to be.

 

Some food for thought:

1,000 attempts allows enough time for a QB to completely develop. It allows time to correct a "sophomore slump" but also will show if a good early season is a fluke (Josh Freeman). In that time, a QB will probably see most of everything that NFL defenses can throw at him, and defenses will have gotten enough film to adjust to said QB. It isn't reliant on starts, which can skew a passer's ability if they throw either a lot early on (Peyton), or not so much early on (Vick, Rivers).

 

The next group of QBs:

Since 2011, 5 drafted QBs have already hit 1,000 attempts: Newton, Dalton, Luck, Ponder, and Tannehill. Their ratings in the 1,000 season are, respectively: 86.2, 87.4, 87.0, 77.9, and 81.7. So the 1,000 theory predicts good things for each one save Ponder/Tannehill.

 

The following QBs are close to 1,000 and will hit that mark this season (if given enough PT in some cases): RGIII: 849 attempts, R. Wilson: 800 attempts, Weeden: 784 attempts, Gabbert: 777 attempts, Kaepernick: 639 attempts, Foles: 582 attempts. So whatever you see out of these guys this season has a good chance of being an accurate potrait of their careers going forward. Its especially interesting regarding RGIII and how his career has started,

 

EJ currently sits at 306 attempts. So chances are, no matter how good or bad EJ does in this upcoming year, it won't necessarily be an accurate reflection of how his career will turn out.

 

If anyone wants to know specific QBs and their numbers, fire away (i cant post from work, so it may be a bit before i respond)

 

Very interesting. Thanks for the post! Kind of akin to the '10,000 hours' rule Malcolm Gladwell talked about in Outliers.

 

 

Link to comment
Share on other sites

Very interesting. Thanks for the post! Kind of akin to the '10,000 hours' rule Malcolm Gladwell talked about in Outliers.

Kind of like that, except for Ramius actually did his own research instead of cherry picking others' to generate a pop science theory and make himself the darling of a certain set of intellectuals. ;)

Link to comment
Share on other sites

There was a guy on the local radio a few weeks back (i can't remember who) who stated that what projecting how well a young QB will develop, you can't look so much at years or starts, but rather pass attempts. He said that with young QBs he believed you really need to .......

 

 

If anyone wants to know specific QBs and their numbers, fire away (i cant post from work, so it may be a bit before i respond)

 

This was interesting.

Link to comment
Share on other sites

There was a guy on the local radio a few weeks back (i can't remember who) who stated that what projecting how well a young QB will develop, you can't look so much at years or starts, but rather pass attempts. He said that with young QBs he believed you really need to see 1,000 pass attempts before making a legitimate conclusion on how their career will turn out.

 

I wanted to see what kind of validity there was to the 1,000 attempt theory, so i put it to a small test. I took a list of recent QBs to reflect the "current" NFL landscape (going from 2001 to 2010, but including Peyton, Brady, and Warner) and checked their passer ratings in 3 categories: the season they hit 1,000 pass attempts, the season prior, and their career rating. I then checked the correlations between the ratings in those 3 categories. (i'm a scientist, so yes i understand the limits of a simple correlation)

 

Here's some of the methodology:

I used passer rating, because while not perfect, it's a pretty good reflection of the skill of a QB, especially over the course of a career. The season they hit 1,000 pass attempts was chosen for obvious reasons, and the season prior was chosen to see how big of a difference (if any) existed.

 

I considered QBs that started a good chunk of time consecutively, so we were dealing mostly with consistent starters, or guys projected to be long term starters. I also looked at lower drafted guys that had developed into starters. There were 28 QBs in my study.

 

The numbers:

16/28 QBs hit the 1,000 pass attempt plateau in their 3rd season. 1 QB hit it in his 2nd season, and the rest in year 4 or later.

 

I found that there was a 0.71 correlation between the passer rating in the season where a QB hit his 1,000th pass attempt and his career rating. There was only 0.46 correlation between the rating in the season prior to hitting 1,000 and career rating. Interestingly enough, there was one huge outlier, Ben Roethlisberger. He had his far and away worst season of his career in the seaosn he hit 1,000 attempts. When i remove that data point, the numbers become more stark.

 

The correlation in passer rating jumps to 0.77 in the 1,000 season, and drops to 0.43 for the season prior.

 

Conclusion:

With these numbers, i'm confident in saying that a QBs rating in the season where he hits 1,000 pass attempts is a good predictor of what his career rating will turn out to be.

 

Some food for thought:

1,000 attempts allows enough time for a QB to completely develop. It allows time to correct a "sophomore slump" but also will show if a good early season is a fluke (Josh Freeman). In that time, a QB will probably see most of everything that NFL defenses can throw at him, and defenses will have gotten enough film to adjust to said QB. It isn't reliant on starts, which can skew a passer's ability if they throw either a lot early on (Peyton), or not so much early on (Vick, Rivers).

 

The next group of QBs:

Since 2011, 5 drafted QBs have already hit 1,000 attempts: Newton, Dalton, Luck, Ponder, and Tannehill. Their ratings in the 1,000 season are, respectively: 86.2, 87.4, 87.0, 77.9, and 81.7. So the 1,000 theory predicts good things for each one save Ponder/Tannehill.

 

The following QBs are close to 1,000 and will hit that mark this season (if given enough PT in some cases): RGIII: 849 attempts, R. Wilson: 800 attempts, Weeden: 784 attempts, Gabbert: 777 attempts, Kaepernick: 639 attempts, Foles: 582 attempts. So whatever you see out of these guys this season has a good chance of being an accurate potrait of their careers going forward. Its especially interesting regarding RGIII and how his career has started,

 

EJ currently sits at 306 attempts. So chances are, no matter how good or bad EJ does in this upcoming year, it won't necessarily be an accurate reflection of how his career will turn out.

 

If anyone wants to know specific QBs and their numbers, fire away (i cant post from work, so it may be a bit before i respond)

Ah an educated man! That means your not a field agent. Great Work!!!!

Link to comment
Share on other sites

Interesting work, but an r=.71 is not a strong correlation. It's an intuitive theory--nice job!

 

What college did you learn quantitative methods at so I can take it off the list of potential college for my kids?!!!! The value of r can be between -1 and 1. Values closer to -1 and 1 are considered strong negative or positive correlations, respectively. It its generally accepted than any score less than -.70 or greater than .70 have very strong negative or positive correlation, respectively. So you might want to brush up on your stats a bit.

 

Outside of that, I appreciate this type of thinking as data junkie/analyst myself. The only issue I would have is that you're not measuring a "career" properly. I know this was a quick-and-dirty idea and it leads to bigger questions, but it might be best to do this with the population of retired QBs (or retired QBs since 1980 or justifiable selection to capture the more recent NFL) rather than currently active QBs. In this manner, the results would be more robust. Anyway, I think the findings this way might not see an r as high but would still be strong.

Edited by zonabb
Link to comment
Share on other sites

There was a guy on the local radio a few weeks back (i can't remember who) who stated that what projecting how well a young QB will develop, you can't look so much at years or starts, but rather pass attempts. He said that with young QBs he believed you really need to see 1,000 pass attempts before making a legitimate conclusion on how their career will turn out.

 

I wanted to see what kind of validity there was to the 1,000 attempt theory, so i put it to a small test. I took a list of recent QBs to reflect the "current" NFL landscape (going from 2001 to 2010, but including Peyton, Brady, and Warner) and checked their passer ratings in 3 categories: the season they hit 1,000 pass attempts, the season prior, and their career rating. I then checked the correlations between the ratings in those 3 categories. (i'm a scientist, so yes i understand the limits of a simple correlation)

 

Here's some of the methodology:

I used passer rating, because while not perfect, it's a pretty good reflection of the skill of a QB, especially over the course of a career. The season they hit 1,000 pass attempts was chosen for obvious reasons, and the season prior was chosen to see how big of a difference (if any) existed.

 

I considered QBs that started a good chunk of time consecutively, so we were dealing mostly with consistent starters, or guys projected to be long term starters. I also looked at lower drafted guys that had developed into starters. There were 28 QBs in my study.

 

The numbers:

16/28 QBs hit the 1,000 pass attempt plateau in their 3rd season. 1 QB hit it in his 2nd season, and the rest in year 4 or later.

 

I found that there was a 0.71 correlation between the passer rating in the season where a QB hit his 1,000th pass attempt and his career rating. There was only 0.46 correlation between the rating in the season prior to hitting 1,000 and career rating. Interestingly enough, there was one huge outlier, Ben Roethlisberger. He had his far and away worst season of his career in the seaosn he hit 1,000 attempts. When i remove that data point, the numbers become more stark.

 

The correlation in passer rating jumps to 0.77 in the 1,000 season, and drops to 0.43 for the season prior.

 

Conclusion:

With these numbers, i'm confident in saying that a QBs rating in the season where he hits 1,000 pass attempts is a good predictor of what his career rating will turn out to be.

 

Some food for thought:

1,000 attempts allows enough time for a QB to completely develop. It allows time to correct a "sophomore slump" but also will show if a good early season is a fluke (Josh Freeman). In that time, a QB will probably see most of everything that NFL defenses can throw at him, and defenses will have gotten enough film to adjust to said QB. It isn't reliant on starts, which can skew a passer's ability if they throw either a lot early on (Peyton), or not so much early on (Vick, Rivers).

 

The next group of QBs:

Since 2011, 5 drafted QBs have already hit 1,000 attempts: Newton, Dalton, Luck, Ponder, and Tannehill. Their ratings in the 1,000 season are, respectively: 86.2, 87.4, 87.0, 77.9, and 81.7. So the 1,000 theory predicts good things for each one save Ponder/Tannehill.

 

The following QBs are close to 1,000 and will hit that mark this season (if given enough PT in some cases): RGIII: 849 attempts, R. Wilson: 800 attempts, Weeden: 784 attempts, Gabbert: 777 attempts, Kaepernick: 639 attempts, Foles: 582 attempts. So whatever you see out of these guys this season has a good chance of being an accurate potrait of their careers going forward. Its especially interesting regarding RGIII and how his career has started,

 

EJ currently sits at 306 attempts. So chances are, no matter how good or bad EJ does in this upcoming year, it won't necessarily be an accurate reflection of how his career will turn out.

 

If anyone wants to know specific QBs and their numbers, fire away (i cant post from work, so it may be a bit before i respond)

Finally a thread worth reading this off season, well done! :)

 

Now what you need to do is do the same analysis for college QB's and join the Bills analytics team! :)

Edited by CodeMonkey
Link to comment
Share on other sites

Thanks for working the numbers.

 

My question is what is the percentage of QBs that make the 1000 attempts threshold? Clearly, there are backup QBs that never get any opportunity. But, what I'm asking about is the percentage of guys like Trent Edwards and JP Losman, who never reached the 1000 attempts threshold. And, I don't think many people would argue that they need to see him start another season to realize what they got in Trent. :)

 

PS: Matt Leinart is another one-time NFL starter that didn't make the plateau. (OTOH, Vince Young and Tarvaris Jackson both did, the latter in his 6th season.)

Link to comment
Share on other sites

What college did you learn quantitative methods at so I can take it off the list of potential college for my kids?!!!! The value of r can be between -1 and 1. Values closer to -1 and 1 are considered strong negative or positive correlations, respectively. It its generally accepted than any score less than -.70 or greater than .70 have very strong negative or positive correlation, respectively. So you might want to brush up on your stats a bit.

 

Outside of that, I appreciate this type of thinking as data junkie/analyst myself. The only issue I would have is that you're not measuring a "career" properly. I know this was a quick-and-dirty idea and it leads to bigger questions, but it might be best to do this with the population of retired QBs (or retired QBs since 1980 or justifiable selection to capture the more recent NFL) rather than currently active QBs. In this manner, the results would be more robust. Anyway, I think the findings this way might not see an r as high but would still be strong.

 

I should have said very strong, but hey man, thanks for the brush up. In hard sciences, you're not going to get much published pushing .71 as a "very strong correlation" (it's not generally accepted). It might be considered very strong in the social sciences literature--so your kids are probably OK with the colleges they are looking at.

Edited by Mr. WEO
Link to comment
Share on other sites

I should have said very strong, but hey man, thanks for the brush up. In hard sciences, you're not going to get much published pushing .71 as a "very strong correlation" (it's not generally accepted). It might be considered very strong in the social sciences literature--so your kids are probably OK with the colleges they are looking at.

 

This is exactly what I was going to say. Nice work none the less. There is a very strong correlation for sure, but there is wiggle room for confounds. What exactly is the driving force behind these "successful" qbs and "unsuccessful" qbs? Is it their own ability, or is it their system? There are very few that can stand on their own and say that it's their TALENT and not the system. So can this really be about how good the QB is or could it possibly be because of outside influences, in this case, the system. Maybe its the QB coach they have, maybe a combination of all. What if EJ fizzles out here, is it becuase he doesn't have the talent, or was he not put into the right situation?

 

This is a strong correlation, but I wouldn't stand behind it wholeheartedly, there's too many questions and possible confounds. It would be very tough to control for these conditions. It would likely have to be a case study, ie, Peyton Manning. He left the Colts, is in a very different system in Denver, and still wins. There's a very strong argument that he stands on his talent.

 

Anyway, very well thought out study. Good work!

Link to comment
Share on other sites

It’s amazing this forum, I’ll tell ya. Everyone now is a research scientist and is arguing that because they have some remembrance of correlation and how it is interpreted, their opinion is unquestionable. But there are two problems here 1) the strength of the correlation coefficient, across the hard and soft sciences, although sometimes using different terms, has the same interpretation despite the above-comment to the effect that it won’t get you published “much” because it’s “generally” not accepted (by the way that statement is internally contradictory with your argument because it indicates that in some cases works are published with these scores, thereby invalidating your argument that). That wasn’t the argument anyway; the argument is the interpretation of the value of r. I didn’t say it was “very strong” I said it was strong and it took me all of 3 minutes to find two sources in the so-called hard sciences that suggest anything from .70 to .89 is consider “high” (Asuero et al, 2006, journal: Critical Reviews in Analytical Chemistry) and .70 to .89 “strong” correlation (Fowler et al, 2005, Practical Statistics for Field Biology). Nice try with the attempted personal misdirection that insinuates my kids aren’t good enough for the hard sciences as a way to shield your mistake. If you want to talk precision and stats, then write what you mean and say it without terms like “generally” and “much” as a way to not commit to a stance you seem committed to. You won’t get published at all with errors. 2) With everyone focused on the interpretation of r to flex their stats muscle, no one wanted to take a shot at his research design, which tells me a lot. It tells me you took a stats course once but have minimal understanding of the limitations of research design, and in particular this one. I’m not here to slam it, I think it’s an excellent exploratory analysis that leaves enough questions for further development and I applaud the OP. This is where your mettle as a researcher should have been focused, not on what is an obvious continual debate over the interpretation of r, which we’ll never agree.

Link to comment
Share on other sites

×
×
  • Create New...