League Wide Draft Success 2017-2019 - A Follow Up

[Billl]

31 minutes ago, Ya Digg? said:

Haha I'm sorry I don't mean to nitpick and I actually agree with a lot of what you are saying, but when I read that part right there it made me laugh out loud - like it's some amazing skill to hold the ball on a field goal or extra point!!  😆

Yeah, that was a passive aggressive joke about not needing a punter. 😇

[JGMcD2]

2 hours ago, Billl said:

You’re comparing stats using DIFFERENT POSITIONS.  Of course a WR is going to skew the stats on a per touch basis versus a RB.  
 

Let’s “take names out of it” again.  Would you rather have 240 touches, 1925 yards, and 25 TDs or 378 touches, 2027 yards, and 19 TDs?

 

If you chose the first, congratulations.  You got Robert Tonyan, Jonnu Smith, and David Johnson.  I’ll stick with Derrick Henry.

I guess we will have to agree to disagree. You’re sharp man, it was good going back and forth with you and I did pick some things up. I think it’s unfair you’re throwing this out because it does a good job with more data. I said this year  wasn’t very stable because the sample size is small. 

I would still love some suggestions on how to fix this so it’s not stupid. 

Edited January 27, 2021 by JGMcD2

[Billl]

50 minutes ago, JGMcD2 said:

I guess we will have to agree to disagree. You’re sharp man, it was good going back and forth with you and I did pick some things up. I think it’s unfair you’re throwing this out because it does a good job with more data. I said this year  wasn’t very stable because the sample size is small. 

I would still love some suggestions on how to fix this so it’s not stupid. 

I guess I didn’t realize that this was your own creation.  You’re never going to achieve consensus trying to quantify the unquantifiable, but you need it to pass the common sense test.  If I were to put something similar together, I would start by combining methods that are already established.  For example, draft value charts aren’t exact, but they’re pretty good in terms of estimating trade equivalencies.  My step 1 would be to total up the draft value points used.

 

My second step would be to find a measure of determining a player’s contribution irrespective of draft position.  There are plenty of versions of these as well, so it’s a matter of picking your favorite.  For the sake of argument, I’ll go with Pro-football-reference.com and their Approximate Value stat.  It’s far from perfect (I don’t think Fred Warner had a better season than Patrick Mahomes, for example), but it’s a decent jumping off point.  (That said, it has Willie Gay at 4 and Sneed at 3, so even this method is going to be terrible.)

An possibly better, though more tedious, way would be to reference a redraft and assign players values based on the draft chart value of their redraft position.  Here’s one that I found, but I have no idea on its quality.  https://bleacherreport.com/articles/2926937-redrafting-the-2020-nfl-draft

It shows Sneed at 25 and CEH at 32, for reference.

 

From there, I would take the sum total of AAV (or redraft score) of the draft class and divide it by the total draft chart points.

 

A quick example using Sneed and CEH would show that 627 points of draft value were used on them in the actual draft.  The redraft positions are worth 1310 points, so they would have a “surplus value” of 109% (they returned 2.09 points of value for every point spent on them).  
 

Justin Jefferson returned 1700 points in the redraft versus the 780 points spent on him, so he had a surplus value of 920 points.  (As  I think about it, I don’t really like the percentage method much, as it really overvalues a seventh rounder who may have had a draft value of 3 points who would have gone in the sixth round with a value of 12 points for a 400% return whereas a Justin Herbert was drafted at a cost of 1600 and a redraft taking him first overall would score him at 3000 points for less than a 200% return.)

 

One note is that UDFAs should absolutely be included in any method and given a draft capital score of 0?

 

 

[JGMcD2]

36 minutes ago, Billl said:

I guess I didn’t realize that this was your own creation.  You’re never going to achieve consensus trying to quantify the unquantifiable, but you need it to pass the common sense test.  If I were to put something similar together, I would start by combining methods that are already established.  For example, draft value charts aren’t exact, but they’re pretty good in terms of estimating trade equivalencies.  My step 1 would be to total up the draft value points used.

 

My second step would be to find a measure of determining a player’s contribution irrespective of draft position.  There are plenty of versions of these as well, so it’s a matter of picking your favorite.  For the sake of argument, I’ll go with Pro-football-reference.com and their Approximate Value stat.  It’s far from perfect (I don’t think Fred Warner had a better season than Patrick Mahomes, for example), but it’s a decent jumping off point.  (That said, it has Willie Gay at 4 and Sneed at 3, so even this method is going to be terrible.)

An possibly better, though more tedious, way would be to reference a redraft and assign players values based on the draft chart value of their redraft position.  Here’s one that I found, but I have no idea on its quality.  https://bleacherreport.com/articles/2926937-redrafting-the-2020-nfl-draft

It shows Sneed at 25 and CEH at 32, for reference.

 

From there, I would take the sum total of AAV (or redraft score) of the draft class and divide it by the total draft chart points.

 

A quick example using Sneed and CEH would show that 627 points of draft value were used on them in the actual draft.  The redraft positions are worth 1310 points, so they would have a “surplus value” of 109% (they returned 2.09 points of value for every point spent on them).  
 

Justin Jefferson returned 1700 points in the redraft versus the 780 points spent on him, so he had a surplus value of 920 points.  (As  I think about it, I don’t really like the percentage method much, as it really overvalues a seventh rounder who may have had a draft value of 3 points who would have gone in the sixth round with a value of 12 points for a 400% return whereas a Justin Herbert was drafted at a cost of 1600 and a redraft taking him first overall would score him at 3000 points for less than a 200% return.)

 

One note is that UDFAs should absolutely be included in any method and given a draft capital score of 0?

 

 

Thanks, this is very well thought out.

 

Related to your second step, I did find a measure of determining a player's contribution irrespective of draft position. I did use Pro Football Reference's Approximate Value metric in order to create this. I don't mean to be rude, but you told me you fully understood everything that was laid out in this analysis, but you didn't know that I used AV?

 

What I did was found the average value of a player drafted in each round (I'm toying with switching to median for several reasons). The average value won't be the same each draft, because teams are only selecting from a limited pool of players. What I am trying to do is to see how GMs perform against other GMs in extracting value from the round compared to their peers. After I found the average value of 1st round pick, 2nd round pick, 3rd round pick, etc. I subtracted the player's AV from the average value of a player picked in that round to determine the TOT_NETAV. This is how much better that player was than the average player picked in the same round as them in their draft class. 

 

The Chiefs netted positive value from all their picks, outside of their 3rd rounder who opted out, because he was drafted high and didn't contribute, it penalized the Chiefs pretty significantly. Had he just been Net 0 the Chiefs draft class would have graded out better than Buffalo. It's likely over time, with more data (which makes things more stable) things will lean towards the Chiefs. 

 

I do like the idea of using the draft trade points, but if I am not mistaking those points are based on all-time draft value. I don't necessarily think that fits what I am trying to do, because I believe each draft is its own world. I may try and find a way to incorporate it, and it's something that crossed my mind but I couldn't determine the best way. My issue is the talent level varies, so GMs shouldn't be penalized for drafting an average player in a weak draft, when compared to their peers selections in that same draft, they actually found more value with their picks. Basically each year, GMs can only work with and select the talent made available to them in the draft, historical value of those picks isn't necessarily predictive of what players in a given draft class will do.

 

I can definitely do something with UDFA the more I think about it, it's just harder for me to accumulate the information that I need to. I will have to do most of it by hand.

Edited January 27, 2021 by JGMcD2

[Mikie2times]

On 1/25/2021 at 1:34 PM, JGMcD2 said:

I put this together about 2 months ago, but with the season over and AV now updated, I figured it would be a good time to revisit. I agree with a lot of the posters saying that we're missing pieces. Of course we are, but I'm tired of the assertions that we need to draft better. We've objectively been THE BEST drafting team in the NFL from 2017-2019. I don't feel comfortable using a 1 year sample from 2020 just yet, but truthfully at this point it will drop the Bills 1 MAYBE 2 spots overall (I haven't had the chance to break it all down yet, but I will update when I do). 

 

I know people are going to try and rip this to shreds as stupid, but I have read numerous articles using AV as a way to compare draft classes. Many on The Athletic, like this article where they evaluate the Saints 2017 draft class using AV (my original analysis was done almost 3 weeks prior to this coming out, and yes it reflects the Saints having the best draft class in 2017 by a wide margin).

 

Here is my methodology from the original post...

 

Nobody really had a great way to measure success in the draft outside of their perception of a player... I wanted to make an attempt at examining this objectively. Pro Football Reference has their Weighted Approximate Value which assigns a value to a player based on their performance. It's not an all encompassing stat like WAR is in baseball, it definitely has its flaws, but PFR said it's steady to use to measure draft success. I'm going to dive into the results below on the draft, as well as some analysis I have done of the FA signings. Nothing is really over the top, I'm going to take some feedback and try to refine this. I had to do a lot of it by hand in excel and couldn't just scrape everything because different pieces were all over.

 

I chose 2017 specifically because that is when Sean McDermott and Brandon Beane arrived. Some folks want to argue that Beane shouldn't get credit for that draft and FA, which is perfectly fine. I am in the camp that believes McDermott and Beane are in lockstep, they make up the upper management of the football department, therefore the decisions made since 2017 have been made with certain goals in mind. I don't feel the need to omit 2017 because it was scrambled, it's very clear these two are close and the decisions made were made by McBeane in some way, shape or form starting in 2017. 

 

"Sometimes, for example if you want to assess a trade or determine the top draft classes of all time, you need a metric that is capable of comparing players across positions and eras. In baseball and basketball, lots of stats have been cooked up to do this, and they can do so with a reasonable degree of precision. In football, no such stat exists. In most cases, people use "starter" or "number of years as a starter" or "number of pro bowls" as the metric when they have to compare across positions.

AV is intended to be an improvement over those metrics, and nothing more. It is not Not NOT an ubermetric whose purpose is to decide once and for all who the best players in NFL history were." - Doug from PFR

 

The career AV is computed by summing 
100 percent of the AV of his best season,
95 percent of the AV of his next-best season,
90 percent of the AV of his third-best season,
and so on 

 

What I did was found the average value for a player drafted in each round (1-7) in each year (2017-2019) and the calculated what I am calling the Net Drafted Accumulated Value (NETDrAV) for each pick in each round. I only compared each draft to itself. I then found the Total Net Drafted Accumulated Value (TOT_NETDrAV) for each team in each draft and ranked them against each other. Rather than just looking at how much raw value the Bills brought in as compared to the 31 other teams, this gives an idea of how much extra value they extracted in each round as compared to the 31 other teams in the league. 

 

TOT_NETDrAV is the AV that a team has benefitted from directly on their team as compared to players and teams in their draft class.

 

TOT_NETCarAV is just the pure amount of value that players drafted by a given team have produced as compared to players and teams in their draft class.

 

 

Yes, I agree that they're missing something, but this constant assertion that this regime are "average drafters" or that we "have too many busts" is flat out wrong. You are only good or bad by comparison, and they're better than the rest of the NFL. 

 

They've extracted more value than any other team in the league over their first 3 years in the NFL. This will continue to change as players perform, but right now they have a clearly performed well drafting. It's not even debatable, Buffalo, New Orleans and Baltimore have been far and away the best drafting teams in the NFL from 2017-2019.

 

Do we need more impact players? Yes, every team does and actively seeks them out. But this notion that we don't draft well is garbage, complete and utter garbage. 

I really applaud your efforts to evaluate this, that said AV in a sample size this small, used for this purpose, is subject to a lot of problems. It was created to be a one size fits all stat for value of a player across any time in league history and at any position. If you just read on how AV is calculated for football you would understand immediately it can’t be used nor was it intended to be used for this purpose.
 

As a simple example, just the success of a player like Brees, never drafted by the Saints, would automatically increase the AV of any drafted player that stepped on the field for the Saints on offense.
 

Did you not start to notice a pattern of the highest rated teams also being some of the more successful teams the last 3 years? In sum, with AV, if you have a top unit and some draft picks played on that unit, they will get more credit assigned just based on the way AV works, even if what is making it a top unit isn’t players you drafted. Further, if you draft a QB who plays on a top unit you basically go to the top regardless. That last part probably has some validity. That said, I’m certainly not labeling this regime good at drafting based on AV average or one player.
 

You would have a hard time naming more than 3-4 impact guys since 2017.  That said we have drafted droves of role players that have playing time, which certainly helps juice that AV score. 
 

Read for yourselves and decide how relevant this data is in evaluating draft classes.

 

 

https://www.pro-football-reference.com/blog/index37a8.html

Edited January 27, 2021 by KzooMike

[JGMcD2]

39 minutes ago, KzooMike said:

I really applaud your efforts to evaluate this, that said AV in a sample size this small, used for this purpose, is subject to a lot of problems. It was created to be a one size fits all stat for value of a player across any time in league history and at any position. If you just read on how AV is calculated for football you would understand immediately it can’t be used nor was it intended to be used for this purpose.
 

As a simple example, just the success of a player like Brees, never drafted by the Saints, would automatically increase the AV of any drafted player that stepped on the field for the Saints on offense.
 

Did you not start to notice a pattern of the highest rated teams also being some of the more successful teams the last 3 years? In sum, with AV, if you have a top unit and some draft picks played on that unit, they will get more credit assigned just based on the way AV works, even if what is making it a top unit isn’t players you drafted. Further, if you draft a QB who plays on a top unit you basically go to the top regardless. That last part probably has some validity. That said, I’m certainly not labeling this regime good at drafting based on AV average or one player.
 

You would have a hard time naming more than 3-4 impact guys since 2017.  That said we have drafted droves of role players that have playing time, which certainly helps juice that AV score. 
 

Read for yourselves and decide how relevant this data is in evaluating draft classes.

 

 

https://www.pro-football-reference.com/blog/index37a8.html

He talks about using it in the exact context you’re saying it’s not supposed to be used. 
 

“Essentially, AV is a substitute for --- and a significant improvement upon, in my opinion --- metrics like 'number of seasons as a starter' or 'number of times making the pro bowl' or the like. You should think of it as being essentially like those two metrics, but with interpolation in between. That is, 'number of seasons as a starter' is a reasonable starting point if you're trying to measure, say, how good a particular draft class is, or what kind of player you can expect to get with the #13 pick in the draft. But obviously some starters are better than others. Starters on good teams are, as a group, better than starters on bad teams. Starting WRs who had lots of receiving yards are, as a group, better than starting WRs who did not have many receiving yards. Starters who made the pro bowl are, as a group, better than starters who didn't, and so on. And non-starters aren't worthless, so they get some points too."

 

I am also pretty confident you didn’t actually read what I wrote. I’m not doing anything based on a team’s AV average. It’s based on how much better a player has been graded via AV than the average player drafted in the same round as them nor does it say they’re good at drafting based on drafting one player. 
 

I think you’re really missing the point, sure they only have 3-4 impact guys since 2017... how many teams have drafted more than that? I’m going to go out an a limb and say not many, and if they have it’s only 1-2 more. And why wouldn’t a team be rewarded for getting value out of the draft? They’re not role players if they’re starting on a Playoff Caliber team... look at excerpt above. Starters on good teams, as a group are better than starters on bad teams. 
 

You’re dismissing the concept of finding role players, but other teams simply don’t do that well. You can’t take for granted the value of finding contributors. While I concede you need impact players, you cannot expect to get them with every pick. The better you do finding contributors in the draft, the more options you have as an organization. Accumulating a lot productive talent with no non-productive talent is better than accumulating some productive talent and some non-productive talent. 
 

That’s unbelievably common in professional scouting... just because a player is getting playing time on a bad team and putting up numbers doesn’t mean he should be rewarded greater than a player on a good team who is facing challenges for touches but is still producing at a high level. The player producing moderately on a great team is more valuable than the player producing well as the only option on a bad team. There are only so many touches to go around. 
 

The final point is that a gave a point of caution to the small sample size and how drafts that the 2017 draft is a much more accurate reflection of the value than 2020, as the 2020 draft is really just a snapshot in time at this moment. 
 

EDIT: More Excerpts 

 

Which teams have done the best jobs of drafting? To answer this, we'd need a tool that measures value across positions. Likewise, this post about how teams are built could be made more accurate. Instead of simply counting a starter as a starter, we could weight the more important starters more heavily, and we could include the non-starters as well. In other words, instead of saying things like "Team X got 4 of its 22 starters in the first round", we could say more meaningful things like, "Team X got 31% of its contributions from first round picks."

 

The value approximation method is a tool that is used to make judgements not about individual seasons, but about groups of seasons. The key word is approximation, as this is the one tool in our assortment which makes no attempt to measure anything precisely. The purpose of the value approximation method is to render things large and obvious in a mathemtatical statement, and thus capapble of being put to use so as to reach other conclusions.

 

 

Win shares is a top-down approach which starts with the number of games a team won, and then attempts to assign credit to players, proportionally based on their statistics.

 

It’s not identical to win shares, but it uses the same methodology. 

Edited January 27, 2021 by JGMcD2

[Big Turk]

5 hours ago, Billl said:

What an amazing cherry pick.  You just compared the two most productive players from the Bills class.  Scratch that.  You just compared the ENTIRETY of the production of the Bills draft class to one pick from the Chiefs class.  Were all of the Bills apples more productive in 29 games played than one of the Chiefs oranges was in 13 games played?  Sure.  Would any GM in the league, if given the option, take Davis and Moss over Edwards-Elaire?  Absolutely not.

 

That said, the rest of the Bills class to date has netted a roughly average season by a Kicker and absolutely nothing else.  The rest of the Chiefs class has produced a starting LB, a #1 CB, and a DE who played 50% of the snaps.

 

I just don’t get why this is even a conversation.  Without Davis, who had a solid but unspectacular season as a #4 Wideout, the Bills draft has been a total bust.  Again, it’s way way too early to start labeling picks as busts, but if we’re talking about one season of sample size...yikes.

 

Epenesa (who I wanted the Chiefs to draft) just doesn’t look like an NFL player walking out of the locker room, and his performance on the field has been negligible.  So far, he looks like a bad pick.

 

Moss looks like a guy with some power in short yardage but who has no speed and no value catching the ball out of the backfield.  There are 50 guys on practice squads who can do that.  After consecutive drafts taking a RB in the third round, the Bills biggest hole on offense is at RB.  Looks like a bad pick.

 

Davis has looked good.  He doesn’t look elite, but he could turn into a nice piece.  Good pick.

 

Bass...was a kicker taken in the 6th round.  That’s a total waste of a pick.  He’d likely have been available as an UDFA.  If not, Sloman or Blankenship would have been.  
 

The rest didn’t even see the field.  
 

Let’s talk “value” though.  The Bills made selections worth 580 points according to the draft value chart.  The Chiefs made selections worth 1040 points.  If you throw out the first round pick to make a more fair comparison, KC had 450 points.  For the sake of discussion, we can call that even.  Buffalo got the players mentioned above (11 total starts) while Kansas City has a starting LB, a #1 CB, and a productive rotational DE.  How’s that for surplus value?

 

 

CEH was nothing special this year. Not sure what he did to deserve the excessive ball washing everyone seems to be giving him. James Robinson and Jonathan Taylor had better years than he did as rookies.

[Mikie2times]

21 minutes ago, JGMcD2 said:

He talks about using it in the exact context you’re saying it’s not supposed to be used. 
 

“Essentially, AV is a substitute for --- and a significant improvement upon, in my opinion --- metrics like 'number of seasons as a starter' or 'number of times making the pro bowl' or the like. You should think of it as being essentially like those two metrics, but with interpolation in between. That is, 'number of seasons as a starter' is a reasonable starting point if you're trying to measure, say, how good a particular draft class is, or what kind of player you can expect to get with the #13 pick in the draft. But obviously some starters are better than others. Starters on good teams are, as a group, better than starters on bad teams. Starting WRs who had lots of receiving yards are, as a group, better than starting WRs who did not have many receiving yards. Starters who made the pro bowl are, as a group, better than starters who didn't, and so on. And non-starters aren't worthless, so they get some points too."

Like I said, I applaud your efforts to solve what can’t be solved. But instead of using articles or partial sections I encourage you to discuss how AV is calculated in its entirety if you wish to defend against what I’m saying. Not just use partial sections. I’ve read that entire document and all the sub documents on how AV came to be a long time ago. 

 

Look at the Correlation between win % and your list, it’s very strong, now is that because they drafted well or is based on how AV is calculated? It assigns greater value to higher performing units. High performing units can be a result of one player you didn’t draft and 5 you did who are essentially role players. Those 5 players then get assigned much more value then if they played in moderate performing units.
 

Just look at how

many draft picks we started on defense since 2017 in context with it being a high performing unit. That said, who is the star? Tre is for sure, then you have  Edmonds? Milano? Well, you don’t need much more than that once you throw Allen in. QB gets assigned the most points of all. 
 

This regime has a moderate draft record. It’s not bad. It’s not great. It’s certainly not the best. This analysis is causation/correlation. We are good. We have started a lot of draft picks. Most of those draft picks aren’t primary reasons we are good. They are complimentary players with only a couple qualifying as players we have to retain. 

Edited January 27, 2021 by KzooMike

[MJS]

I remember this from before and really liked your methodology then. I believe we had a conversation about AV used to show the quality of free agent pickups over the same timeframe and I believe you showed that Buffalo shows well with their free agents too, correct? So, great at drafting and great at free agent signings as far as value added is concerned. Beane is clearly a great GM and if he keeps it up he'll be in the hall of fame one day.

 

I think people tend to look at the busts and ignore some of the successes. And they also look at their team in a vacuum, not understanding that EVERY team in the league has draft picks that bust and free agents that don't work out. People assume that the average success rate is much higher than it actually is.

 

Good stuff. I'd like to see this analysis continued for future seasons.

2 minutes ago, KzooMike said:

Like I said, I applaud your efforts to solve what can’t be solved. But instead of using articles or partial sections I encourage you to discuss how AV is calculated in its entirety if you wish to defend against what I’m saying. Not just use partial sections. I’ve read that entire document and all the sub documents on how AV came to be a long time ago. 

 

Look at the Correlation between win % and your list, it’s very strong, now is that because they drafted well or is based on how AV is calculated? It assigns greater value to higher performing units. High performing units can be a result of one player you didn’t draft and 5 you did who are essentially role players. Those 5 players then get assigned much more value then if they played in moderate performing units.
 

Just look at how

many draft picks we started on defense since 2017 in context with it being a high performing unit. That said, who is the star? Tre is for sure, then you have  Edmonds? Milano? What else do we have? Do Edmonds and Milano even count? Well, you don’t need much more than that once you throw Allen in. QB gets assigned the most points of all. 
 

This regime has a moderate draft record. It’s not bad. It’s not great. It’s certainly not the best. This analysis is causation/correlation. We are good. We have started a lot of draft picks. Most if those draft picks aren’t primary reasons we are good. They are complimentary players with only a couple qualifying as players we have to retain. 

I think you are dead wrong. What metrics are you using to back this up? You are just going by your gut. You are assuming a higher success rate for the NFL than is actual. You are also making huge assumptions with the AV metric that may or may not be true (and are likely false).

[Mikie2times]

9 minutes ago, MJS said:

I remember this from before and really liked your methodology then. I believe we had a conversation about AV used to show the quality of free agent pickups over the same timeframe and I believe you showed that Buffalo shows well with their free agents too, correct? So, great at drafting and great at free agent signings as far as value added is concerned. Beane is clearly a great GM and if he keeps it up he'll be in the hall of fame one day.

 

I think people tend to look at the busts and ignore some of the successes. And they also look at their team in a vacuum, not understanding that EVERY team in the league has draft picks that bust and free agents that don't work out. People assume that the average success rate is much higher than it actually is.

 

Good stuff. I'd like to see this analysis continued for future seasons.

I think you are dead wrong. What metrics are you using to back this up? You are just going by your gut. You are assuming a higher success rate for the NFL than is actual. You are also making huge assumptions with the AV metric that may or may not be true (and are likely false).

Lmao....read the link I posted. All of it. By default, any player on a higher performing unit who starts games gets more AV assigned. So by default any team that performs better has a higher AV assigned to players in that unit. Look at the correlation on his list with win% and do your own research. 

Edited January 27, 2021 by KzooMike

[MJS]

1 minute ago, KzooMike said:

Lmao....read the link I posted. All of it. By default, any player on a higher performing unit who starts games gets more AV assigned. So by default any team that performs better has a higher AV assigned to players in that unit. Look at the correlation on his list as do your own research. 

I've done my research. AV is a good metric. They do their best to isolate each player. Obviously that's not always possible. But their grades match closely, in my opinion, to what we see on the field from these players. And I'm not talking only about Bills players. I haven't actually looked at the grades for Bills players much. But I have looked league wide and at grades for historical players.

[JGMcD2]

23 minutes ago, KzooMike said:

Like I said, I applaud your efforts to solve what can’t be solved. But instead of using articles or partial sections I encourage you to discuss how AV is calculated in its entirety if you wish to defend against what I’m saying. Not just use partial sections. I’ve read that entire document and all the sub documents on how AV came to be a long time ago. 

 

Look at the Correlation between win % and your list, it’s very strong, now is that because they drafted well or is based on how AV is calculated? It assigns greater value to higher performing units. High performing units can be a result of one player you didn’t draft and 5 you did who are essentially role players. Those 5 players then get assigned much more value then if they played in moderate performing units.
 

Just look at how

many draft picks we started on defense since 2017 in context with it being a high performing unit. That said, who is the star? Tre is for sure, then you have  Edmonds? Milano? Well, you don’t need much more than that once you throw Allen in. QB gets assigned the most points of all. 
 

This regime has a moderate draft record. It’s not bad. It’s not great. It’s certainly not the best. This analysis is causation/correlation. We are good. We have started a lot of draft picks. Most of those draft picks aren’t primary reasons we are good. They are complimentary players with only a couple qualifying as players we have to retain. 

I’m defending against your obviously wrong point that says AV wasn’t designed to be used to evaluate draft classes. I don’t even need to continue after that, because you’re off base. 
 

You can look at the individual years as well if you’d like... in 2017 4 of the top 10 teams missed the playoffs this year (Chargers, Texans, Vikings, Panthers). 
 

LAC: Playoffs 1/4 times since 2017

HOU: Playoffs 2/4

MIN: 2/4

Panthers 1/4

 

Combined 6/16
 

In 2018 4 of the top 10 teams missed the playoffs this year (Falcons, 49ers, Eagles, Lions). 
 

ATL: Playoffs 0/3 years since 2018

49ers: Playoffs 1/3 years since 2018

Eagles: Playoffs 2/3 years 

Lions: Playoffs 0/3

Combined 3/12

 

In 2019 4 of the top 10 teams missed the playoffs this year (Dolphins, 49ers, Raiders, Cardinals). 
 

Dolphins: 0/2

Raiders: 0/2

Cardinals: 0/2

Combined 0/6 

 

In 2020 4/10 teams again didn’t make the playoffs (LAC, CAR, CIN, NYG) 

 

0/4 this year 

 

You’ve got 16 top 10 drafts and they’ve made the playoffs 9 times out of 36 possible chances? 25% of the time they’re making the playoffs and this is favoring good teams? 

14 minutes ago, KzooMike said:

Lmao....read the link I posted. All of it. By default, any player on a higher performing unit who starts games gets more AV assigned. So by default any team that performs better has a higher AV assigned to players in that unit. Look at the correlation on his list with win% and do your own research. 

You really don’t think I sat down and read all of that BEFORE I put all of this together? That’s how I knew exactly where to find everything that eradicated your statement that “It’s not supposed to be used like this.” 

Edited January 27, 2021 by JGMcD2

[Mikie2times]

1 minute ago, MJS said:

I've done my research. AV is a good metric. They do their best to isolate each player. Obviously that's not always possible. But their grades match closely, in my opinion, to what we see on the field from these players. And I'm not talking only about Bills players. I haven't actually looked at the grades for Bills players much. But I have looked league wide and at grades for historical players.

I agree AV is an excellent tool once the dust settles. But until then I can’t justify a rookie getting a larger pool of win shares to work with because his QB is Aaron Rogers. I can’t justify that the best drafting teams just so happen to all be teams that made the playoffs multiple times the last 3 years according to this list. We had the smallest representation of home grown players of any team in the AFC championship on our roster. The eye test tells me we have only drafted 3-4 players, we really want to keep.
 

I love this type of analysis and it’s probably as close to a sabermetric philosophy as you can get and I love that as well, but it’s not something you can just drop the mic on. It has a ton of flaws. Namely, who the heck are the players that make us so elite? Tre, Josh? Who else? If we draft that well

that shouldn’t be a hard thing to answer. Obviously Milano was very solid for where we landed him. 

1 minute ago, KzooMike said:

I agree AV is an excellent tool once the dust settles. But until then I can’t justify a rookie getting a larger pool of win shares to work with because his QB is Aaron Rogers. I can’t justify that the best drafting teams just so happen to all be teams that made the playoffs multiple times the last 3 years according to this list. We had the smallest representation of home grown players of any team in the AFC championship on our roster. The eye test tells me we have only drafted 3-4 players, we really want to keep.
 

I love this type of analysis and it’s probably as close to a sabermetric philosophy as you can get and I love that as well, but it’s not something you can just drop the mic on. It has a ton of flaws. Namely, who the heck are the players that make us so elite? Tre, Josh? Who else? If we draft that well

that shouldn’t be a hard thing to answer. Obviously Milano was very solid for where we landed him. 

 

6 minutes ago, JGMcD2 said:

I’m defending against your obviously wrong point that says AV wasn’t designed to be used to evaluate draft classes. I don’t even need to continue after that, because you’re off base. 
 

You can look at the individual years as well if you’d like... in 2017 4 of the top 10 teams missed the playoffs this year (Chargers, Texans, Vikings, Panthers). 
 

LAC: Playoffs 1/4 times since 2017

HOU: Playoffs 2/4

MIN: 2/4

Panthers 1/4

 

Combined 6/16
 

In 2018 4 of the top 10 teams missed the playoffs this year (Falcons, 49ers, Eagles, Lions). 
 

ATL: Playoffs 0/3 years since 2018

49ers: Playoffs 1/3 years since 2018

Eagles: Playoffs 2/3 years 

Lions: Playoffs 0/3

Combined 3/12

 

In 2019 4 of the top 10 teams missed the playoffs this year (Dolphins, 49ers, Raiders, Cardinals). 
 

Dolphins: 0/2

Raiders: 0/2

Cardinals: 0/2

Combined 0/6 

 

In 2020 4/10 teams again didn’t make the playoffs (LAC, CAR, CIN, NYG) 

 

0/4 this year 

You really don’t think I sat down and read all of that BEFORE I put all of this together? That’s how I knew exactly where to find everything that eradicated your statement that “It’s not supposed to be used like this.” 

You went this far. Care running number of wins by team from 2017-2020 against your final

roll up? See how lopsided this gets. Team is good, therefor they draft good. You said it yourself.  It follows the same concept as win shares. I can also run that data for you if you like tomorrow. 

[JGMcD2]

8 minutes ago, JGMcD2 said:

Look at the Correlation between win % and your list, it’s very strong, now is that because they drafted well or is based on how AV is calculated? It assigns greater value to higher performing units. High performing units can be a result of one player you didn’t draft and 5 you did who are essentially role players. Those 5 players then get assigned much more value then if they played in moderate performing units.

This is just a fundamental misunderstanding of value then, he lays it out. Players who start on good teams are much more valuable than players if they start on a bad team. It’s not that difficult. 

[MJS]

8 minutes ago, KzooMike said:

I agree AV is an excellent tool once the dust settles. But until then I can’t justify a rookie getting a larger pool of win shares to work with because his QB is Aaron Rogers. I can’t justify that the best drafting teams just so happen to all be teams that made the playoffs multiple times the last 3 years according to this list. We had the smallest representation of home grown players of any team in the AFC championship on our roster. The eye test tells me we have only drafted 3-4 players, we really want to keep.
 

I love this type of analysis and it’s probably as close to a sabermetric philosophy as you can get and I love that as well, but it’s not something you can just drop the mic on. It has a ton of flaws. Namely, who the heck are the players that make us so elite? Tre, Josh? Who else? If we draft that well

that shouldn’t be a hard thing to answer. Obviously Milano was very solid for where we landed him. 

Having elite players is not what that metric is about. Drafting well does not mean that you have a bunch of home grown elite players. You can draft a bunch of average starters and still have good AV relative to other teams. Another team could draft one or two elite players but the rest of their draft picks could be out of the league.

 

I think it makes perfect sense for the most successful teams, by wins and losses, to also be the teams who draft the best. And if you look at the teams on the top of the list, I think they do draft well and those teams are respected around the league for their ability to draft and develop players. Beane and the Buffalo Bills certainly get a ton of respect around the league as well.

 

I think where the metrics might be inflated are actually for the bad teams. They draft players and they start even if they suck because those bad teams have no other choice. So those bad players perhaps get a higher score even though they likely would not be playing at all or even make the roster of some of the good teams. And they try to account for that.

 

Edited January 27, 2021 by MJS

[Mikie2times]

7 minutes ago, JGMcD2 said:

This is just a fundamental misunderstanding of value then, he lays it out. Players who start on good teams are much more valuable than players if they start on a bad team. It’s not that difficult. 

Over the course of a career I agree with that statement. In this small a sample it’s really more about did they play for a good team or not.

 

 

I’m hitting the sack MJS, like I said. I applaud the work. I may not fully agree with how strongly you feel about it’s conclusions but I appreciate how you went about it. Look forward to future posts. 

Edited January 27, 2021 by KzooMike

[JGMcD2]

23 minutes ago, KzooMike said:

But until then I can’t justify a rookie getting a larger pool of win shares to work with because his QB is Aaron Rogers. I can’t justify that the best drafting teams just so happen to all be teams that made the playoffs multiple times the last 3 years according to this list. We had the smallest representation of home grown players of any team in the AFC championship on our roster. The eye test tells me we have only drafted 3-4 players, we really want to keep.

Ironically the Packers have been one of the worst drafting teams over the past 4 years...

 

That AFC Championship numbers is skewed and you know it too. Beane and McDermott traded, cut or let every player walk that was drafted by previous regimes. They have legitimately have 4 draft classes to go off of... and I count 9 players that were listed as starters in the AFC Championship game on Sunday. It's nearly impossible for them to have trotted out a championship roster after purging the roster of past regimes picks and only having 4 drafts to bring in their own guys. 

 

23 minutes ago, KzooMike said:

I love this type of analysis and it’s probably as close to a sabermetric philosophy as you can get and I love that as well, but it’s not something you can just drop the mic on. It has a ton of flaws. Namely, who the heck are the players that make us so elite? Tre, Josh? Who else? If we draft that well

that shouldn’t be a hard thing to answer. Obviously Milano was very solid for where we landed him. 

 

You may very well be underestimating how BAD some teams are at actually drafting players. It wasn't intended to be a mic drop, and I've been perfectly candid about it not being completely perfect or even THE BEST, although I argue it's better than any completely subjective argument someone presents. I've been candid about newer drafts not necessarily being stable because of the small sample size, but the other thing you're not taking into account is each draft is compared to the performance of itself. It's how much more value White provided than drafting the average value of a player in the 1st round of his draft class. So it's not completely unstable either in recent years because you're comparing players with equal time in the league to each other... I would argue that recent drafts actually may favor weaker teams because there is more opportunity for drafted players to come in and play right away...

14 minutes ago, KzooMike said:

Over the course of a career I agree with that statement. In this small a sample it’s really more about did they play for a good team or not.

Yeah, but why wouldn't a rookie/younger player on a good team be rewarded for playing a significant amount of snaps? It's harder to gain the ability to contribute on a competitive team.

 

A rookie/younger guy on a bad team in the short term is likely going to have more opportunities to play right away than not.

Edited January 27, 2021 by JGMcD2

[JGMcD2]

@KzooMike

 

In comparison to Buffalo, this is New England (2017-2019 as I just grabbed it quick). The worst drafting team over the last 4 years (#32). They've made the playoffs as often as Buffalo has, extremely competitive team. They actually have a better record over the last 4 years than we do, they're actually the 4th most winning team over that time span. Belichick has done an absolutely horrible job of finding any sort of value consistently. He pretty much gets 1 contributor each year, but a lot of his early picks are just horrible values. He's done better with guys like Wise (4th), Bentley (5th), Winovich (3rd) than he has with anything in earlier rounds. 

 

 

I'll throw in GB (#23) seeing that they've been in the conference championship the last two seasons. That's a lot success... out of the team's listed starters on Sunday, they had 8 drafted between 2017-2019. 

 

 

Houston (#13). Generally do a pretty good job finding some value, but really propped up by Watson. Rightfully so, he's the most important draft pick they have made and most important player on the field for them each week. 

Edited January 27, 2021 by JGMcD2

[Mikie2times]

5 minutes ago, JGMcD2 said:

@KzooMike

 

In comparison to Buffalo, this is New England (2017-2019 as I just grabbed it quick). The worst drafting team over the last 4 years. They've made the playoffs as often as Buffalo has, extremely competitive team. They actually have a better record over the last 4 years than we do, they're actually the 4th most winning team over that time span. Belichick has done an absolutely horrible job of finding any sort of value consistently. He pretty much gets 1 contributor each year, but a lot of his early picks are just horrible values. He's done better with guys like Wise (4th), Bentley (5th), Winovich (3rd) than he has with anything in earlier rounds. 

Tee Higgins and Gabe Davis had the same AV in 2020 (6).
 

Higgins had 67, 908, and 6 TD’s with career backups throwing him the ball after Burrow got hurt.

 

Davis had the 2nd best QB in football throwing him the ball according to AV, and went 35, 599, and 7 TD’s. 
 

Devin Singletary is an 8 along with Nick Chubb and his 1000+ yards. 
 

I can do this all day and with way more extreme examples than I used. The point is over a career this stuff balances out. Over this small a sample it just doesn’t. 

[JGMcD2]

18 minutes ago, KzooMike said:

Tee Higgins and Gabe Davis had the same AV in 2020 (6).
 

Higgins had 67, 908, and 6 TD’s with career backups throwing him the ball after Burrow got hurt.

 

Davis had the 2nd best QB in football throwing him the ball according to AV, and went 35, 599, and 7 TD’s. 
 

Devin Singletary is an 8 along with Nick Chubb and his 1000+ yards. 
 

I can do this all day and with way more extreme examples than I used. The point is over a career this stuff balances out. Over this small a sample it just doesn’t. 

Again, that wasn’t your initial issue with the process. You claimed it wasn’t even meant to compare draft classes in the first place. 
 

Again, I never said it was stable over a short time period. I initially only looked at 2017. Which has 4 years worth of information on players, the Bills did well. You picked two players from the most recent draft to prove your point, I’ve already said and agreed it’s not as stable with lack of volume. But again, Gabe is rewarded for producing on a competitive team. There’s nothing wrong with that. That’s important, you don’t think if they swapped places their statistic would look similar? 

In scouting it’s very common for these discussions to happen. I’ll use baseball as a better example. John Means is the ace of the Baltimore Orioles. He’s the ace on the Baltimore Orioles because they’re bad. If you take John Means and stick him on a playoff caliber team, is John Means still the ace? No, he’s a #5 starter on a contending team. Is he more valuable just because he’s an ace on a bad team? No, he’s more valuable as a 5th starter on a contending team. 
 

I think a player gaining SOME value because they’re starting and/or contributing on a successful team is more than valid. Gabe Davis is contributing to a better football team and there’s a difficulty in cracking that lineup when there’s talent on the roster. Generally better teams have better players at every position. Don’t get me wrong, Higgins is a great player, but is he contributing more than Davis on this Bills team? Maybe or maybe he looks about the same because he’s fighting for targets with a bunch of players on the Bills roster. 
 

I’m looking now. The Bengals had 566 passing targets. Higgins, Green and Boyd were all targeted over 100 times. After that 59 targets for Bernard and 53 for Sample. Then a bunch of guys getting anywhere from 1-20. 
 

The there’s Buffalo. 572 targets. Diggs has 166 and Beasley has 107. Davis at 62. Brown at 52. Singletary at 50. Knox at 44. McKenzie 34. More mouths to feed. Is Higgins really going to come in and have Davis’ 62 targets and then steal 40 more targets from somewhere else? 
 

On the flip side, is Davis going to walk into Cincinnati and still only get 62 targets with their lack of options? It’s likely going to be around 100 like Higgins. Which in that case he blows Higgins numbers out of the water. 
 

Give Davis the same volume of targets as Higgins and he’s 61/1,034/12. Give Higgins the same volume as Davis and you get 38/521/4. 

 

For every extreme example you find, I can find one too. That happens, that doesn’t mean the process is broken. Flawed, sure, I’m not debating that.
 

As I’ve asked others,  I invite you to find a better way to do this objectively.