Once and for all....PFFffffffft

[Jay_Fixit]

14 hours ago, Einstein said:

 

You really think NFL teams would pay millions of dollars for useless data? The idea that you think it's some schmuck sitting at home is part of the problem.

 

PFF employs former NFL players, especially at Tier 2 and 3 of grading. The reason why PFF grades take a couple day to update after a game is because they are not graded by one player. It gets graded by a Tier 1 analyst, then a Tier 2 analyst, and then sometimes a Tier 3 analyst. 

People will often write "they don't know the play call". These people clearly think that football is rocket science. It's not. We can see on All-22 what the play is. Especially by mid season when teams are running the same plays over and over. And no, we do not accept the idea that maybe this particular team told their left tackle to get pancaked on a counter-run because the coach told him to. No, he just got beat. 

The other excuse is "teams pay for the data, not the grades". How do you think PFF grades the player? On the data!

Ladies and gentlemen, I present Mrs PFF.

[4merper4mer]

The PFF ratings on the Thanksgiving game when Tre’ White got hurt, so healthy Tre’ White, had him ranked as the 98th best CB in the NFL.  I agree with JJ Watt about half as frequently as I agree with a broken clock but this is the time.

[boyst]

When pff was coming or years ago I had friends in sports management at BGSU. They offered jobs to students who passed a few basic tests to chart and grade players per game. These were students who may not even watch the game or didn't even know football.

 

Pff is ***** 

[4merper4mer]

1 minute ago, boyst said:

When pff was coming or years ago I had friends in sports management at BGSU. They offered jobs to students who passed a few basic tests to chart and grade players per game. These were students who may not even watch the game or didn't even know football.

 

Pff is ***** 

Bowling Green has an orange logo.  

[boyst]

33 minutes ago, 4merper4mer said:

Bowling Green has an orange logo.  

And brown!

[DCOrange]

13 hours ago, BlazinBill said:

IMO having a response to criticism like that just screams lacking belief in what you do-it’s really not productive. If PFF believes in what they do a simple no response says more- to me anyway .

 

 

It’s just an easy way for them to get more clicks.

[HoofHearted]

15 hours ago, Beck Water said:

I'm not a scout, but I'm not sure that's the way "most NFL coaches grade" and would welcome some input on this - any input from guys who've coached at the college or HS level @HoofHearted @Buffalo716?

 

Either way, I think the main point is that the coaches (and scouts) are typically people who have a deep and nuanced knowledge of the game.  They're people who can diagnose the defensive coverage based on subtle cues like footwork and body position.  We saw this kind of thing with Tony Romo when he started his broadcast career, where he could accurately read the defense and predict what the offense was going to do and the network told him to tone it down.  The ability of these people with a lifetime football background, playing and coaching, to deduce the play call, or the coverage, or the protection call, after looking at film, is on a different level.

 

It's my understanding that PFF uses a lot of graders in Ireland and now India whose understanding of the game is taking a few training sessions.  Now they claim that only 10% of their employees finalize grades or something like that, and that everything is reviewed, but 🤷‍♂️?

 

Some critiques of PFF:

https://sportank.com/media/post/is-pff-reliable-are-they-all-that-credible

 

I picked these because neither are out to just dismiss or trash PFF.

PFF grades are for fans. They serve no purpose to teams.

[BarleyNY]

17 hours ago, Einstein said:

Is this really surprising? 

You can have two NFL coaches grade a player completely differently.

Two GM's too.

That's why some will trade down in the draft and others will trade up to get the player that the other left there.

It doesn't mean PFF sucks. It means HIS coach graded it differently.

 

For ratings to be useful people have to understand what they really mean, which includes what they’re actually measuring as well as their limitations. I like a lot of what PFF does, but their grading has limited utility. It’s better with some positions than others, but is still limited to what players are asked to do rather than what they’re capable of. For instance, a QB like Cousins who plays a much more limited role in his team’s offense, can more easily score a higher grade than a QB like Allen, who much more is asked of.

 

Sam Monson recently mentioned this on a podcast when PFF’s grading came up. He used Dodson as an example of a player who graded very highly due to him doing well in a very limited role. It makes sense that coaches would be more interested in a player’s overall impact on a game and would factor in things like the difficulty of the assignment, the ability of the player, the contract of the player, etc. Each has their use and a value, but they aren’t going to be the same. 

[DCOrange]

13 hours ago, JoPoy88 said:

I 100% agree with you. If PFF put out more publicly on how these graders are trained and instructed, it might alleviate some of this bewilderment for some of their grades and hatred for them in general. But they don’t do that. Wonder why?

They have done this several times over the years. Unsurprisingly, most of the people (not saying you, just the general people that go crazy on Twitter and such) that complain about them don't really care to invest time in reading/listening/watching. The link below includes guides as to how they grade the different position groups.

 

It does not include answers regarding who is physically doing the grading, who reviews the graders, etc., but I believe that's been answered elsewhere as well. If I remember right there's like a three-level review process on every game.

 

https://profootballfocussupport.zendesk.com/hc/en-us/articles/360029567633-How-does-PFF-grade-each-position-

 

 

Edited January 18 by DCOrange

[JoPoy88]

10 minutes ago, DCOrange said:

They have done this several times over the years. Unsurprisingly, most of the people (not saying you, just the general people that go crazy on Twitter and such) that complain about them don't really care to invest time in reading/listening/watching. The link below includes guides as to how they grade the different position groups.

 

It does not include answers regarding who is physically doing the grading, who reviews the graders, etc., but I believe that's been answered elsewhere as well. If I remember right there's like a three-level review process on every game.

 

https://profootballfocussupport.zendesk.com/hc/en-us/articles/360029567633-How-does-PFF-grade-each-position-

 

 

I’ve read all this in the past, along with Matthew Coller’s book. The example clips they give do help add some understanding of it. 
 

Still need to know who’s doing these evaluations and how they are qualified and vetted. 
 

There are still too many head-scratching grades on a semi-regular basis with this outfit. 

30 minutes ago, HoofHearted said:

PFF grades are for fans. They serve no purpose to teams.

exactly. The player grades are an entertainment product. They’re glorified power rankings.

[DCOrange]

3 minutes ago, JoPoy88 said:

I’ve read all this in the past, along with Matthew Coller’s book. The example clips they give do help add some understanding of it. 
 

Still need to know who’s doing these evaluations and how they are qualified and vetted. 
 

There are still too many head-scratching grades on a semi-regular basis with this outfit. 

At the end of the day, I don't think it's wise for people to treat PFF grades as "gospel" as I believe JJ Watt said it. As best as they try, grading football players is ultimately a subjective exercise, regardless of if it's a nerd or a coach doing the grading. If you're willing to pay for what IMO is a very expensive subscription, I believe you are able to get a somewhat detailed breakdown of what went into each player's grade in a given week but that's just not an additional expense that I need in my budget lol.

 

Anecdotally, I feel like when there's occasional viral moments like this where they do come out and explain a specific rating, it generally makes sense. I'm sure there are plays that can be nitpicked but I appreciate when they show their work on stuff like Stroud vs. Love and that one game several years ago when Aaron Rodgers threw for like 400 yards and 5 TDs but received a pretty mediocre grade because it was basically all off of screen passes and busted coverages. I generally feel like I agree with or at least understand situations where a player has great box score stats but a relatively low grade (Jordan Phillips 9.5 sack season with us is one I remember off the top of my head). But obviously I'm only really looking at players that I care about; I can't say I have any idea if Peter Skoronski's run blocking grade is accurate or not.

[Einstein]

7 minutes ago, BarleyNY said:

 

For ratings to be useful people have to understand what they really mean, which includes what they’re actually measuring as well as their limitations. I like a lot of what PFF does, but their grading has limited utility. It’s better with some positions than others, but is still limited to what players are asked to do rather than what they’re capable of. For instance, a QB like Cousins who plays a much more limited role in his team’s offense, can more easily score a higher grade than a QB like Allen, who much more is asked of.

 

Sam Monson recently mentioned this on a podcast when PFF’s grading came up. He used Dodson as an example of a player who graded very highly due to him doing well in a very limited role. It makes sense that coaches would be more interested in a player’s overall impact on a game and would factor in things like the difficulty of the assignment, the ability of the player, the contract of the player, etc. Each has their use and a value, but they aren’t going to be the same. 

 

I’ve noticed that whenever it comes to PFF, people turn to confirmation bias. They will latch onto anything that proves their side in the debate. So let’s simply look at the facts.

 

”NFL teams don’t care about PFF grading” - Except every single NFL team pays PFF, as well as 80+ college teams. Guess they see PFF as a charity!

 

”PFF grades serve no purpose to teams” - Except teams and players use these grades in contract negotiations. “You'd be amazed that Pro Football Focus is used in almost every contract negotiation known to man. Because either the team likes our grade because it helps them negotiate, or the player likes the grade because it helps them negotiate. A lot of times, we get asked by agents and by teams, 'All right, give us the strength of this guy' or 'Give us the weakness of this guy to help us negotiate.'” - Chris Collinsworth

 

”Well yeah the teams pay PFF but it’s just for their data, not for their grades” - How do you think PFF grades? From their data…

 

“But they just hire a bunch of schmucks to grade” - The final grade is approved or denied by former NFL coaches. To quote PFF, “about 40 or 50 former coordinators and head coaches that have been in the league that have the final say. So we have four different people look at every single play that takes place, and the former coaches have all the veto power.”

 

I guess we could just ask some NFL coaches too… 

 

“Pro Football Focus has done a nice job with their stats and their analysis. I think they’re pretty accurate with things.” - Andy Reid, Chiefs head coach

 

“I think it’s impressive the work they do” - Gary Kubiak, head coach when the Denver Broncos won the Super Bowl

 

"I don't think there's any question that they do a great job, and I think they're a valuable resource for us." - Kyle Shanahan, San Francisco 49ers head coach

 

“We attributed all three special teams return TDs in 2017 to PFF data.” – FBS Special Teams Coach

 

 

 

So, let me put this all together and get this straight. NFL teams see no value in a product that pay a ton of money for, which is graded by former NFL coaches (which to my knowledge, no one on this forum is a former NFL coach), but the fans on this forum know better.

Ok.

 

[JoPoy88]

3 minutes ago, DCOrange said:

At the end of the day, I don't think it's wise for people to treat PFF grades as "gospel" as I believe JJ Watt said it. As best as they try, grading football players is ultimately a subjective exercise, regardless of if it's a nerd or a coach doing the grading. If you're willing to pay for what IMO is a very expensive subscription, I believe you are able to get a somewhat detailed breakdown of what went into each player's grade in a given week but that's just not an additional expense that I need in my budget lol.

 

Anecdotally, I feel like when there's occasional viral moments like this where they do come out and explain a specific rating, it generally makes sense. I'm sure there are plays that can be nitpicked but I appreciate when they show their work on stuff like Stroud vs. Love and that one game several years ago when Aaron Rodgers threw for like 400 yards and 5 TDs but received a pretty mediocre grade because it was basically all off of screen passes and busted coverages. I generally feel like I agree with or at least understand situations where a player has great box score stats but a relatively low grade (Jordan Phillips 9.5 sack season with us is one I remember off the top of my head). But obviously I'm only really looking at players that I care about; I can't say I have any idea if Peter Skoronski's run blocking grade is accurate or not.

That’s all perfectly fine - so long as it’s made clear that these things are purely subjective analyses. These are people watching tape and assigning grades on a scale that they created. 
 

My main problem with them is these two sides of their business are conflated together and now everything is shrouded under PFF’s data analytics. And they really don’t seem interested in speaking up and differentiating the two to the public. 

[Sierra Foothills]

17 hours ago, Punch said:

Not to defend PFF, but "literally putting a coaches grade next to a PFF grade" doesn't really prove much considering how many meatheads are coaching in the NFL. Coaches are constantly wrong when evaluating talent and situation. 

 

Yes player evaluation is highly subjective... even for NFL coaches.

 

17 hours ago, Einstein said:

Is this really surprising? 

You can have two NFL coaches grade a player completely differently.

Two GM's too.

That's why some will trade down in the draft and others will trade up to get the player that the other left there.

It doesn't mean PFF sucks. It means HIS coach graded it differently.

 

Subjective even for NFLscouts and GMs.

 

17 hours ago, Buffalo716 said:

Their grades are highly subjective because they do not know the play call

 

Pro scouts who work for NFL teams and scouting syndicates also don't usually know the play call.

 

16 hours ago, MJS said:

Wow! I'm a huge fan of JJ Watt now. Absolutely savage. I love it.

 

Dude is a meathead and he spurned the Bills so strikes 2 and 3.

 

16 hours ago, eball said:

You really think some schmuck sitting at home "grading" a play when he doesn't know the play call or the individual player responsibilities is the same as a coach grading a play?

 

1) One can't assume the competency of PFF evaluators. Some have worked at high levels in the NFL as coaches and players.

 

2) Your argument dictates that only the handful of people who know the play call are justified in judging the results. This argument then disqualifies NFL coaches and scouts who usually don't know the play call.

 

16 hours ago, BADOLBILZ said:

Just like scouts grades.    Totally subjective because they do not know the play call.  

 

Exactly.

 

16 hours ago, Einstein said:

to be honest, I disagree with several of PFF grade. Ed is just one of them.

but keep in mind that two NFL coaches may disagree with each other on their own grading.so, in my opinion, that’s not that unusual.

 

Correct.

 

15 hours ago, Mango said:

I don't disagree on this topic, but JJ Watt is also a dummy. 

I have no other value to offer to this conversation. 

 

JJ Watt is Gronk without the rings.

 

14 hours ago, Punch said:

Meanwhile, PFF grades are "molded by the NFL" as our old friend Matthew Coller tweeted (I'll reiterate for the third time, I do not necessarily wholeheartedly endorse PFF):

 

 

Andre Whitworth, Paul Alexander and Bobby Slowik? Not bad.

 

Critics of PFF are assuming the caliber of the evaluators without actually knowing who they are.

 

14 hours ago, hondo in seattle said:

The play a couple of weeks ago when Josh threw to Gabe in the endzone and was picked...  How do you grade that?  Did Josh just make a bonehead play under duress?  Or did Gabe go to the wrong spot based on the defense?   How does the PFF grader know?  Even if the grader is a former All-Pro, he can't know.  He can only guess.

 

So PFF is the only entity capable of this type of error?

 

How about QC coaches and advance scouts? Most scouting evaluations, including those done by NFL scouts are done without knowing for sure each players assignment.

 

The same goes for esteemed and highly-credible evaluators like Greg Cosell.

 

Critics of PFF seem like they don't understand the intrinsic nature of player evaluation... or maybe they just want to complain about something.

 

 

[Sierra Foothills]

8 minutes ago, JoPoy88 said:

That’s all perfectly fine - so long as it’s made clear that these things are purely subjective analyses. These are people watching tape and assigning grades on a scale that they created. 
 

My main problem with them is these two sides of their business are conflated together and now everything is shrouded under PFF’s data analytics. And they really don’t seem interested in speaking up and differentiating the two to the public. 

 

To the bolded... if that has to be explained to people they most likely won't understand the explanation.

 

[BADOLBILZ]

1 hour ago, HoofHearted said:

PFF grades are for fans. They serve no purpose to teams.

 

As they shouldn't.    Teams pay a whole lotta' people whose professional ceiling without the NFL is otherwise "gym teacher" to do their own research and offer their own opinions.

 

Every industry has this.   In my experience, the few coaches, scouts and the GM I've known..........not particularly smart people and often with very poor recollection.   But in some cases they've had great success.  

 

When you are IN an industry you have an understanding that from within all that matters is your results there.   I've experienced it too.   People who couldn't make a dime in any of the areas I've had success in often knew more about the minutia of them than I have.   That was worthless to me........but I didn't have to defend myself against it like people in the NFL do.

 

 And PFF grades are all about that minutia.   As Monson said,  they don't grade like coaches.   

 

It's also why a meathead like JJ Watt can rant about the only bad plays are plays that have ACTUAL bad results.   Avid fans often have very unrealistic expectations of these people in the NFL.   It's not life and death or rocket science.   It's entertainment.   The plans are slapped together with purpose but hastily from week to week.  The sample sizes are small. And frankly,  these aspects help make it so unpredictable and entertaining.  

 

As I said earlier..........we should all just be glad they don't play 162 games each.   Because THEN the data gets real and changes everything.

 

[hondo in seattle]

55 minutes ago, Sierra Foothills said:

...

 

Critics of PFF seem like they don't understand the intrinsic nature of player evaluation... or maybe they just want to complain about something.

 

 

 

So, wait...  You're saying your smarter than us PFF critics because we don't understand "the intrinsic nature of player evaluation" but you do?   I'm not sure why you're taking low shots at the intelligence of fellow mafioso, but I disagree.

 

Most of us don't like the notion that you can reduce a player to a set of numbers.  Let me give another example.  Maybe Diggs is playing with broken ribs.  The injury will lower his grade.  But it won't lower the way a coach evaluates him as a player.  PFF lacks context.  Their graders don't know what the position coaches have been telling the player, the playcall, the assignment, the assignment of the guy next to the graded player which may effect his decision-making, his injury status, and so on.  

 

Additionally, how PFF grades each play and how they weigh (or don't weigh) different things is ultimately subjective.   Let's talk about weighting for a moment.  Let's say the Bills call a run to the right.  Shakir is lined up on the left and completely muffs his block. So PFF gives him a -2 grade, even though it didn't impact the run.  The next play Shakir makes a tremendous one-handed grab and dances through the defense for a dazzling 63-yard TD that wins the game with just seconds left.   PFF grades him a +2, their highest grade.  Shakir's PFF average for those two plays is ZERO.  But a coach would call him the hero of the game.  Weighting matters. 

 

The idea that you can entirely remove subjectivity from football evaluation is absurd.  Maybe PFF knows this, maybe not.  I don't care.  What I know is that their attempt to reduce players to a metric is flawed.  And while many of their grades seem accurate, some clearly are not.  

 

I don't hate PFF for trying.  In fact, when the Bills acquire a FA, one of the first things I do is look at his PFF grade for a ballpark estimate of his value.  But if I wanted a better evaluation, I'd talk to a coach.  

 

Edited January 18 by hondo in seattle

[Shaw66]

Man, I freakin love this!   I'm so happy to hear and see Watt saying that.   

 

I don't remember if it was on video, but I remember seeing or reading a piece of an interview with Kyle Williams when someone asked about his or someone else's rating on PFF or some other system.  First, Kyle said, "What's that?"   He didn't even know any of these outside rating systems existed.   So, someone explained to him what it was, and Kyle said, "Are you kidding?"

 

He went to say that it was totally absurd to think that some people outside the team reviewing game film can evaluate a player's performance.   He said they don't know the play call, they don't know the adjustments that may have been made during the week, they don't know the audibles, they don't the assignment on the play call, the weekly-adjusted play call, or the audibled play.   They don't know anything about what the player is actually supposed to be doing to make the play work, and they want us to believe that their rating actually means something.   How can it possibly mean something if they don't know what the player was supposed to do in the first place?

 

Kyle was incredulous.   And that's exactly what Watt is saying here.  Good for him. 

[BullBuchanan]

16 hours ago, Royale with Cheese said:

 
Okay so you with PFF grade that Ed Oliver has been below average this year. 

They have Jordan Davis as a top 10 DT and he’s been terrible this year.

 

Ed Oliver, below average.

 

15 hours ago, JoPoy88 said:

There is nothing empirical about PFF player grades. Jesus Christ.

 

Edit: and this is the biggest problem with PFF - they’re liars (or at least, manipulators.) PFF holds themselves out as leaders in analytical football data. and they do in fact do a lot of that - that’s what they sell to teams. But PFF player grades are not that, never have been. That’s their fan facing product. That’s to drive those clicks and engagements. And they rely on fans like ol’ Bull here, who’s never met a topic he knew nothing about he wouldn’t comment on, to not know the difference. And they’re succeeding.

And here lies the problem. PFF isn't saying a player is good or bad. They are providing a grade based on what a player does on each play. It's YOU and others who are ascribing a meaning to that grade.

PFF isn't saying that Jordan Davis is a top 10 DT. They are saying that within their grading system he has the 10th highest score. It's not at all a complete measure of a football player. It's one data point that measures the impact of each and every play of a player. It's entirely possible for you to think that Jordan Davis is "terrible" because of whatever reasons, while still having a higher aggregate score using PFFs metrics than most other players. If you were forced to break down why you thought Davis was terrible this year, I'm sure you'd find an answer as to why he still has a high PFF grade

This isn't just PFF. It goes for all data from all sources. You can't just take two results from a data source and automatically say that the higher result is the better thing.

As for what I know and don't know - I know how PFF says they score players and I work with data and insights professionally, so I think that's probably more experience than most people have. It doesn't make me an expert in PFF, but the types of foaming at the mouth posts raging against PFF don't require me to be an expert.

[Matt_In_NH]

19 hours ago, Punch said:

Not PFF specific, but it seemed almost the entire analytics crowd hated Josh Allen when he was coming out of college. Now, by large, the analytics community is constantly defending him whereas most of the criticism he receives comes almost exclusively from former players and talking heads--- typically, the least informed sports commentators wrt anything but the most casual biased stereotypes.

Talking heads get paid because they have an opinion they back up emphatically and will use whatever data they have to support it while ignoring data that refutes their point.  Unfortunately that approach works and people love to respond to it and talk about the take as right or wrong.  But in doing so it encourages more  of the same.  It’s like the guy I. The bar that just shoots from the hip with a string opinion.   It’s an opinion and lacks substance but it gets a reaction and that pays.  

[Royale with Cheese]

3 minutes ago, BullBuchanan said:

 

And here lies the problem. PFF isn't saying a player is good or bad. They are providing a grade based on what a player does on each play. It's YOU and others who are ascribing a meaning to that grade.

PFF isn't saying that Jordan Davis is a top 10 DT. They are saying that within their grading system he has the 10th highest score. It's not at all a complete measure of a football player. It's one data point that measures the impact of each and every play of a player. It's entirely possible for you to think that Jordan Davis is "terrible" because of whatever reasons, while still having a higher aggregate score using PFFs metrics than most other players. If you were forced to break down why you thought Davis was terrible this year, I'm sure you'd find an answer as to why he still has a high PFF grade

This isn't just PFF. It goes for all data from all sources. You can't just take two results from a data source and automatically say that the higher result is the better thing.

As for what I know and don't know - I know how PFF says they score players and I work with data and insights professionally, so I think that's probably more experience than most people have. It doesn't make me an expert in PFF, but the types of foaming at the mouth posts raging against PFF don't require me to be an expert.

 

LOL then this entire thing doesn't make any sense then.

 

Within their grading system, he has the 10th highest score?  Is it not based on performance or something lol.

How is he scoring so high on PFF when he's doing nothing on the field?  Is no effort and walking on the field a metric they score favorably or something?

 

https://www.themirror.com/sport/american-football/philadelphia-eagles-bench-draft-pick-267699

 

"Davis in particular has come under scrutiny as former NFL offensive lineman Brian Baldinger delivered a brutal assessment of the physical condition he believes the 23-year-old is in.

“It is clear that [he] is out of shape,” the 64-year-old PHLY Eagles podcast. "I gotta believe they fine him every week. He’s overweight, and it shows. He’s not pursuing the ball, he’s nothing in the pass rush. So if he’s going to play with the effort and be overweight like that, then you think that they’re a man short in rotation.

“He’s not effective right now, they need another guy inside in their defensive tackle rotation. I’m not sure why he keeps dressing, he's ineffective. When plays are going on, he’s basically walking on the field, it’s bad effort.”

[Generic_Bills_Fan]

I think pffs analysis works pretty well for some positions/schemes/etc and it’s not as useful for others 

 

[Beck Water]

2 hours ago, BarleyNY said:

 

For ratings to be useful people have to understand what they really mean, which includes what they’re actually measuring as well as their limitations. I like a lot of what PFF does, but their grading has limited utility. It’s better with some positions than others, but is still limited to what players are asked to do rather than what they’re capable of. For instance, a QB like Cousins who plays a much more limited role in his team’s offense, can more easily score a higher grade than a QB like Allen, who much more is asked of.

 

Sam Monson recently mentioned this on a podcast when PFF’s grading came up. He used Dodson as an example of a player who graded very highly due to him doing well in a very limited role. It makes sense that coaches would be more interested in a player’s overall impact on a game and would factor in things like the difficulty of the assignment, the ability of the player, the contract of the player, etc. Each has their use and a value, but they aren’t going to be the same. 

 

I could be wrong here and again, hope I will be corrected but I think actual coaches grading game film are not considering things like the contract of the player, the difficulty of the assignment, and the ability of the player when grading.  Those are things which are taken into account when "valuing the positions" and "ranking the players against positional value" which Beane has talked about the FO and coaches doing after the season.

 

I redirect to what @Buffalo716 said upthread since I know he knows something about it:

https://www.twobillsdrive.com/community/topic/252639-once-and-for-allpffffffffft/page/3/#comment-8885301

 

The coaches grade on assignment, technique, and effort.  Did you "bring it" on that play?  Did you use good technique?  And did you "do your job"?  If a player (like AJ Klein filling in for Milano in 2020) is giving maximal effort and using sound technique, but simply can't complete his assignment because he's being asked to do things beyond his physical abilities, then the coaches need to see that and adjust his assignment to something within his physical capabilities while he's on the field, but it's not taken into account in the grade.

[BullBuchanan]

23 minutes ago, Royale with Cheese said:

 

LOL then this entire thing doesn't make any sense then.

 

Within their grading system, he has the 10th highest score?  Is it not based on performance or something lol.

How is he scoring so high on PFF when he's doing nothing on the field?  Is no effort and walking on the field a metric they score favorably or something?

 

https://www.themirror.com/sport/american-football/philadelphia-eagles-bench-draft-pick-267699

 

"Davis in particular has come under scrutiny as former NFL offensive lineman Brian Baldinger delivered a brutal assessment of the physical condition he believes the 23-year-old is in.

“It is clear that [he] is out of shape,” the 64-year-old PHLY Eagles podcast. "I gotta believe they fine him every week. He’s overweight, and it shows. He’s not pursuing the ball, he’s nothing in the pass rush. So if he’s going to play with the effort and be overweight like that, then you think that they’re a man short in rotation.

“He’s not effective right now, they need another guy inside in their defensive tackle rotation. I’m not sure why he keeps dressing, he's ineffective. When plays are going on, he’s basically walking on the field, it’s bad effort.”

I already explained this to you, you just didn't want to hear it. It's clear you'd rather just make something up to believe.

[Royale with Cheese]

2 minutes ago, BullBuchanan said:

I already explained this to you, you just didn't want to hear it. It's clear you'd rather just make something up to believe.

 

Just because you explained it, doesn't mean it made sense.

 

This is straight from PFF.  Their "metrics" states that Jordan Davis performed very well in his role.  

And it grades Ed Oliver as average.

Explain this.

 

"PLAYER RANKINGS

The grades allow for easy player comparisons, whether using an overall grade or a facet grade. While we believe the grade is an excellent baseline for how well a player performed his given role, we also believe the using the entire context is crucial when evaluating players. Perhaps a slot receiver had to play more than expected on the outside or a nose tackle was forced to play more three-technique than his coaching staff initially desired. Both players are being evaluated based on what they did, and that context is important when using the grades. PFF+ allows users to sort by player grades, but they can also see the simple and advanced stats that tell the story for each player."

 

https://www.pff.com/grades

[BullBuchanan]

1 hour ago, hondo in seattle said:

 

So, wait...  You're saying your smarter than us PFF critics because we don't understand "the intrinsic nature of player evaluation" but you do?   I'm not sure why you're taking low shots at the intelligence of fellow mafioso, but I disagree.

 

Most of us don't like the notion that you can reduce a player to a set of numbers.  Let me give another example.  Maybe Diggs is playing with broken ribs.  The injury will lower his grade.  But it won't lower the way a coach evaluates him as a player.  PFF lacks context.  Their graders don't know what the position coaches have been telling the player, the playcall, the assignment, the assignment of the guy next to the graded player which may effect his decision-making, his injury status, and so on.  

 

Additionally, how PFF grades each play and how they weigh (or don't weigh) different things is ultimately subjective.   Let's talk about weighting for a moment.  Let's say the Bills call a run to the right.  Shakir is lined up on the left and completely muffs his block. So PFF gives him a -2 grade, even though it didn't impact the run.  The next play Shakir makes a tremendous one-handed grab and dances through the defense for a dazzling 63-yard TD that wins the game with just seconds left.   PFF grades him a +2, their highest grade.  Shakir's PFF average for those two plays is ZERO.  But a coach would call him the hero of the game.  Weighting matters. 

 

The idea that you can entirely remove subjectivity from football evaluation is absurd.  Maybe PFF knows this, maybe not.  I don't care.  What I know is that their attempt to reduce players to a metric is flawed.  And while many of their grades seem accurate, some clearly are not.  

 

I don't hate PFF for trying.  In fact, when the Bills acquire a FA, one of the first things I do is look at his PFF grade for a ballpark estimate of his value.  But if I wanted a better evaluation, I'd talk to a coach.  

 

PFF accounts for weighting, and they would never give Shakir a -2 for just a missed block. If you look at their rubric, those numbers are reserved for explosive big plays. That said, you're absolutely right that it doesn't account for a player that's a big outlier. Let's say you have a QB that throws 2 passes in the dirt or out of the stadium and then throws a hail mary TD. He proceeds to do this every drive of his career. He'd be the all-time greatest QB ever, but he'd likely have a worse score than a guy that dinks and dunks it down the field and only occasionally scores. Now of course in reality things aren't that extreme, so if you have a guy that plays mostly solid, but has huge screwups you probably end up with a higher score than you'd think or vice-versa.

That doesn't make the system or analysis flawed. Every system has limitations. The flaw is using it to ascribe value that was never intended. You need to be real with what the data tells you and what it doesn't. It should be one piece of the puzzle. More often than not data should make you ask questions, not tell you answers. It's a very common mistake. Even c-level executives do it in fortune 500 companies - even when we tell them not to.

3 minutes ago, Royale with Cheese said:

 

Just because you explained it, doesn't mean it made sense.

 

This is straight from PFF.  Their "metrics" states that Jordan Davis performed very well in his role.  

And it grades Ed Oliver as average.

Explain this.

 

"PLAYER RANKINGS

The grades allow for easy player comparisons, whether using an overall grade or a facet grade. While we believe the grade is an excellent baseline for how well a player performed his given role, we also believe the using the entire context is crucial when evaluating players. Perhaps a slot receiver had to play more than expected on the outside or a nose tackle was forced to play more three-technique than his coaching staff initially desired. Both players are being evaluated based on what they did, and that context is important when using the grades. PFF+ allows users to sort by player grades, but they can also see the simple and advanced stats that tell the story for each player."

 

https://www.pff.com/grades

Man, all you had to do was read one more sentence.

Edited January 18 by BullBuchanan

[Beck Water]

19 hours ago, Einstein said:

 

You really think NFL teams would pay millions of dollars for useless data? (.....)

The other excuse is "teams pay for the data, not the grades". How do you think PFF grades the player? On the data!

 

You earned *respect* for your Good Samaritan action of sending a snowplow to the drive of a brother fan with a snowblower.

 

But I know it's been explained to you over and over that the data PFF pays for is not their grades.  It would be nice if you would acknowledge this.

 

In particular, in the first link, there are a LOT of data in those fields - of which player grades are only two fields.  Ultimately useful player grades are based on 1) knowledge of the play call and the player's assignment in the called play 2) perception of how they executed their assignment, their technique and effort.  The first is not data PFF has.  The second is not objective data per se though coaches who know their players and scouts who watch a ton of film can be very consistent in this assessment.

 

When former players such as Kurt Warner, JT O'Sullivan etc are putting together film they frequently caveat their critique "I don't know what the reads were in this play, for me it would be....".  That's because even these guys who have forgotten more football than the rest of us will know, are aware that they don't understand the details of the play.  And sometimes they're caught out by that (Warner calling out Reggie Gilliam as the first read on the throw where Knox got mugged)

 

 

Edited January 18 by Beck Water

[Einstein]

Just now, Beck Water said:

But I know it's been explained to you over and over that the data PFF pays for is not their grades.  It would be nice if you would acknowledge this.
 

I did address this. Here is the link: https://www.twobillsdrive.com/community/topic/252639-once-and-for-allpffffffffft/?do=findComment&comment=8886231

 

 

 

[Royale with Cheese]

1 minute ago, BullBuchanan said:

Man, all you had to do was read one more sentence.

 

When it's that big of a discrepancy, its legitimacy can be questioned.  

It's great that you defend PFF player grades but have no idea how they grade them.  

 

So when you have something as bizarre as Ed Oliver being graded as average, I think actually rated him below average and Jordan Davis performing well....I think it's pure BS.

 

In your opinion, outside of PFF, has Ed Oliver been below average this year in his play?

Is Brian Baldinger completely off his rocker with his assessment of Jordan Davis's play?

[hondo in seattle]

25 minutes ago, BullBuchanan said:

That doesn't make the system or analysis flawed. Every system has limitations. The flaw is using it to ascribe value that was never intended. You need to be real with what the data tells you and what it doesn't. It should be one piece of the puzzle. More often than not data should make you ask questions, not tell you answers. It's a very common mistake. Even c-level executives do it in fortune 500 companies - even when we tell them not to.

 

 

I do think analytics has a place in player evaluations.  And I believe it'll get better with time. 

 

But the bolded remark is a great observation.  

[BullBuchanan]

33 minutes ago, Royale with Cheese said:

 

When it's that big of a discrepancy, its legitimacy can be questioned.  

It's great that you defend PFF player grades but have no idea how they grade them.  

 

So when you have something as bizarre as Ed Oliver being graded as average, I think actually rated him below average and Jordan Davis performing well....I think it's pure BS.

 

In your opinion, outside of PFF, has Ed Oliver been below average this year in his play?

Is Brian Baldinger completely off his rocker with his assessment of Jordan Davis's play?

I don't have a professional assessment of Ed Oliver. I know he's made some great plays, but I haven't broken down his tape and looked every one of his plays to build an aggregate score. While he has made great plays, i do know he hasn't looked like prime Aaron Donald or JJ Watt. Not that that means anything.

As for Brian Baldinger's assessment. It's one guy with a super subjective opinion. Not sure if he's completely off in his assessment, because he doesn't really say that much that takes away from PFFs assesment. He can be a limited factor in pass rush due to his weight  and still perform well on his assignments without performing to potential or desired impact.

[Royale with Cheese]

16 minutes ago, BullBuchanan said:

I don't have a professional assessment of Ed Oliver. I know he's made some great plays, but I haven't broken down his tape and looked every one of his plays to build an aggregate score. While he has made great plays, i do know he hasn't looked like prime Aaron Donald or JJ Watt. Not that that means anything.

As for Brian Baldinger's assessment. It's one guy with a super subjective opinion. Not sure if he's completely off in his assessment, because he doesn't really say that much that takes away from PFFs assesment. He can be a limited factor in pass rush due to his weight  and still perform well on his assignments without performing to potential or desired impact.

 

Baldinger says he shouldn't be dressing.  So yes, it's completely the exact opposite of what PFF's assessment is.  Baldinger is saying he's useless, PFF says he's doing well.

[BullBuchanan]

Just now, Royale with Cheese said:

 

Baldinger says he shouldn't be dressing.  So yes, it's completely the exact opposite of what PFF's assessment is.  Baldinger is saying he's useless, PFF says he's doing well.

If you want to find a talking head saying a thing about any player, you can absolutely find it.

[Beck Water]

19 hours ago, Einstein said:

PFF employs former NFL players, especially at Tier 2 and 3 of grading. The reason why PFF grades take a couple day to update after a game is because they are not graded by one player. It gets graded by a Tier 1 analyst, then a Tier 2 analyst, and then sometimes a Tier 3 analyst. 

People will often write "they don't know the play call". These people clearly think that football is rocket science. It's not. We can see on All-22 what the play is. Especially by mid season when teams are running the same plays over and over. And no, we do not accept the idea that maybe this particular team told their left tackle to get pancaked on a counter-run because the coach told him to. No, he just got beat.

 

I already addressed your first and last sentence, but I'd like to address these points.  Again, you deserve a Shout! for your good Samaritan action taking care of a guy's snow removal crisis.  You still need to be responsive of people's points.

 

Let's delve into PFF graders and do some simple math.  PFF states they employ 600 full and part time analysts but less than 10% are trained to the level where they grade plays and 2-3% percent to the level of reviewing and finalizing grades.  OK, so that means they have less than 60 people grading plays and 12-18 people reviewing and finalizing them.  We'll use 60 and 12-18 in our assessment.

 

In a typical game there are average 63 offensive and 63 defensive plays.  That's 126 plays per game for each team, or 4032 plays per week.  Each of those plays involves 11 players PFF gonna grade, so 44352 player plays per week to grade.

 

If less than 10% of their 600 analysts grade plays, that means  each analyst is grading at least 739 plays over a span of a couple days.  For the data to be useful for teams, they would need the grades at latest Tuesday after the game - Monday for their own players, since they review film and do corrections on Monday; Tuesday for their next opponent since that's when the coaches create the game plan for the next week, which they install with the team Wednesday.

 

So less than 60 analysts would be reviewing 44352 plays in 24 hrs so that 12-18 people can review and finalize them.  Let's say they work a 12 hr shift (brutal, but people do it).  That would say they're reviewing 61.6 plays per hour.  They say "Each grade is reviewed at least once, and usually multiple times, using every camera angle available, including All-22 coaches’ tape."  2 All-22 views, usually 3 or 4 broadcast camera views, so what - 10 seconds per view?

 

Then there are those senior analysts.  Let's say they only review 25% of the plays or 11088 plays, as the other 75% are considered straightforward.  Again, to be useful to the team, these data are needed by Tuesday am at latest, so let's say we have 12 or 18 people reviewing those plays in another 12 hrs, that's 51 - 77 plays per hour.

 

Let's contrast this with the NFL, where, let's say on offense, the Bills average 65.5 plays per game on offense.  The OL coach has an assistant coach (and possibly an intern or another offensive assistant) and 5 players in on every play, so and his assistant have 328 plays to review and grade.  That's half the number of plays, for twice the number of people per play, by people who know the play call and the assignment to start with.  And that's the busiest room on offense.  The WR coaches are grading 180 plays per game since 2 or 3 WR sets.  The RB coach is grading an average of 71 plays per game since the Bills do very few 2 back sets.  etc.

 

Just on sheer arithmetic, it doesn't make sense to me that teams would rely on PFF to grade their players.

 

Now: to your statement "we can all see on All-22 what the play is".  I would like to remind you of a discussion that took place earlier this season between yourself, @Buffalo716, and @HoofHearted regarding what the defensive coverage was on a specific play thus what the assignment responsibilities were.  You're obviously someone who knows some ball, but does the phrase "I run the coverage you’re referring to and it’s not played the way you think it’s played. If you’d like we can go to PMs and I can teach it to you" ring a bell?  Yes, football isn't rocket science, but there are a lot of subtleties, and those subtleties impact what each player is actually supposed to be doing.  You even ran into that here about 2 months ago, so I'm not sure how (if you're intellectually honest in discussion) it's so challenging for you to acknowledge this.

The point is even guys who know ball and understand coverages can have to look at a play a couple times to understand what the coverage and thus the roles and responsibilities were supposed to be before zeroing in on how well each filled their assignment.

[Royale with Cheese]

4 minutes ago, BullBuchanan said:

If you want to find a talking head saying a thing about any player, you can absolutely find it.

 

"A thing".  Yes of course, you can find several million of them.

 

But to say someone is so bad they shouldn't be dressing?  We can find that?

Let me see if I can find a talking head saying that Aaron Donald is so ineffective that he shouldn't be dressing.  You think I can find it?

[BullBuchanan]

5 minutes ago, Royale with Cheese said:

 

"A thing".  Yes of course, you can find several million of them.

 

But to say someone is so bad they shouldn't be dressing?  We can find that?

Let me see if I can find a talking head saying that Aaron Donald is so ineffective that he shouldn't be dressing.  You think I can find it?

Probably yes. By the way, Davis had 2.5 sacks while only playing 45% of snaps. based on that alone, I'm guessing there's more than a tad of hyperbole coming from your boy.

Edited January 18 by BullBuchanan

[Royale with Cheese]

5 minutes ago, BullBuchanan said:

Probably yes. By the way, Davis had 2.5 sacks while only playing 45% of snaps. based on that alone, I'm guessing there's more than a tad of hyperbole coming from your boy.

Your evidence that he wasn't bad because he had 2.5 sacks?

 

Ed Oliver played 50% of snaps and had 9.5 sacks.

 

Let me guess....PFF is more accurate with Davis than they were with Oliver.  

 

You know Oliver played well this year but you won't come out and say it because of the stupid statement below.  Sticking to your guns.

 

On 6/4/2023 at 1:51 AM, BullBuchanan said:

it was 6.5, but that doesn't really matter. What matters is it's A LOT less money and a lot more production. Oliver has never even lived up to his rookie contract, and then they give him an top level extension. Total clown move from Beane. It's starting to look like they're already writing off this season and planning for 2024 and beyond. We're going to be fielding a team this year that has equal or lesser talent at nearly every position. Unless you thought we were a guard and a 220lb mlb away from a championship, of course.

 

 

[BullBuchanan]

2 minutes ago, Royale with Cheese said:

Your evidence that he wasn't bad because he had 2.5 sacks?

 

Ed Oliver played 50% of snaps and had 9.5 sacks.

 

Let me guess....PFF is more accurate with Davis than they were with Oliver.  

 

You know Oliver played well this year but you won't come out and say it because of the stupid statement below.  Sticking to your guns.

 

 

nope.

[Royale with Cheese]

3 minutes ago, BullBuchanan said:

nope.

I guess we will disagree about Oliver.

I think it was a great extension, you still think Beane is a clown for extending him.

[Scott7975]

1 hour ago, Einstein said:

I did address this. Here is the link: https://www.twobillsdrive.com/community/topic/252639-once-and-for-allpffffffffft/?do=findComment&comment=8886231

 

 

 

 

 

Thats not addressing it.  To me it just looks like you are doubling down on saying teams are paying for player grading.  This is like saying you paid for a 12 course fine dining meal for the Andes mint you get with the check.