PFF Ranks for Bills

[D. L. Hot-Flamethrower]

.4*Great+.6*Poor=Poor

 

Got it!

Edited September 10, 2019 by D. L. Hot-Flamethrower

[GunnerBill]

Just now, D. L. Hot-Flamethrower said:

.6*Great+.4*Poor=Poor

 

Got it!

 

It was the other way around. It was 4 rush plays for which he got a great grade. 6 receiving plays for which he got a poor grade. 

[Alphadawg7]

11 minutes ago, GunnerBill said:

 

It is not worthless and there is nuance. Zay did not get a good receiving grade. His run blocking dragged his grade up (he got an elite run blocking grade from 13 run block snaps). 

 

Singletary caught 5 of his 6 targets but didn't break many tackle on those plays, bobbled two of the ones he caught and dropped a ball. 

 

Again, you have to dig deeper. 

 

 

Sorry bud, there is no digging deeper to justify Singletary getting a poor grade.  Bobbling but still catching the ball shouldn’t take a game with 9 touches and 98 yards down to poor.  

 

7 minutes ago, GunnerBill said:

 

It was the other way around. It was 4 rush plays for which he got a great grade. 6 receiving plays for which he got a poor grade. 

 

6 receiving plays where he caught 5 does not equate to poor, let alone so poor that it brings a guy who averaged 17.5 yards per carry all the way down to a poor overall grade.

 

This is just not justifiable on any level.

Edited September 10, 2019 by Alphadawg7

[GunnerBill]

1 minute ago, Alphadawg7 said:

 

Sorry bud, there is no digging deeper to justify Singletary getting a poor grade.  Bobbling but still catching the ball shouldn’t take a game with 9 touches and 98 yards down to poor.  

 

You take it as too much of a definitive judgment on players. They are not saying Singletary was rubbish. That isn't the aim of their grading. They are looking at how much a player maximised the opportunity for them on each play. Bobbling a ball before you catch it means you likely haven't done that and so it attracts a negative grade. 

 

It is a tool. It has to be taken that way. 

 

[Alphadawg7]

34 minutes ago, GunnerBill said:

 

You take it as too much of a definitive judgment on players. They are not saying Singletary was rubbish. That isn't the aim of their grading. They are looking at how much a player maximised the opportunity for them on each play. Bobbling a ball before you catch it means you likely haven't done that and so it attracts a negative grade. 

 

It is a tool. It has to be taken that way. 

 

 

I understand the tool.  I understand they are not saying he is trash as a player.  I am not in any way confused.

 

But bobbling but STILL catching a pass is not so poor that it offsets the fact he did catch the passes and also produced 70 yards rushing on 4 touches. 

 

No one is arguing that bobbled, but caught, passes could lower his receiving grade a LITTLE.  But to go all the way down to poor is absurd.  To then furthermore lower his overall grade to poor after rushing for 70 yards on 4 carries as a result is about the most ridiculous grading system and just doesn't provide any real analytical value.

 

Sorry Gunner, no disrespect intended from me on this, but IMO there is absolutely no logical defense that can justify Singletary receiving a poor grade overall.  Honestly, there isn’t even a good defense for him to get a poor receiving grade when he still caught the 2 bobbled passes that we’re barely bobbled to begin with.

Edited September 10, 2019 by Alphadawg7

[GunnerBill]

1 hour ago, Alphadawg7 said:

 

I understand the tool.  I understand they are not saying he is trash as a player.  I am not in any way confused.

 

But bobbling but STILL catching a pass is not so poor that it offsets the fact he did catch the passes and also produced 70 yards rushing on 4 touches. 

 

No one is arguing that bobbled, but caught, passes could lower his receiving grade a LITTLE.  But to go all the way down to poor is absurd.  To then furthermore lower his overall grade to poor after rushing for 70 yards on 4 carries as a result is about the most ridiculous grading system and just doesn't provide any real analytical value.

 

Sorry Gunner, no disrespect intended from me on this, but IMO there is absolutely no logical defense that can justify Singletary receiving a poor grade overall.  Honestly, there isn’t even a good defense for him to get a poor receiving grade when he still caught the 2 bobbled passes that we’re barely bobbled to begin with.

 

Then we are firmly in agree to disagree territory. I understand their grade and how they got there. Doesn't mean I don't think Singletary was impressive or that he was critical to the comeback. 

[D521646]

1 hour ago, GunnerBill said:

 

You take it as too much of a definitive judgment on players. They are not saying Singletary was rubbish. That isn't the aim of their grading. They are looking at how much a player maximised the opportunity for them on each play. Bobbling a ball before you catch it means you likely haven't done that and so it attracts a negative grade. 

 

It is a tool. It has to be taken that way. 

 

 

Tool, being the appropriate word here.  What "tool" is this good for?  Fantasy FB, defensive/offensive coordinators, coaches of any kind?  What is it good for?  Is it good for predicting ANYTHING, and if so, what is the measuring stick, what is it's success rate?

 

No, Sir, this is no tool.  The screwdriver I have in my toolbox is a tool, this is pure garbage for fans who don't know anything.

 

Tim-

[DCOrange]

12 hours ago, buffalo2218 said:

4 carries for 70 yards is poor? wow

 

He was graded the 4th highest rated runner this week. The reason his ranking is so low is because the vast majority of his snaps were in the passing game where he had nearly the lowest grade in the league.

 

Edit: I see Gunner has been going back and forth with posters about this, so I'll add this: it's not just the 5 catches on 6 targets that impact Singletary's receiving grade. It's the plays where he wasn't targeted and the context around all of that. I have no idea what they saw; I didn't even watch the game, but I would guess they felt his routes were poor/sounds like he bobbled and or dropped a few passes.

Edited September 10, 2019 by DCOrange

[Alphadawg7]

2 minutes ago, GunnerBill said:

 

Then we are firmly in agree to disagree territory. I understand their grade and how they got there. Doesn't mean I don't think Singletary was impressive or that he was critical to the comeback. 

 

All good Gunner, you know I respect you overall as a poster.  

 

But yeah, agree to disagree here as there is just no logical reasoning I can come too that will weigh slightly bobbling a couple of made catches so hard it drops his overall performance to “poor”.  I just can’t in any way support this kind of a grading scale.  I’m fine dinging him some, but to have his overall grade be this low over it is just not balanced analytics IMO.

[BigDingus]

13 hours ago, Seoulofstone said:

 

  I was just reading a thread on jetnation saying that pff rated Darnold QB#11 @ 68,7 and Allen QB#25 @ 52,3. So that's amusing.

 

Darnold was the better QB in that game.

If Allen played like Darnold did, we would've been in cruise control by the end of the game. Simply not turning the ball over would've done wonders.

Our defense + the Jets not converting on turnovers + Mosley getting injured was what won us the game. Allen finally playing like a decent starter at the end of the game, doesn't mean he had anywhere close to a good performance.

 

Edit: Forgot the part where the Jets' kicker completely screwed them over... If he could hit a FG, we still lose.

Edited September 10, 2019 by BigDingus

[GunnerBill]

Just now, D521646 said:

 

Tool, being the appropriate word here.  What "tool" is this good for?  Fantasy FB, defensive/offensive coordinators, coaches of any kind?  What is it good for?  Is it good for predicting ANYTHING, and if so, what is the measuring stick, what is it's success rate?

 

No, Sir, this is no tool.  The screwdriver I have in my toolbox is a tool, this is pure garbage for fans who don't know anything.

 

Tim-

 

Well most NFL teans are subscribers so they cleary think it is a useful tool. It isn't intended as a mechanism to predict anything. It is an attempt to apply numerical value to every play in football. Now football isn't a game that lends itself to that very easily, but that doesn't mean one should dismiss any analysis that attempts to do such. I don't agree with PFF grades as any sort of definitve barrometer for a player's performance but the approach of applying a numerical value to every player's role in each play is an interesting one and can identify patterns that are useful in evaluation. 

 

The fans who hate it generally hate the fact that it is imperfect. As far as I am aware it has never claimed to be perfect. It acknowledges the imperfect nature of applying numbers to football. But to dismiss it as worthless is short sighted. And often it is just because people don't like specific outcomes. 

4 minutes ago, Alphadawg7 said:

 I’m fine dinging him some, but to have his overall grade be this low over it is just not balanced analytics IMO.

 

Analytics is not supposed to be balanced. That is why using them as a single determinate measure almost always fails. You need the human brain and the human eye to add the nuance and the context. Numbers are a blunt and cruel mistress. That is kind of the point of them. 

[BigDingus]

2 hours ago, GunnerBill said:

 

It is not worthless and there is nuance. Zay did not get a good receiving grade. His run blocking dragged his grade up (he got an elite run blocking grade from 13 run block snaps). 

 

Singletary caught 5 of his 6 targets but didn't break many tackle on those plays, bobbled two of the ones he caught and dropped a ball. 

 

Again, you have to dig deeper. 

 

 

Expecting these guys to dig deeper beyond their pure emotional reactions is asking too much.

PFF isn't stupid, and they're also not gods. They're very informative, and do very in depth analysis of the overall performance, beyond the stat lines, and grade full performances. Run blocking is a key aspect of being a WR, as is getting open & running good routes. They don't grade every single thing ever, but they give good insight into more than just simple numbers.

[Seoulofstone]

1 minute ago, BigDingus said:

 

Darnold was the better QB in that game.

If Allen played like Darnold did, we would've been in cruise control by the end of the game. Simply not turning the ball over would've done wonders.

Our defense + the Jets not converting on turnovers + Mosley getting injured was what won us the game. Allen finally playing like a decent starter at the end of the game, doesn't mean he had anywhere close to a good performance.

 

You're absolutely entitled to that opinion. Darnold played safe the whole game hence his throw yardage average. He never really hurt the Bills but for box score afficionados his statline looks acceptable  I believe the Bills tried that approach with Tyrod. I for one am glad the coaches are saying to JA try your arm. He led a 4th quarter comeback. He had a positive impact-but you see what you want to see.     

 

   Another parallel is Eli. I went to the Giants forum ahead of the upcoming game. His statline is excellent but reading fan comments, his insipid short passing game is hated by their fans. They are already begging for Jones who was a deeply unpopular draft selection.

[Seoulofstone]

18 minutes ago, BigDingus said:

Our defense + the Jets not converting on turnovers + Mosley getting injured was what won us the game.

 

Edit: Forgot the part where the Jets' kicker completely screwed them over... If he could hit a FG, we still lose.

 

  It's so funny to me that so many Bills fans are always so desperate to diagnose the cause of wins based on things that could have gone differently or errors and Injuries to opponents. if you did this in everyday life you would go crazy  This exact same thing happened when Buffalo last made the playoffs. You are so desperate to talk about alternate universes that you can't even credit your QB for coming back from 13 points behind in the last quarter 

[GreggTX]

2 surprises -- Singletary and Sweeney. I have to guess that Singletary was making mistakes on plays where he didn't have the ball in his hands. Well, I know how to fix that problem. Put the ball IN his hands more often. Can Sweeney really be this great? I can't wait to see more of him.

[Mr. WEO]

Sweeney IS Elite!

[Solomon Grundy]

My concern is that they had 1/2 the offensive line rated as POOR

Edited September 10, 2019 by the skycap

[RememberTheRockpile]

52 minutes ago, GunnerBill said:

 

Well most NFL teans are subscribers so they cleary think it is a useful tool. It isn't intended as a mechanism to predict anything. It is an attempt to apply numerical value to every play in football.

 

Most teams probably subscribe to Sports Illustrated too. PFF provides a variety of statistical packages of which some may prove useful to an NFL team.

https://www.pff.com/news/pro-pff-signature-statistics-a-glossary 

 

The fact that some or most NFL teams subscribe doesn't prove that PFF performance evaluations under discussion here are of any use to NFL teams.

 

52 minutes ago, GunnerBill said:

Now football isn't a game that lends itself to that very easily, but that doesn't mean one should dismiss any analysis that attempts to do such. I don't agree with PFF grades as any sort of definitve barrometer for a player's performance but the approach of applying a numerical value to every player's role in each play is an interesting one and can identify patterns that are useful in evaluation. 

 

I don't "dismiss any analysis" but I do look for indications that the so called analysis is BS masquerading as legitimate statistical analysis. Statistics are not exact and always include a margin of error. If a hundred people evaluate the same player on a given play there are going to undoubtedly going to be a range of differences. Failure to include at least the margin of error is a sure sign the statistic is BS.  

 

Here is my evaluation of Singletary's 6 pass targets:

 

2nd Qtr - 14:21
Flat out drops it. Started looking upfield before catching the ball.

 

2nd Qtr - 6:07
Poor pass from Allen. Pass at his feet. Not a catchable ball.

 

2nd Qtr - Short pass hits him up in his right shoulder pad /facemask. Bounces up and he snatches out of the air without breaking stride. A Jet at the 20 has a good angle but Singletary makes a nice cut at the 21 and gets to the 18. Without the cut he is likely down at the 20. Pass needed to be a foot more in front of him. In my view he did a nice job catching a poorly place pass that in my book is a net positive.

 

4th Qtr - 11:54
Another short pass coming out of the backfield. Catches it cleanly and gets bracketed by 2 Jets. Pretty much the definition of average play.

 

4th Qtr - 5:55
Short pass coming out of the backfield with a clean catch. Makes the 1st guy miss and the second Jet gets enough of him to force him out of bounds but he still gets an 2 extra yards pass the push before going out of bounds.

 

4th Qtr - 3:52
Coming out of the backfield Allen is late throwing the ball giving the Jet line backer time to close. He makes a clean catch and tackled immediately for a 1 yard loss.

 

 

[GunnerBill]

47 minutes ago, RememberTheRockpile said:

 

Most teams probably subscribe to Sports Illustrated too. PFF provides a variety of statistical packages of which some may prove useful to an NFL team.

https://www.pff.com/news/pro-pff-signature-statistics-a-glossary 

 

The fact that some or most NFL teams subscribe doesn't prove that PFF performance evaluations under discussion here are of any use to NFL teams.

 

 

I don't "dismiss any analysis" but I do look for indications that the so called analysis is BS masquerading as legitimate statistical analysis. Statistics are not exact and always include a margin of error. If a hundred people evaluate the same player on a given play there are going to undoubtedly going to be a range of differences. Failure to include at least the margin of error is a sure sign the statistic is BS.  

 

Here is my evaluation of Singletary's 6 pass targets:

 

2nd Qtr - 14:21
Flat out drops it. Started looking upfield before catching the ball.

 

2nd Qtr - 6:07
Poor pass from Allen. Pass at his feet. Not a catchable ball.

 

2nd Qtr - Short pass hits him up in his right shoulder pad /facemask. Bounces up and he snatches out of the air without breaking stride. A Jet at the 20 has a good angle but Singletary makes a nice cut at the 21 and gets to the 18. Without the cut he is likely down at the 20. Pass needed to be a foot more in front of him. In my view he did a nice job catching a poorly place pass that in my book is a net positive.

 

4th Qtr - 11:54
Another short pass coming out of the backfield. Catches it cleanly and gets bracketed by 2 Jets. Pretty much the definition of average play.

 

4th Qtr - 5:55
Short pass coming out of the backfield with a clean catch. Makes the 1st guy miss and the second Jet gets enough of him to force him out of bounds but he still gets an 2 extra yards pass the push before going out of bounds.

 

4th Qtr - 3:52
Coming out of the backfield Allen is late throwing the ball giving the Jet line backer time to close. He makes a clean catch and tackled immediately for a 1 yard loss.

 

 

 

PFF is allowing a margin of error though because it is not pretending to be definitive. It is how people read and analyse their work that is problem 

[RememberTheRockpile]

1 minute ago, GunnerBill said:

 

PFF is allowing a margin of error though because it is not pretending to be definitive. It is how people read and analyse their work that is problem 

Margin of error is numerical. So where is the margin of error for Singletary? 

[GunnerBill]

7 minutes ago, RememberTheRockpile said:

Margin of error is numerical. So where is the margin of error for Singletary? 

 

My point is you only need it if you are claiming a definitive verdict. They are not. People are still misunderstanding a bit what PFF do. 

[nedboy7]

Some good old PFF bashing. Seems like good info. 

[Simstim]

My issue with PFF is more to do with their podcasts.

 

With the Bills, they look at the negatives far harder than the positives. They seem to just not take into account all the game nuances - especially with Allen - and stick to their initial ideas about the players, like they are stuck in a narrative loop. It feels like an agenda albeit that I don't think it is. I think they just don't believe in the players or the team.

 

The problem is resolved if we keep winning, in all likelihood, but it is more than mildly annoying.

[Koko78]

11 hours ago, Buffalo Barbarian said:

Patriots ??

 

Guess again.

[ColoradoBills]

2 hours ago, GunnerBill said:

 

Well most NFL teans are subscribers so they cleary think it is a useful tool. It isn't intended as a mechanism to predict anything. It is an attempt to apply numerical value to every play in football. Now football isn't a game that lends itself to that very easily, but that doesn't mean one should dismiss any analysis that attempts to do such. I don't agree with PFF grades as any sort of definitve barrometer for a player's performance but the approach of applying a numerical value to every player's role in each play is an interesting one and can identify patterns that are useful in evaluation. 

 

The fans who hate it generally hate the fact that it is imperfect. As far as I am aware it has never claimed to be perfect. It acknowledges the imperfect nature of applying numbers to football. But to dismiss it as worthless is short sighted. And often it is just because people don't like specific outcomes. 

 

Analytics is not supposed to be balanced. That is why using them as a single determinate measure almost always fails. You need the human brain and the human eye to add the nuance and the context. Numbers are a blunt and cruel mistress. That is kind of the point of them. 

 

Gunner, first I am not disputing that analytics is a useful tool but I do have a question for you.

I did some simple math, 22 players on the field for every snap with at least 140 plays a game and a total of 16 games played a week.

That comes out to 50,000 individual evaluations per week.

 

The individual evaluating each play would need to determine what the call was and watch each player (maybe rewinding multiple times) to

determine the +/- result.  It just seems like too much information to digest to come up with these scores this fast.

 

Do you have any info on how this is done?  Are there multiple people inputting evaluations for each game?  If so, is the average used?

Is an individual judged as to who he is up against?  (example:  A OT going against Mack on a individual play would have to be scored differently than

if he was going against a rotational scrub).  I could go on but I think you get my point.

 

I have used analytics in my job for years and it's how the data is gathered and examined that means everything.

Thanks.

[GunnerBill]

1 minute ago, ColoradoBills said:

 

Gunner, first I am not disputing that analytics is a useful tool but I do have a question for you.

I did some simple math, 22 players on the field for every snap with at least 140 plays a game and a total of 16 games played a week.

That comes out to 50,000 individual evaluations per week.

 

The individual evaluating each play would need to determine what the call was and watch each player (maybe rewinding multiple times) to

determine the +/- result.  It just seems like too much information to digest to come up with these scores this fast.

 

Do you have any info on how this is done?  Are there multiple people inputting evaluations for each game?  If so, is the average used?

Is an individual judged as to who he is up against?  (example:  A OT going against Mack on a individual play would have to be scored differently than

if he was going against a rotational scrub).  I could go on but I think you get my point.

 

I have used analytics in my job for years and it's how the data is gathered and examined that means everything.

Thanks.

 

Each game is analysed by a lead analyst and then quality control checked before initial grades are applied and then re-watched and re-graded again when the all22 is available is my understanding.

 

Individuals are not judged on who they were up against or on how critical a play is in the game. They are some of the reasons I say, repeatedly, it is imperfect as a basis for any definitive determinations. You have to use it appropriately. It is a tool in the toolbox when properly applied. 

 

I have been a critic of plenty of its individual outcomes over the years. Its grading system led to far too much credit being given to Tyrod Taylor for example and last season Tre White ranked relatively average because it is hard to get the elite coverage grades when nobody throws your way any more. That is where nuance and the need for an experienced eye come into play in my mind. But it doesn't invalidate everything PFF do.

[RememberTheRockpile]

27 minutes ago, GunnerBill said:

 

My point is you only need it if you are claiming a definitive verdict. They are not. People are still misunderstanding a bit what PFF do. 

Your point is statistically wrong. The point of margin of error is because the mean is NOT definitive. If I tell you I have a basket of 50 apples that is definitive. If I tell you I have a basked of 50 apples +/- 5 apples I am telling you that the number of apples is between 45 and 55 which is not definitive. By presenting just one number without a margin of error they are implying the number is definitive. Oh they can claim it isn't "definitive" but without also providing the margin of error the number they provide is essentially useless.

 

For example this week PFF rated:

Ed Oliver 80.3 
Tremaine Edmunds 78.2

 

If the error margin is +/- 2 points then both players would be statistically tied. One player cannot be said to rate higher than the other because the margin of error overlaps. The margin of error is an admission that the numbers are NOT definitive but the true value lies someplace in between with some level of confidence. 

[GunnerBill]

2 minutes ago, RememberTheRockpile said:

Your point is statistically wrong. The point of margin of error is because the mean is NOT definitive. If I tell you I have a basket of 50 apples that is definitive. If I tell you I have a basked of 50 apples +/- 5 apples I am telling you that the number of apples is between 45 and 55 which is not definitive. By presenting just one number without a margin of error they are implying the number is definitive. Oh they can claim it isn't "definitive" but without also providing the margin of error the number they provide is essentially useless.

 

For example this week PFF rated:

Ed Oliver 80.3 
Tremaine Edmunds 78.2

 

If the error margin is +/- 2 points then both players would be statistically tied. One player cannot be said to rate higher than the other because the margin of error overlaps. The margin of error is an admission that the numbers are NOT definitive but the true value lies someplace in between with some level of confidence. 

 

Nobody is saying the numbers are definitive. You want to portray it as such because it helps your agenda that PFF is worthless. You are entitled to that view. You are also wrong. 

[ColoradoBills]

10 minutes ago, GunnerBill said:

 

Each game is analysed by a lead analyst and then quality control checked before initial grades are applied and then re-watched and re-graded again when the all22 is available is my understanding.

 

Individuals are not judged on who they were up against or on how critical a play is in the game. They are some of the reasons I say, repeatedly, it is imperfect as a basis for any definitive determinations. You have to use it appropriately. It is a tool in the toolbox when properly applied. 

 

I have been a critic of plenty of its individual outcomes over the years. Its grading system led to far too much credit being given to Tyrod Taylor for example and last season Tre White ranked relatively average because it is hard to get the elite coverage grades when nobody throws your way any more. That is where nuance and the need for an experienced eye come into play in my mind. But it doesn't invalidate everything PFF do.

 

Thanks for your details about the process.  It's what I figured.  Like I said, I don't negate the importance of analytics and this system seems to be

a loose scoring of individual play.  I would also think that teams that use analytics to a higher degree would add some of the "variables" we both

just highlighted.

 

Your Tre White example is a great one.  I'm just now wondering how often PFF re-examines there inputs to adjust for more glaring deficiencies.

A good tool always needs to be tweaked.  Thanks again.

[oldmanfan]

To me the issue with any such assessments is trying to put a quantitative assessment on qualitative data.  Others have made very cogent points about the statistical limits inherent within the PFF analysis.  The biggest to me is that you are relying from what Gunner says on one primary analyst to do the grading, then reviewed by whatever a lead person is.  If I am reviewing a paper on scoring say tissue reactions to a given treatment (as I do commonly when reviewing scientific manuscripts), I need to see a measure of inter and intra-observer bias.  Intra-observer bias = how reproducible is an individual's grade if you have him look at the same sample multiple times.  Interobserver = how two or more different observers agree on the grade of an individual assessment.  Without knowing that, without knowing the training of the observers, without understanding how they know or assume the specific role for a given player on a given play, I just don't see how this kind of information can be that useful.  I wonder if the teams who subscribe are using it simply to compare to their own assessments, using it as a source of film for their use, et.

6 minutes ago, GunnerBill said:

 

Nobody is saying the numbers are definitive. You want to portray it as such because it helps your agenda that PFF is worthless. You are entitled to that view. You are also wrong. 

If the numbers are not definitive then I'm not sure what the value is of them supplying the numbers they supply.  I'm curious as to how you think teams should use this kind of data then.

[Reader]

10 minutes ago, RememberTheRockpile said:

Your point is statistically wrong. The point of margin of error is because the mean is NOT definitive. If I tell you I have a basket of 50 apples that is definitive. If I tell you I have a basked of 50 apples +/- 5 apples I am telling you that the number of apples is between 45 and 55 which is not definitive. By presenting just one number without a margin of error they are implying the number is definitive. Oh they can claim it isn't "definitive" but without also providing the margin of error the number they provide is essentially useless.

 

For example this week PFF rated:

Ed Oliver 80.3 
Tremaine Edmunds 78.2

 

If the error margin is +/- 2 points then both players would be statistically tied. One player cannot be said to rate higher than the other because the margin of error overlaps. The margin of error is an admission that the numbers are NOT definitive but the true value lies someplace in between with some level of confidence. 

 

For the sake of argument from a mathematical perspective. Saying there is an error of +/- 2 doesn't invalidate a difference between Oliver and Edmunds. If we pretend to ignore standard deviations beyond the first then the only way they are even would be if Oliver averages a -1 and Edmunds a +1. (Of course Oliver -2 and Edmunds 0 also works/all the other possibilities) It has been a while since I took statistics so I'm a little rusty, but I think that would mean there is about a 90% chance Oliver played better and while the ratings are still fallible, it is significant.

24 minutes ago, GunnerBill said:

 

Each game is analysed by a lead analyst and then quality control checked before initial grades are applied and then re-watched and re-graded again when the all22 is available is my understanding.

 

Individuals are not judged on who they were up against or on how critical a play is in the game. They are some of the reasons I say, repeatedly, it is imperfect as a basis for any definitive determinations. You have to use it appropriately. It is a tool in the toolbox when properly applied. 

 

I have been a critic of plenty of its individual outcomes over the years. Its grading system led to far too much credit being given to Tyrod Taylor for example and last season Tre White ranked relatively average because it is hard to get the elite coverage grades when nobody throws your way any more. That is where nuance and the need for an experienced eye come into play in my mind. But it doesn't invalidate everything PFF do.

 

Yeah, PFF was way too complimentary of Tyrod and didn't say a word when he bombed in Cleveland proving us right and them wrong. That being said, and I think you and I are in agreement here. It is useful to see their grades for players that we're not watching. People can say the oline had a good game, but did they? I wasn't watching them, I was watching Allen and I have no idea how well Spain picked up a stunt play after play.

[RememberTheRockpile]

3 minutes ago, GunnerBill said:

 

Nobody is saying the numbers are definitive. You want to portray it as such because it helps your agenda that PFF is worthless. You are entitled to that view. You are also wrong. 

You just don't get it. If they present a single number they are implicitly declaring it definitive. You claim they are not then where is the margin of error? Is it so big that the number they are presenting would become entirely useless? 

 

You don't like what I am saying because it is conflict with what you want to believe. You have failed to address any of the points I have made. You now accuse me of having an agenda which is a rather cute ad hominem considering you have failed to address what is basic statistical practice. It isn't my view, it is the view of anybody that has taken even  a basic statistics course. You conclude with saying I am wrong based entirely on your ignorance of statistics.

[GunnerBill]

7 minutes ago, RememberTheRockpile said:

You just don't get it. If they present a single number they are implicitly declaring it definitive. You claim they are not then where is the margin of error? Is it so big that the number they are presenting would become entirely useless? 

 

You don't like what I am saying because it is conflict with what you want to believe. You have failed to address any of the points I have made. You now accuse me of having an agenda which is a rather cute ad hominem considering you have failed to address what is basic statistical practice. It isn't my view, it is the view of anybody that has taken even  a basic statistics course. You conclude with saying I am wrong based entirely on your ignorance of statistics.

 

Haha. Of course. You can talk statistical practice at me all you like. These are not statistics in the traditional sense. You are saying "ah numbers, let me revert to statistics 101". That isn't what PFF are doing. Sorry to break it to you. 

[RememberTheRockpile]

5 minutes ago, Reader said:

 

For the sake of argument from a mathematical perspective. Saying there is an error of +/- 2 doesn't invalidate a difference between Oliver and Edmunds. If we pretend to ignore standard deviations beyond the first then the only way they are even would be if Oliver averages a -1 and Edmunds a +1. (Of course Oliver -2 and Edmunds 0 also works/all the other possibilities) It has been a while since I took statistics so I'm a little rusty, but I think that would mean there is about a 90% chance Oliver played better and while the ratings are still fallible, it is significant.

The 90% chance would be based on a gaussian distribution which I doubt is valid. In fact I would expect due to the subjective nature of the analisis a well known players distribution would be significantly different from some unknown player entirely due to bias. Oldmanfan does a nice job of highlighting the enormity of pitfalls involved even with trained evaluators. 

[ColoradoBills]

3 minutes ago, Reader said:

 

For the sake of argument from a mathematical perspective. Saying there is an error of +/- 2 doesn't invalidate a difference between Oliver and Edmunds. If we pretend to ignore standard deviations beyond the first then the only way they are even would be if Oliver averages a -1 and Edmunds a +1. (Of course Oliver -2 and Edmunds 0 also works/all the other possibilities) It has been a while since I took statistics so I'm a little rusty, but I think that would mean there is about a 90% chance Oliver played better and while the ratings are still fallible, it is significant.

 

After hearing Gunner's reply about the evaluation process (which is along the lines of what I figured) you could determine that the results is a loose (generic if 

you will) scoring of the players effectiveness.

It seems to me IF the scoring is done that way having the result reflecting a specific number with a decimal place is a bit conflicting.

To say it another way, if you are examining in generalities the score should be in generalities.

I would think now, PFF would be better off scoring both Edmunds and Oliver as a B+ (or whatever the 78-80 score equates to).

 

The decimal point score implies exactness which like Gunner said, is not the way the evaluation is done.

[RememberTheRockpile]

2 minutes ago, GunnerBill said:

 

Haha. Of course. You can talk statistical practice at me all you like. These are not statistics in the traditional sense. You are saying "ah numbers, let me revert to statistics 101". That isn't what PFF are doing. Sorry to break it to you. 

They are not statistics at all. What they are doing is selling you dubious numbers dressed up as "advanced statistics". 

[Reader]

6 minutes ago, RememberTheRockpile said:

The 90% chance would be based on a gaussian distribution which I doubt is valid. In fact I would expect due to the subjective nature of the analisis a well known players distribution would be significantly different from some unknown player entirely due to bias. Oldmanfan does a nice job of highlighting the enormity of pitfalls involved even with trained evaluators. 

 

I appreciate the response and I think you are right. I guess for me it's trying to find that balance of PFF is laughable to PFF is gospel and I guess I err on the side of the latter in an attempt to balance how I feel most of the board leans towards the former.

[oldmanfan]

9 minutes ago, GunnerBill said:

 

Haha. Of course. You can talk statistical practice at me all you like. These are not statistics in the traditional sense. You are saying "ah numbers, let me revert to statistics 101". That isn't what PFF are doing. Sorry to break it to you. 

Then what are they doing?  People are explaining to you why their methods can be questioned, but you're not explaining what they want to accomplish with their site.  When you use numbers to compare variables, that is statistical analysis by definition.

[Reader]

7 minutes ago, ColoradoBills said:

 

After hearing Gunner's reply about the evaluation process (which is along the lines of what I figured) you could determine that the results is a loose (generic if 

you will) scoring of the players effectiveness.

It seems to me IF the scoring is done that way having the result reflecting a specific number with a decimal place is a bit conflicting.

To say it another way, if you are examining in generalities the score should be in generalities.

I would think now, PFF would be better off scoring both Edmunds and Oliver as a B+ (or whatever the 78-80 score equates to).

 

The decimal point score implies exactness which like Gunner said, is not the way the evaluation is done.

 

PFF's ratings are weird. Fwiw, I think both of them are around a B+. I think for them 60-65 is average, 65-75 is good, 75-85 is great, and 85+ is elite. If average is a C, I'm not exactly sure about the breakdown, but B+ would be my guess.

[RememberTheRockpile]

18 minutes ago, ColoradoBills said:

 

After hearing Gunner's reply about the evaluation process (which is along the lines of what I figured) you could determine that the results is a loose (generic if 

you will) scoring of the players effectiveness.

It seems to me IF the scoring is done that way having the result reflecting a specific number with a decimal place is a bit conflicting.

To say it another way, if you are examining in generalities the score should be in generalities.

I would think now, PFF would be better off scoring both Edmunds and Oliver as a B+ (or whatever the 78-80 score equates to).

 

The decimal point score implies exactness which like Gunner said, is not the way the evaluation is done.

 

Using letter grades would be a crude form of error margin which would be an improvement. 

 

12 minutes ago, Reader said:

 

I appreciate the response and I think you are right. I guess for me it's trying to find that balance of PFF is laughable to PFF is gospel and I guess I err on the side of the latter in an attempt to balance how I feel most of the board leans towards the former.

 

I seriously doubt what is available to the NFL teams is available to the general public. I suspect they have at least 2 markets, pro/college, and consumer where the products are significantly different. I would also expect the pro/college products are customized (big bucks there) to the customers specs and heavily quantitative. The consumer products OTOH are highly subjective that appeal to the general publics desire. 

 

As much as we hate the hoodie we respect his knowledge of football. Here is what he has to say:

https://bostonsportsmedia.com/2014/06/04/can-pro-football-focus-stats-be-blindly-trusted/

Quote

But believe me, I’ve watched plenty of preseason games this time of year and you’re looking at all the other teams in the league and you try to evaluate players and you’re watching the teams that we’re going to play early in the season and there are plenty of plays where I have no idea what went wrong. Something’s wrong but I don’t…these two guys made a mistake but I don’t know which guy it was or if it was both of them. You just don’t know that. I don’t know how you can know that unless you’re really part of the team and know exactly what was supposed to happen on that play. I know there are a lot of experts out there that have it all figured out but I definitely don’t. This time of year, sometimes it’s hard to figure that out, exactly what they’re trying to do. When somebody makes a mistake, whose mistake is it?

 Moral of the story. If people want something that is not possible to provide someone will provide a product that looks like it.