How would moneyball work in Football?

[cage]

I'm surprised there isn't a thread on this yet, but one of Brandon's big announcements was the Bills will develop a sophisticated analytics operation to evaluate players....

 

NFL.com article

 

I'm interested in thoughts on several things.... does anyone know if any other NFL teams have dedicated any significant resources or had success with this approach? Where and how would it be applied outside of salary negotiations? I'm wondering what examples people can point to on the good/bad decisions that might be made with this approach?

 

If we were using sophisticated analytics I imagine that we would never have signed Fitz to a big contract, may have passed on Mario Williams in free agency and might have let Stevie walk?

[Al Cowlings]

Its a salary capped league, what are you talking about? Everyone spends the same amount of $$.

[jr1]

Kyle Moore instead of Kelsey

[K-9]

I'm skeptical that baseball's "moneyball" metrics can work in the NFL. Baseball is the epitome of individual performance. Football is the exact opposite. Every play depends on the proper execution of several individuals on both sides of the ball. Failure of a play is often a failure of one or more people not executing their assignments properly. Within that context, I would think it's difficult to always ascertain individual performance measurements without deep analysis of each player's specific role during a play. While some measurements are easy, especially for skill players, quantifying others would depend on breaking down each play and assignment. Not impossible, but not as easy as simply inputting numbers.

 

GO BILLS!!!

[EmotionallyUnstable]

Interesting post. I'm curious to what stats the management would key in on. In baseball it is pretty straight forward how they did it. On base percentage all the way. In football i don't think it is so cut and dry. What stat do you use? YAC, competition percentage, etc.? A cool idea but I think football is too complex of a sport that makes this system insignificant. I'd love to see if someone could figure out a system to weigh potential based on stats in football. Personally, I don't think it can be done. The intangibles that makes up this great sport is something that separates players at every position. How do you quantify this?

 

 

Its a salary capped league, what are you talking about? Everyone spends the same amount of $$.

 

I believe he is referring to the assessment of talent as opposed to the financial aspect.

[K-9]

Its a salary capped league, what are you talking about? Everyone spends the same amount of $$.

 

That's certainly true. But it's inarguable that some teams spend it smarter than others.

 

GO BILLS!!!

[playman]

yep. more than just " eyeball "

[Best Player Available]

A couple of examples of what good teams use that for. Calculated odds of hitting a FG from the 34 yard line instead of punting. Going for 2 points on the first TD instead of kicking the PAT. Chip Kelly of the ducks is a master of those type of things and a reason the Ducks think they can still succeed without him. Since his analytics are a huge part of their program.

[Kelly the Dog]

One way is to discover how well an OL or a DL is performing. If you base a DL's performance on the number of sacks he gets or the number of tackles, it's a ridiculously inefficient way of looking at how well he plays. Obviously, they don't just do that, they study a lot of film on the guy and grade him. But breaking down a OL or DL's performance based on metrics, how many times he beats his man, gets a draw or gets beat, gives you a much better indication of his play. It can be broken down 20 different ways, and when combined with film study, seems to me to be a far, far superior way of evaluating him.

[dave mcbride]

San Francisco tried a moneyball approach around 2005 or so, and it failed for a number of years. The guy behind it is Paraag Marathe: http://sfist.com/2006/01/03/those_moneyball_49ers.php . He's still one of the powers in the organization, though, so I guess one could argue that it paid off in the end. They were in the wilderness for a long time, though. Basically, it's really hard, not just because of the way you value players and assign credit on plays with 22 moving parts. Injuries are much more frequent in the NFL and careers shorter. Hence when you're thinking about signing someone long term the formula doesn't really work given the projection issues.

[hondo in seattle]

This is a good OP.

 

Here's the little I've gleaned from a variety of sources - hopefully credible.

 

* Many, if not most, NFL teams use some sort of sabremetric-type analytics when evaluating player performance. These might be used by scouts and/or by coaches, depending on the team and circumstance.

 

* For example, I've read that some teams use their own in-house analytical metrics to measure QB performance (instead of using passer efficiency or QBR).

 

* I've also read that no NFL franchise is as devoted to sabremetrics as some baseball clubs are. The use in the NFL is far more limited.

 

* Sabremetrics in football is difficult because, as mentioned, it's a team sport. If the players around you are untalented or confused, your performance will suffer. If you're coached poorly, your performance will suffer.

 

* Sabremetrics is also logistically difficult. Ideally, each play would be broken down 22 times - once for each player on the field.

 

* Breaking down each player's performance on each play involves a lot of subjectivity.

 

* NFL teams are secretive and it's hard to really know what they're actually doing behind closed doors.

 

Like the OP, I'd like to know more what Russ intends.

[bigK14094]

That's certainly true. But it's inarguable that some teams spend it smarter than others.

 

GO BILLS!!!

Yes, but remembering the Jason Peters case, the Bills were very smart. Peters was locked up for three years.....but he decided that he wouldn't play for the paltry 3 mill/year he signed for, as he was an all pro. So, he held out, and forced the trade to Philly (hows that working out btw) Andhow, they had him, and he choose not to honor the contract. Lots of cases like this in football. I am thinking that moneyball won't work particularily well in the non contract compliance envieonment that is the current NFL.

[K-9]

Yes, but remembering the Jason Peters case, the Bills were very smart. Peters was locked up for three years.....but he decided that he wouldn't play for the paltry 3 mill/year he signed for, as he was an all pro. So, he held out, and forced the trade to Philly (hows that working out btw) Andhow, they had him, and he choose not to honor the contract. Lots of cases like this in football. I am thinking that moneyball won't work particularily well in the non contract compliance envieonment that is the current NFL.

 

Agreed. In my mind it's quite simple and it trumps any metrics used in quantifying a player's worth. If the player doesn't want to be here, I'm looking for a way to ship his ass out of town for as best a return as I can get, especially such an important cog on the OLine as the LT. As talented as he is, Peters' unprofessional behavior during that off season and subsequent sub-par performance while he got himself into game shape, was a slap in the face to his teammates and the fans alike.

 

GO BILLS!!!

[voodoo poonani]

The NE Patriots for one. Here's an old article from MIT's magazine: link.

[OGTEleven]

I could see moneyball being applied somewhat readily to defensive players and not so much to offensive players. Oddly enough, this is similar to baseball using it for offensive players over defensive players.

 

A football defender and a batter in baseball are both reacting to something being thrown at them. The defense (pitcher) initiates every play in baseball where the offense initiates it in football. You can look at film and determine most of the time how the football defender matched up against the other team. Did he draw a double team? Penetrate? Cover his man? Sometimes you won't be sure if he executed his assignment but most of the time you'll be able to tell by watching. Even so, you can't have others do the watching for you by reading OBP stats like baseball. That is a lot of people watching a lot of film and that makes it subject to varying interpretations.

 

I haven't read the book Moneyball (movie only) but I am a baseball fan. I have not seen anything indicating that Beane used the metrics on pitching/defense but he didn't completely ignore them building that winning team. They did have Hudson/Zito/Mulder after all. Evaluating Scott Hatteberg as a value against Jeremy Giambi is one thing and I think it can be important, but you still need pitching. That is where everything starts. A quarterback is like a #1 starter that starts every game. Take the Bills this year and plug in Manning for Fitzpatrick. What would the record be?

 

I am not sure how to apply moneyball to WR (how do you know if a player operating from another team's playbook ran the right route?); RB (can apply it to rushes certainly but what about blocking/receiving? And even rushing is dependent on the OL as has been pointed out). I guess I can see applying it to the OL somewhat but again the assignments are not always clear.Moneyball worked (works) for the A's brilliantly with respect to hitters and production. They have been able to develop pitching talent consistently too and make some timely trades to keep the farm stocked. Some of these things could be done in football but some could not.

 

I am not against the moneyball concept but it would have to be a square one re-design for football. I'm hopeful but skeptical.

Edited January 2, 2013 by OGTEleven

[Orton's Arm]

I'm surprised there isn't a thread on this yet, but one of Brandon's big announcements was the Bills will develop a sophisticated analytics operation to evaluate players....

 

NFL.com article

 

I'm interested in thoughts on several things.... does anyone know if any other NFL teams have dedicated any significant resources or had success with this approach? Where and how would it be applied outside of salary negotiations? I'm wondering what examples people can point to on the good/bad decisions that might be made with this approach?

 

If we were using sophisticated analytics I imagine that we would never have signed Fitz to a big contract, may have passed on Mario Williams in free agency and might have let Stevie walk?

 

Developing a Moneyball tool for football would, as K9 has pointed out, be a difficult task. But not necessarily an impossible one.

 

If I was tasked with developing such a system, I'd do the following:

 

1. Define the variable I want to maximize

2. Develop metrics to measure players' contributions to those variables.

3. Measure each current and potential player on those metrics.

4. Obtain players who provide the maximum ratio of contribution to money spent.

 

For step 1, I think a fairly decent metric would be yards gained per play (offense) and yards yielded per play (defense). Those could be further broken down into yards/running play and yards/passing play. One could refine the above by also taking into account turnovers. Yards per passing play should be weighted three times as heavily as yards per running play or turnovers. (The number three comes from a regression analysis performed by The New York Times.)

 

For step 2, you define each player's task on a specific play. Then you rate how well he achieved his task. A quarterback's job is to throw the ball accurately, an offensive lineman's job is to win his battle against the guy across the line from him, etc. You could rate the accuracy of each pass on a scale from 1 - 5, the quality of a QB's decision on a scale from 1 - 3, etc. You could also rate the quality of a WR's catching on a scale from 1 - 5; where a 5 counts as catching a very poorly thrown ball, a 3 counts as catching a well-thrown ball, and a 2 counts as dropping a difficult-to-catch (but still catchable) pass.

 

Step 3 involves watching a ton of film. It will probably be necessary to watch the same game many times, just so that each player can be assigned a grade on the above-described scale. Ideally, you and your staff would watch at least 1 - 2 seasons' worth of games, with each player assigned a grade for each play. Once you had that data, it would be fairly straightforward to develop a statistical model which could answer questions like, "If an offensive lineman loses his battle against a defender, how many fewer yards is the running play likely to gain than would have been the case, had he won the battle?" Once you can answer questions like that, then you can determine your average expected yards per running play with Offensive Lineman A versus Offensive Lineman B.

 

For step 4, you look for guys who can provide the most yardage per play at the lowest possible cost. This does not necessarily mean weeding out star players and their high salaries. A guy like Aaron Rodgers or Peyton Manning almost certainly contributes enough to his team's yards per pass play to more than justify a princely salary.

[San Jose Bills Fan]

I can see where this discussion is headed.

 

You guys are gonna start communicating in algebraic equations, aren't you?

[BuffOrange]

Not sure what reason Brandon has to go public with this, but I'm glad to hear it nevertheless.

As a proud member of the "numbers guy" club I can't help but shake my head at a lot of the skepticism.

 

"Football isn't as easy quantifiable as baseball" - I don't think anyone on the planet including guys like Aaron Schatz or Bill Barnwell who write about it (football analytics) for a living would argue with this. That doesn't mean it can't be a useful tool.

 

"Who has done this that's been really successful?" If you lose $100 playing blackjack using basic strategy when you otherwise would've lost $500 playing on "gut feel" does that mean using basic strategy was a failure? Obviously talent & QB are more important than everything else as it relates to football games. That doesn't mean you shouldn't attempt to give yourself any little edge you can...especially when you're the Bills & don't have many edges.

 

A couple of examples of what good teams use that for. Calculated odds of hitting a FG from the 34 yard line instead of punting. Going for 2 points on the first TD instead of kicking the PAT. Chip Kelly of the ducks is a master of those type of things and a reason the Ducks think they can still succeed without him. Since his analytics are a huge part of their program.

 

This, exactly. Which makes it a bit hard to believe that we're interviewing Doug Marrone, who punts from the opponent's 35 with the best of 'em.

[GOBILLS78]

Nerds!

 

(Chugs can of PBR, smashes it against forehead, coaches losing football team.)

[jjamie12]

Seriously? No one here has heard of Football Outsiders?

[BuffOrange]

Seriously? No one here has heard of Football Outsiders?

 

I was thinking the same thing....for a crowd that loves to gripe about espn, there is a smarter world out there.

[San Jose Bills Fan]

Seriously? No one here has heard of Football Outsiders?

 

Actually they, along with Cold, Hard, Football Facts, and Pro Football Focus have been discussed here numerous times.

[yungmack]

What makes Money Ball work in MLB is the absence of shared revenue, which leads to massive differences in the amounts of money available to spend on salaries from the "poorer" teams to the extremely wealthy.

 

The "money" part of Money Ball is useless in the NFL where teams each receive the same amount money and now have to spend a minimum on salary, and all are capped on the maximum.

 

HOWEVER, the formulae for identifying and ranking the value of one player over the other is still highly useful even if finding bargains is virtually nil. In other words, the money you intend to spend on players still gets spent but hopefully the sabremetrics approach does a better job of identifying the right players on which to spend that money.

[BisonMan]

There is really no limit to the number of things that you could measure in football because the game is so intricate. The important factors are the ones that smart clubs would rely upon. For example, turnovers are likely a key metric in driving winning percentage. They also might be the most random. For example, forced fumbles might be good but fumble recoveries might not be. Interceptions in man-to-man coverage might be weighted higher than interceptions in zone coverage, etc.

 

For individual positions, each team likely has their own evaluation standards that should be tracked and standardized based on the competition. For example, if a Bills OT gets beat 3 times in one week by JJ Watt, that's not going to be rated as poorly as somebody getting beat 2 times by a second stringer. Essentially, they have to control for the environment (Cetus Paribus for all of your statistics geeks).

 

 

They may find some metrics telling that nobody really thought were important. For example, a normally stellar CB might experience a blown coverage once every 20 plays but that blown coverage wipes out all the good he's done the other 19 plays in terms of yardage given up or scores. The team might decide that he is worth less than a mediocre CB who never gives up big plays. Everybody knows Revis is great but how do you evaluate two lesser talents with different tendencies?

 

 

Once they evaluate performances, they have to pour the information into their salary system in order to make sure they aren't overpaying for talent. This is where the Moneyball factors come into play. If the stellar CB is priced more highly by "traditional football minds" and your evaluation systems shows that the mediocre CB delivers the same about in terms of WINS, you pay for the mediocre CB and let Philly sign the other guy.

 

I also think that metrics might dictate the way you play the game as far as the QB position. The A's played the game differently in the early 2000s than other teams did. They took more walks and their slugging percentages were lower. Nonetheless, they won more games than almost anyone else. We may be seeing this played out this year in Seattle where the Seahawks QB does not throw as well by traditional measures as a more traditional QB but they've changed the style of play for the position to get just as much out of him (run option). Obviously, teams will catch up to this going forward to some extent but in the meantime, they are succeeding with a 3rd round rookie at the helm.

 

Anyone wanting to read a cool book on sports statistics and their use by team, try Scorecasting: The Hidden Influences Behind How Sports Are Played and Games Are Won by Tobias Moskowitz and L. Jon Wertheim. It's terrific.

[dave mcbride]

Actually they, along with Cold, Hard, Football Facts, and Pro Football Focus have been discussed here numerous times.

I don't think they're discussed enough, truth be told.

 

For instance, check this out: http://www.footballoutsiders.com/extra-points/2012/jj-watt-obliterates-record-defensive-defeats .

 

Notice that in 2006 - London Fletcher's last year with the Bills - he registered one of the best records of "defeats" in the past 15 year or so years. The Bills thought his time was up, as we all know.

[K-9]

I don't think they're discussed enough, truth be told.

 

For instance, check this out: http://www.footballo...fensive-defeats .

 

Notice that in 2006 - London Fletcher's last year with the Bills - he registered one of the best records of "defeats" in the past 15 year or so years. The Bills thought his time was up, as we all know.

 

I don't know. Realizing that Defeats for a DL and LB are created differently, and taking nothing away from Watt or any other great player, when I see Nick Barnett is tied 11th in the league for Defeats through week 13 of this season, I gotta wonder about this stat. He was simply horrible this year and his numbers would reflect the sheer number of times teams went after him more than anything else IMHO.

 

http://www.footballoutsiders.com/extra-points/2012/nfl-leaders-defeats-weeks-1-13

 

GO BILLS!!!

[DC Tom]

Developing a Moneyball tool for football would, as K9 has pointed out, be a difficult task. But not necessarily an impossible one.

 

If I was tasked with developing such a system, I'd do the following:

 

1. Define the variable I want to maximize

2. Develop metrics to measure players' contributions to those variables.

3. Measure each current and potential player on those metrics.

4. Obtain players who provide the maximum ratio of contribution to money spent.

 

For step 1, I think a fairly decent metric would be yards gained per play (offense) and yards yielded per play (defense). Those could be further broken down into yards/running play and yards/passing play. One could refine the above by also taking into account turnovers. Yards per passing play should be weighted three times as heavily as yards per running play or turnovers. (The number three comes from a regression analysis performed by The New York Times.)

 

For step 2, you define each player's task on a specific play. Then you rate how well he achieved his task. A quarterback's job is to throw the ball accurately, an offensive lineman's job is to win his battle against the guy across the line from him, etc. You could rate the accuracy of each pass on a scale from 1 - 5, the quality of a QB's decision on a scale from 1 - 3, etc. You could also rate the quality of a WR's catching on a scale from 1 - 5; where a 5 counts as catching a very poorly thrown ball, a 3 counts as catching a well-thrown ball, and a 2 counts as dropping a difficult-to-catch (but still catchable) pass.

 

Step 3 involves watching a ton of film. It will probably be necessary to watch the same game many times, just so that each player can be assigned a grade on the above-described scale. Ideally, you and your staff would watch at least 1 - 2 seasons' worth of games, with each player assigned a grade for each play. Once you had that data, it would be fairly straightforward to develop a statistical model which could answer questions like, "If an offensive lineman loses his battle against a defender, how many fewer yards is the running play likely to gain than would have been the case, had he won the battle?" Once you can answer questions like that, then you can determine your average expected yards per running play with Offensive Lineman A versus Offensive Lineman B.

 

For step 4, you look for guys who can provide the most yardage per play at the lowest possible cost. This does not necessarily mean weeding out star players and their high salaries. A guy like Aaron Rodgers or Peyton Manning almost certainly contributes enough to his team's yards per pass play to more than justify a princely salary.

 

Except in your weird-ass mathematical world, that would never work, because error would always cause performance to regress to the mean, you !@#$ing froot loop.

[nashvillebills]

"I saw a great film recently. It was called "Moneyball". That Brad Pitt is so handsome. It was a real entertaining motion picture. In the movie, they used mathematics and these thingeys called "statistics" to evaluate a players overall worth..... I figured, what the hell, let's give it a try, we'll be viewed as innovative and forward-thinking!" -Russ B.

 

gagbarfgaggggbarfffffff..gllllllllllllwoosh.

[Orton's Arm]

Except in your weird-ass mathematical world, that would never work, because error would always cause performance to regress to the mean, you !@#$ing froot loop.

 

The mods haven't gotten around to banning you yet?

[BiLLsMaFiA1982]

Kyle Moore instead of Kelsey

 

This is a great analytics example in upside!

[K-9]

This is a great analytics example in upside!

 

Except the analytics didn't tell us how bad Kyle Moore is against the run.

 

GO BILLS!!!

[Buffalo Barbarian]

I'm surprised there isn't a thread on this yet, but one of Brandon's big announcements was the Bills will develop a sophisticated analytics operation to evaluate players....

 

NFL.com article

 

I'm interested in thoughts on several things.... does anyone know if any other NFL teams have dedicated any significant resources or had success with this approach? Where and how would it be applied outside of salary negotiations? I'm wondering what examples people can point to on the good/bad decisions that might be made with this approach?

 

If we were using sophisticated analytics I imagine that we would never have signed Fitz to a big contract, may have passed on Mario Williams in free agency and might have let Stevie walk?

 

 

hopefully this will help buddy see and feel a lot better.

 

 

"More traditional GMs, like the Bills' Buddy Nix, don't know what to make of it.

 

"You know, obviously, I'm old-school in more ways than one," Nix said, laughing. "It'll be something I'll have to get used to because I go a lot on feel and what I see."

[Kelly the Dog]

There was a good interview with the dude from Football Outsiders on GR this afternoon. It's probably in their audio vault. He repeatedly said that his metrics could be used in the NFL as a guide to make coaches and talent scouts evaluate guys on film. He went out of his way to say football metrics can only do so much and you should never base your decisions on them. It should point out stuff and players that may go unnoticed and then you should take a closer look at that player on film and then make your decision. It was a pretty interesting admission from a guy on the forefront of the football metrics business.

 

He also pointed out the vast difference between baseball metrics and football metrics, especially the obvious difference being that if you take a baseball player off one team and put him on another, he's going to be the same hitter with the possible exception of the park dimensions. But that doesn't at all apply to football players.

[San Jose Bills Fan]

Hopefully this will help Buddy see and feel a lot better.

 

"More traditional GMs, like the Bills' Buddy Nix, don't know what to make of it.

 

"You know, obviously, I'm old-school in more ways than one," Nix said, laughing. "It'll be something I'll have to get used to because I go a lot on feel and what I see."

 

Even the biggest proponents of analytics/sabermetrics admit that it has limitations.

 

But the Nix quote above is exactly the sort of statement that Michael Lewis attacked in Moneyball.

 

It's probably useful to consider the old school and the new approach.

[Buffalo Barbarian]

San Francisco tried a moneyball approach around 2005 or so, and it failed for a number of years. The guy behind it is Paraag Marathe: http://sfist.com/200...yball_49ers.php . He's still one of the powers in the organization, though, so I guess one could argue that it paid off in the end. They were in the wilderness for a long time, though. Basically, it's really hard, not just because of the way you value players and assign credit on plays with 22 moving parts. Injuries are much more frequent in the NFL and careers shorter. Hence when you're thinking about signing someone long term the formula doesn't really work given the projection issues.

 

Maybe now it is more refined and have tweaked it to work after some early failures like many inventions today that we take for granted.

[Buffalo Barbarian]

Even the biggest proponents of analytics/sabermetrics admit that it has limitations.

 

But the Nix quote above is exactly the sort of statement that Michael Lewis attacked in Moneyball.

 

It's probably useful to consider the old school and the new approach.

 

I agree that all methods should be used, but I basically was saying that I don't like how he as drafted so far.

[Buftex]

"I saw a great film recently. It was called "Moneyball". That Brad Pitt is so handsome. It was a real entertaining motion picture. In the movie, they used mathematics and these thingeys called "statistics" to evaluate a players overall worth..... I figured, what the hell, let's give it a try, we'll be viewed as innovative and forward-thinking!" -Russ B.

 

gagbarfgaggggbarfffffff..gllllllllllllwoosh.

 

Well...I am not saying I agree with the "moneyball" principle for football, but Russ Brandon worked for the Florida Marlins, prior to the Bills...the franchise won a World Series using "moneyball"...and another one later... I am skeptical...

[pimp 2]

The part of using this to attack the opponents weakness in different formations, is what intrigues me. This I believe is what Bill-cheat does so will against his opponents...what I get from Russ is that he wants a HC that buys into this concept of strategy and not just the traditional way of film study. Andy Reid (other coaches also) used the scripted 8-12 plays with different formations to tip the DC's gameplan, then flip his charts to adjust his offensive gameplan to the Def formations weakness...pretty sure he used some form of analytically data for player or formational weakness.Hell, I used to scream at Chan (thru the TV) because it never seemed like the Bills had a glue of how to attack the opponents weakness. The game that convinced me of this was the Jets game when Maybin had the 2 sacks. Knowing Maybins weakness at DE, CG never ran not once at him while he was on the field...not even a draw or screen.

[Phlegm Alley]

http://sportsillustrated.cnn.com/vault/article/magazine/MAG1206508/1/index.htm

 

Everyone should read this article to help them understand exactly what types of analytics teams are using. Not just for player evaluation (attaching a GPS to each player to get an idea of WR's route running, or closing time for a CB in pass coverage), but also for game situations, like how to attack in OT (should teams be more conservative, or run the 2-minute drill?).

 

A lot of teams have had analytics departments for some time now, and they make it no secret. Whatever advantage you have to win should be exploited.

[bowery4]

I mean there is this link, so I get the jist of it... but my question is how new and effeftive are they?

http://www.wgrz.com/news/article/194705/37/What-The-Heck-Is-Football-Analytics

I haven't seen any discussion about it on here, just wondering what you all think.

 

Sorry just saw the money ball thread this can be deleted mods