Are ACL injuries no big deal anymore? The data suggest they are

[FireChans]

Most players are probably out of the league after an ACL because teams don’t want to keep a roster spot for a rehabbing player who isn’t an impact player.

 

If Dane Jackson tore his ACL instead of Tre, instead of “keeping his roster spot warm for him” we would just find a replacement player.

[Rubes]

1 hour ago, Hapless Bills Fan said:

 

 

The major problem I have with this is lack of matched cohort.

 

The joke is that NFL stands for "Not For Long"; the average overall career length in the NFL is 3.3 years, with 2.9 being the average for CB, 2.8 for WR, and 2.5 for RBs

https://www.statista.com/statistics/240102/average-player-career-length-in-the-national-football-league/

 

So given that, the right comparator likely isn't to players in other leagues post-injury, but to uninjured players at that position. 

 

Yeah, I mentioned that a few posts above. The problem is that they chose the wrong comparator group—they compared the injured players with themselves pre-injury, and also injured players who returned to play vs. those who did not return to play. Both are incorrect comparisons. They should be comparing injured players to age-, NFL experience-, and position-matched players who did not suffer an ACL injury at that point in their careers.

 

[djp14150]

13 hours ago, Warcodered said:

Less of not a big deal and more of not a death sentence anymore.

i have an uncle thru marriage ( they are divorced now).

 

 He was a top CB in college. Played at senior bowl. Had a knee injury and his career was over.  Had that happened today, he would have come back and had a career.

[Mat68]

Its tough to look at the total number and correlate that to White.  The average career length of NFL players is 3 years.  How many were all pros already on there second contract.  White is a different human than the majority of those on that list.  Top of the top athletes heal different and react different than any of us.  Heck, even guys that made it to the league.  ACL is no longer career ending.  If your a Wolverine like Adrian Peterson you come back in 6 months and have a career year.  

[Paup 1995MVP]

15 hours ago, Rubes said:

I'll admit that I've started believing that ACL injuries have become less of a concern in recent years, with players seemingly coming back to near pre-injury performance maybe the year after returning to play. Is that necessarily the case? Maybe not so much.

 

From the March 7th Orthopaedic Journal of Sports Medicine:

 

Return to Play and Performance After Anterior Cruciate Ligament Reconstruction in National Football League Players

(full article may be behind a paywall, but you can at least read the abstract)

 

 

 

 

 

Obviously this is a population-level study, and conclusions at the population level don't necessarily apply to a particular individual and his specific injury. Not all ACL injuries are the same, depending on the nature and extent of the ligament injury. It is also specific to positions: interestingly, WRs were similar to QBs in losing only about a quarter (18% and 17%, respectively) of their total snap count postinjury. So perhaps something to consider with regard to someone like Jameson Williams.

 

But still, this is a pretty sobering study, though I haven't gone through it in great detail to critique the methods. It's important to bear in mind that their study group was only up to 2018, so any improvements in ACL repair since then are not taken into consideration. Also of interest, they note that "NFL players have significantly shorter careers postoperatively (2.1 years) compared with players in other professional leagues such as the National Basketball Association (NBA; 4.5 years), National Hockey League (NHL; 4.5 years), and Major League Baseball (MLB; 2.9 years)."

 

Overall, ACL injuries are still bad news for NFL players who aren't QBs. Thought some of you might be interested to read that.

 

I think the physicality of football versus baseball basketball and hockey is a big factor in NFL players having shorter careers postoperatively.  A lot of baseball players are not exactly physical specimens to begin with.  And they come back, and hobble around for awhile.  As long as they can hit reasonably well, and move some they can play first base or DH.  Or even 3rd base for that matter.  Hockey is a rough game (at least it used to be) but the players on the whole are considerably smaller mass wise then football players.  And that extra mass has to put more of a pounding on the knees.  Basketball similar to hockey as far as the mass of the players.  I am not a doctor (I just play one on TV.  Just kidding)  But that is my conclusions from a lifetime of watching sports.  

 

As an aside.  My son tore his ACL in his college football teams first game of the 2019 season.  He plays DE, and got caught up in a big pile.  He was out for the rest of the 2019 season after having surgery to repair the injury.  He was back in time for the first game of the 2020 season.  But he said it took him several games in 2020 before he was back up to speed, mentally and physically.  He is a 6th year senior this coming season.  He has been moved off the ball to more of an OLB/edge rusher type player.  Hopefully he has a big year.  His team has a big year, and he will get his shot in the NFL in 2023.  And of course stay healthy.   

[Mr. WEO]

5 hours ago, Stranded in Boston said:

Yeah, guys. I haven't looked at the article yet (maybe check out on PubMed later today), but without a control group, the results are impossible to interpret. Not to wonk out here (too much), but short of a matched cohort (which might be tough to assemble given small N) this is exactly where you'd need to apply some sort of nested multiple regression or proportional hazards model including ALL NFL players to first account for variance (in subsequent # years played) that is attributable to non-ACL factors like age, position played, previous injury history, previous # snaps (for non-overt related wear and tear), etc. -- and only then add the ACL/non-ACL factor to the model to see if it explains any residual variance. Pretty sloppy oversight if they didn't do that ... (They're lucky we at TBD didn't review their paper!  🤓  )

 

5 hours ago, Rubes said:

 

Don't be so sure...most people in medicine are not trained to do research, and a good deal of research that is done is not done well. It's the job of the journal reviewers to spot problems with study design or analysis, and that process is fraught with issues—not the least of which is that reviewers aren't paid to do it, so you never know how carefully they do their job. Journals also gain from articles that draw attention, even if they are not done well, so there is a motivation in some cases to publish rather than reject. It's hard to tease apart.

 

Still, these authors are from Drexel and Duke, two pretty strong research universities, so it's a little surprising. The problem here is that they didn't use the right comparator group. They were comparing individuals pre- and post-injury, and comparing those who were injured and returned to play vs. those who were injured and never returned to play. The right comparator, as I mentioned earlier, would be those who were injured vs. those who weren't injured, matched based on their age, years of NFL experience, position, and maybe other factors like round they were drafted, whether they were a starter or backup at the time of injury, and so on.

 

 

3 hours ago, Hapless Bills Fan said:

 

 

The major problem I have with this is lack of matched cohort.

 

The joke is that NFL stands for "Not For Long"; the average overall career length in the NFL is 3.3 years, with 2.9 being the average for CB, 2.8 for WR, and 2.5 for RBs

https://www.statista.com/statistics/240102/average-player-career-length-in-the-national-football-league/

 

So given that, the right comparator likely isn't to players in other leagues post-injury, but to uninjured players at that position. 

 

The other big differentiators are probably skill and contract guarantees.  When a player has guaranteed money, or a high draft pick invested in him, or has been a star prior to his injury, being given time to get back to his old form is more likely.

 

The average player on a non-guaranteed contract is fighting for a "hat" every year.   If he tears an ACL, the team he's been playing in is obligated to pay for his medical care and rehab, but there's probably someone of similar skill level with two good knees in the draft or kicking around as a FA, so the injured player's chance of RTP are poor.  That may reflect a team's lack of interest in "taking a chance" on an injured player who may play at a lesser level his first year back, vs. a player of similar traits who is uninjured.

 

Consider Harrison Phillips.  The Bills had a 3rd round pick invested in him and few options on the roster at 1TDT when Star opted out in 2020.   He was NOT the same player when he initially returned from his ACL.  He was a "healthy scratch" for 4 games, and down to 23-33% of the snaps after that, until the very end of the season, and the Bills did move on from him this season, but they were patient with him in 2020 and 2021.

 

OTOH for a player like Justin Zimmer who was a low-level FA barely hanging on to the team with a 1 year contract in 2021, his ACL may well have ended his career.

 

The control in tis study is the injured player before injury.  What would they be comparing if they included non-injured players?  The variable is the injury: does it affect the play (function as an NFL player) going forward.  Maybe breaking down by age would have given more insight.  

 

 

 

The "average player career length" data aren't really useful as they include any guy who signed a contract or got a check.  There's a ton of such who never make it onto a week 1 roster. So all of those players who quickly wash out after training camp would render any "control" comparison meaningless--and not appropriate to answer the question the study was asking..

 

The exclusion criteria for the study would eliminate rookies and any player who never played another game.  The latter number made up 12.5 % of the entire pool of NFL ACL injuries in the time period (only 6 seasons).  The study did note the different outcomes found in different positions and noted their draft position, as well as, interestingly, BMI (the average BMI was over 30: "obese").

 

It's not a bad study.  It shows that players, overall, aren't the same after the injury, but there is variability among different player positions. 

 

Scrolling through the paper's references, there's this re: draft position and skilled vs nonskilled players position:

 

https://journals.sagepub.com/doi/10.1177/2325967117729334

 

"The overall RTP rate was 61.7%, with skilled players and unskilled players returning at rates of 64.1% and 60.4%, respectively (P = .74). Early draft-round players and unskilled late draft-round players had greater rates of RTP compared to skilled late draft-round players and both unskilled and skilled undrafted free agents (UDFAs). Skilled early draft-round players constituted the only cohort that played significantly fewer games after an injury. Unskilled UDFAs constituted the only cohort to show a significant increase in the number of games started and ratio of games started to games played, starting more games in which they played, after an injury."

 

here's another:

 

https://www.sciencedirect.com/science/article/pii/S074980631830238X

 

looked at ACL injured vs. "controls" re: post injury snap counts, etc

 

Edited April 6, 2022 by Mr. WEO

[Rubes]

2 hours ago, Mr. WEO said:

The control in tis study is the injured player before injury.  What would they be comparing if they included non-injured players?  The variable is the injury: does it affect the play (function as an NFL player) going forward.  Maybe breaking down by age would have given more insight.  

 

Ultimately, the best comparison would be between what happened to the player with the injury, and what would have happened to the player had he never had the injury (ie, the counterfactual). But since that's impossible, you try to do the next best thing: compare what happened to the player with the injury vs. what happened to other, similar players beginning at the same time point in their careers. If you can make the comparison group as similar to the injured players as possible, then the results are more valid.

 

So what you should look for are other players in the league who are similar in age and size, play the same position, and have a similar history (college career, time in the NFL, snaps at the NFL level, etc) but who did not have an ACL injury up to the same time point in their careers as the matched injured player. So you would do this matching for every injured NFL player using one or more of these controls, and then compare their subsequent performance and longevity (accounting for the time out due to the injury).

 

Comparing the injured player before vs. after their injury is not an ideal comparison because so many other non-injury-related factors can impact the outcomes being measured—eg, how much longevity and playing time they experience.

 

[Rubes]

2 hours ago, Mr. WEO said:

here's another:

 

https://www.sciencedirect.com/science/article/pii/S074980631830238X

 

looked at ACL injured vs. "controls" re: post injury snap counts, etc

 

Yeah, that last article is a decent example of what I mean—it's a matched case-control study where they looked at players who participated in the NFL Combine and who had a prior ACLR to examine the outcomes of draft position, games started, games played, and snap percentage. Each participant with an ACLR was matched with control players—players who participated in the NFL Combine who had no knee injuries or surgeries prior to the Combine, matched by position. In this case they matched only by position, but it at least allowed them to "eliminate variability among the different positions in terms of the unique stresses on the ACLR for each position." They were also essentially matched by age by virtue of the timing of the start point (participation in the NFL Combine). They could have matched on other characteristics, but this was probably the most important matching variable.

 

The study of ACL injuries during an NFL career gets a little more complicated given the timing of the injury and the more detailed pre-injury history needed to match against.

 

[Beast]

Unless you're Frank Gore.

[Mr. WEO]

49 minutes ago, Rubes said:

 

Ultimately, the best comparison would be between what happened to the player with the injury, and what would have happened to the player had he never had the injury (ie, the counterfactual). But since that's impossible, you try to do the next best thing: compare what happened to the player with the injury vs. what happened to other, similar players beginning at the same time point in their careers. If you can make the comparison group as similar to the injured players as possible, then the results are more valid.

 

So what you should look for are other players in the league who are similar in age and size, play the same position, and have a similar history (college career, time in the NFL, snaps at the NFL level, etc) but who did not have an ACL injury up to the same time point in their careers as the matched injured player. So you would do this matching for every injured NFL player using one or more of these controls, and then compare their subsequent performance and longevity (accounting for the time out due to the injury).

 

Comparing the injured player before vs. after their injury is not an ideal comparison because so many other non-injury-related factors can impact the outcomes being measured—eg, how much longevity and playing time they experience.

 

 

You would be comparing guys who's careers were obviously limited by their injury with guys whos careers were limited by not being good enough to stay in  the league.  I don't see how that's valid.    The question is what effect this injury has on a players subsequent performance.  Nothing "happened" to your proposed control group.   If, by chance,  such a study showed both groups were out of the league after the same career length, you certainly couldn't conclude the injury had no effect on the career of the injured players.

 

 

35 minutes ago, Rubes said:

 

Yeah, that last article is a decent example of what I mean—it's a matched case-control study where they looked at players who participated in the NFL Combine and who had a prior ACLR to examine the outcomes of draft position, games started, games played, and snap percentage. Each participant with an ACLR was matched with control players—players who participated in the NFL Combine who had no knee injuries or surgeries prior to the Combine, matched by position. In this case they matched only by position, but it at least allowed them to "eliminate variability among the different positions in terms of the unique stresses on the ACLR for each position." They were also essentially matched by age by virtue of the timing of the start point (participation in the NFL Combine). They could have matched on other characteristics, but this was probably the most important matching variable.

 

The study of ACL injuries during an NFL career gets a little more complicated given the timing of the injury and the more detailed pre-injury history needed to match against.

 

 

It seems clear from all evidence that most of these guys don't come back to their pre injury level of play. 

[GoBills808]

Take it for what it’s worth, because I did mine twice but obviously without access to NFL training/rehab facilities 

 

Neither was ever quite the same

[Rubes]

1 hour ago, Mr. WEO said:

You would be comparing guys who's careers were obviously limited by their injury with guys whos careers were limited by not being good enough to stay in  the league.  I don't see how that's valid.    The question is what effect this injury has on a players subsequent performance.  Nothing "happened" to your proposed control group.   If, by chance,  such a study showed both groups were out of the league after the same career length, you certainly couldn't conclude the injury had no effect on the career of the injured players.

 

Not necessarily. The idea is to match cases (injured) and controls (non-injured) on the features or variables you think could have an impact on the outcome (years in the league, starts, snaps, etc). If you do that, then there's no reason to believe the controls are more likely to be "guys whose careers were limited by not being good enough"—that variable should be equally distributed amongst cases and controls. If it's not, then the comparison is not valid, but as researchers you need to do what you can to make sure the cases and controls are as similar as possible.

 

Yes, nothing happened to the control group (ie, no injury)—that's the point. What is the impact of the injury on a player's performance? We can't know what their performance would have been without the injury, so we try to identify players as similar to the injured player as possible, but without the injury.

 

If the study showed that both groups were out of the league after the same period of time, and you did a good job matching the cases with similar controls, then yes, you could conclude that the injury had little effect on player longevity.

 

1 hour ago, Mr. WEO said:

It seems clear from all evidence that most of these guys don't come back to their pre injury level of play. 

 

This I agree with. The idea of the article is to quantify how much of an impact the injury has.

 

[NoSaint]

20 hours ago, gonzo1105 said:

And in this study, how many players who tore their ACLs were contributors, good players, guys who still would have been in the league 3 years later. 
 

Running backs were depicted as one of the worst positions for the study. Absolutely tough to come back from as an RB but we know how NFL teams value RBs in general. Oh he tore his ACL guess I’ll just draft another cheap option. 
 

I think more insight needs to be done on this type of study 

yea, I’d bet if you normalized out based on something like a scout rating (or admittedly imperfect PFF rating) and then even further for contract year - a whole lot of the variability gets accounted for.
 

I can’t think of a young pro bowler stopped in his tracks.
 

I bet most 3rd year depth players that haven’t proven their worth don’t get a vet contract after. But many don’t when healthy either. 

[Sharky7337]

20 hours ago, Buffalo_Stampede said:

Age is probably the biggest factor in returning to a high level of play.

 

That should have been part of the study. A 25/26 year old player vs a player 30+ years old. I’m sure there’s a difference.

 

For healthy RB’s you could probably find huge playing time and performance decreases even before 30 years old. 

This x1000.

 

The biggest downturn post ACL is usually older players.

 

Unless you whack the stem cells out of your children with a switch like adrian peterson.

 

 

 

[Mr. WEO]

38 minutes ago, Rubes said:

 

Not necessarily. The idea is to match cases (injured) and controls (non-injured) on the features or variables you think could have an impact on the outcome (years in the league, starts, snaps, etc). If you do that, then there's no reason to believe the controls are more likely to be "guys whose careers were limited by not being good enough"—that variable should be equally distributed amongst cases and controls. If it's not, then the comparison is not valid, but as researchers you need to do what you can to make sure the cases and controls are as similar as possible.

 

Yes, nothing happened to the control group (ie, no injury)—that's the point. What is the impact of the injury on a player's performance? We can't know what their performance would have been without the injury, so we try to identify players as similar to the injured player as possible, but without the injury.

 

If the study showed that both groups were out of the league after the same period of time, and you did a good job matching the cases with similar controls, then yes, you could conclude that the injury had little effect on player longevity.

 

 

This I agree with. The idea of the article is to quantify how much of an impact the injury has.

 

 

They did know what the player's performance was (no study can look at what it would have been) before the injury.  They compared it to the same player's performance after the injury.   Is your null that these ACL torn players would have had these drops in performance anyway...because so many guys wash out of the league anyway?  The data accounts for draft position and pre injury AVS.  One of the studies showed the higher drafted skill position players had the largest drop-off in performance.  

 

It's like any performance study.  Measure baseline performance at a complex task.  Perform an intervention.  Measure their performance at the same task after the intervention.  Did the intervention (injury in this case) affect performance.  It wouldn't make sense to compare too a control group without an intervention.

 

As a med student long ago (for $120), I volunteered for a study where I did puzzles and memory games, then was given IV benzos (or was it morphine) and had to do the games again (they were also doing MRI on my brain before and after.   They were looking at the effect on my performance...as well as on my MRI.  

 

Each player is their own control.  Their results then were split into position, draft point, Etc.   

[Rocky Landing]

This study seems somewhat anecdotal to me. There are too many variables from one player to the next to really apply this study to Tre White’s situation. 
 

That being said, from the “evidence” I have read regarding Tre’s injury, I would bet a month of my avatar that Tre will be active week one of the regular season. 

Edited April 7, 2022 by Rocky Landing

[Rubes]

13 minutes ago, Mr. WEO said:

It's like any performance study.  Measure baseline performance at a complex task.  Perform an intervention.  Measure their performance at the same task after the intervention.  Did the intervention (injury in this case) affect performance.  It wouldn't make sense to compare too a control group without an intervention.

 

As a med student long ago (for $120), I volunteered for a study where I did puzzles and memory games, then was given IV benzos (or was it morphine) and had to do the games again (they were also doing MRI on my brain before and after.   They were looking at the effect on my performance...as well as on my MRI.  

 

Each player is their own control.  Their results then were split into position, draft point, Etc.   

 

Not exactly. Using a person as their own control is an appropriate design if you think all of the external factors that could impact the outcome are the same before the exposure (injury) vs. after. If you're just measuring, for instance, speed or leg strength or something like that, then that's a reasonable thing to do. I think the point that many people are making here is that this is not the case—when players are injured and are lost for a year (more or less), there are other factors that can impact the outcomes of interest here: the number of starts a player has, the number of snaps they play, etc. Being injured and missing a lot or all of a season can result in other players taking over starting roles, teams deciding to move on to cheaper players, and so on. It may depend on age, on whether they were an entrenched starter or a backup, whether new draft picks have come along, and so on.

 

The purpose of including a control or comparator group is to make sure that the observations seen—a change in starts, a change in snaps, or other change in performance—is due specifically to the exposure (injury). If you do a study as you say to measure performance at the same task after an intervention, you can't really say for sure that the intervention is the cause of any changes seen (eg, differences could be due to various things that change over time). That's why you include a control group made up of similar people with the same features measured at the same time, with presumably the only difference being the absence of the intervention. Then you do things like measure the average change in the intervention group and compare that to the average change in the control group. The difference, presumably, is due to the main difference between the groups—the intervention.

 

Same thing with the benzo study. In order to know that the changes in memory observed were due to the benzos (and not to a placebo effect), they would need a separate but otherwise similar control group who is given an injection of a placebo. Then compare the average changes seen in the benzo group vs. the control group, with the difference (if any) thus attributable to the benzos. It's true for surgical trials, too—in some studies looking at the effect of a surgical intervention, they are often compared to a control group given a "sham" surgical procedure, since just the act of undergoing surgery could result in changes observed in the outcomes.

 

But the latter is addressing prospective randomized trial studies, the gold standard for evidence. What these guys did here is a retrospective observational study. In order to design an observational study to be as similar to a prospective randomized trial as possible, you do the work to choose a historical control group that is as similar to the intervention group as possible, and otherwise has (presumably) the same distribution of "unmeasured" variables. It's the analogy of randomizing in a controlled trial, the purpose of which is to try and ensure that the two comparison groups are identical other than the intervention.

 

[1ManRaid]

Worth noting that lower tier starters and backups are going to be less likely to receive patience and "no expenses spared" treatment from the teams than important All Pro talents like Tre White.  He's much less likely to be out of the league in 3 years or whatever like the study suggests.

[Rubes]

28 minutes ago, 1ManRaid said:

Worth noting that lower tier starters and backups are going to be less likely to receive patience and "no expenses spared" treatment from the teams than important All Pro talents like Tre White.  He's much less likely to be out of the league in 3 years or whatever like the study suggests.

 

Perhaps, but it more likely has to do with baseline performance level. Tre at baseline is a Pro Bowl-caliber player, one of the top at his position. If his performance declines a certain amount, say 10-15%, as a result of his injury and surgery, he'd still be better than most CBs out there and would have a job. By some measures of performance, however, he would be worse off than pre-injury, so it's not entirely about being in the league or not.

 

[Over 29 years of fanhood]

23 hours ago, Rubes said:

I'll admit that I've started believing that ACL injuries have become less of a concern in recent years, with players seemingly coming back to near pre-injury performance maybe the year after returning to play. Is that necessarily the case? Maybe not so much.

 

From the March 7th Orthopaedic Journal of Sports Medicine:

 

Return to Play and Performance After Anterior Cruciate Ligament Reconstruction in National Football League Players

(full article may be behind a paywall, but you can at least read the abstract)

 

 

 

 

 

Obviously this is a population-level study, and conclusions at the population level don't necessarily apply to a particular individual and his specific injury. Not all ACL injuries are the same, depending on the nature and extent of the ligament injury. It is also specific to positions: interestingly, WRs were similar to QBs in losing only about a quarter (18% and 17%, respectively) of their total snap count postinjury. So perhaps something to consider with regard to someone like Jameson Williams.

 

But still, this is a pretty sobering study, though I haven't gone through it in great detail to critique the methods. It's important to bear in mind that their study group was only up to 2018, so any improvements in ACL repair since then are not taken into consideration. Also of interest, they note that "NFL players have significantly shorter careers postoperatively (2.1 years) compared with players in other professional leagues such as the National Basketball Association (NBA; 4.5 years), National Hockey League (NHL; 4.5 years), and Major League Baseball (MLB; 2.9 years)."

 

Overall, ACL injuries are still bad news for NFL players who aren't QBs. Thought some of you might be interested to read that.

 

What percentage of players are in the league 3 years after being drafted? 

[Locomark]

Well Tom Brady had an ACL,PCL, and MCL all tear in 2007 (aka the terrible triad) and he is playing 15 years later so I would say it all depends on whether you can a) play with a brace and b) commit constantly to having strong leg muscles surrounding that injured area. Basketball, tennis, and football are tough sports for that ACL because of all the pivoting where your foot can get stuck planted and that same knee can get twisted or stepped on again. I worry about Tre because a CB isn’t in control of where the play will go so that is harder on the knees IMO. I had triad surgery and it took a long time to trust it but before it all went I only had an ACL tear and I was still able to run and even push weight with the leg. So for a solid pro, an ACL isn’t a career ender at all. Reality is someone like Tre could also end up being moved to safety in the future if his change of direction is impacted.   To some of the prior posts, this isn’t the 80’s or 90’s where tears often ended careers. The world of medicine is way different now. They have much better techniques. 

[Mr. WEO]

10 hours ago, Rubes said:

 

Not exactly. Using a person as their own control is an appropriate design if you think all of the external factors that could impact the outcome are the same before the exposure (injury) vs. after. If you're just measuring, for instance, speed or leg strength or something like that, then that's a reasonable thing to do. I think the point that many people are making here is that this is not the case—when players are injured and are lost for a year (more or less), there are other factors that can impact the outcomes of interest here: the number of starts a player has, the number of snaps they play, etc. Being injured and missing a lot or all of a season can result in other players taking over starting roles, teams deciding to move on to cheaper players, and so on. It may depend on age, on whether they were an entrenched starter or a backup, whether new draft picks have come along, and so on.

 

The purpose of including a control or comparator group is to make sure that the observations seen—a change in starts, a change in snaps, or other change in performance—is due specifically to the exposure (injury). If you do a study as you say to measure performance at the same task after an intervention, you can't really say for sure that the intervention is the cause of any changes seen (eg, differences could be due to various things that change over time). That's why you include a control group made up of similar people with the same features measured at the same time, with presumably the only difference being the absence of the intervention. Then you do things like measure the average change in the intervention group and compare that to the average change in the control group. The difference, presumably, is due to the main difference between the groups—the intervention.

 

Same thing with the benzo study. In order to know that the changes in memory observed were due to the benzos (and not to a placebo effect), they would need a separate but otherwise similar control group who is given an injection of a placebo. Then compare the average changes seen in the benzo group vs. the control group, with the difference (if any) thus attributable to the benzos. It's true for surgical trials, too—in some studies looking at the effect of a surgical intervention, they are often compared to a control group given a "sham" surgical procedure, since just the act of undergoing surgery could result in changes observed in the outcomes.

 

But the latter is addressing prospective randomized trial studies, the gold standard for evidence. What these guys did here is a retrospective observational study. In order to design an observational study to be as similar to a prospective randomized trial as possible, you do the work to choose a historical control group that is as similar to the intervention group as possible, and otherwise has (presumably) the same distribution of "unmeasured" variables. It's the analogy of randomizing in a controlled trial, the purpose of which is to try and ensure that the two comparison groups are identical other than the intervention.

 

 

Sham surgical trials aren't done for obvious ethical reasons, for the most part.

 

The benzo test wasn't asking how they affect performance of the skills games, but the effect on the games as it relates to the concurrent MRI images.  Comparing to a cohort not given benzos would have not been meaningful, as that was also me before I was drugged.

 

Certainly being injured leads to missed games and losing starting roles and teams moving on.  That would be reflected in the decreased performance metrics they listed.  The inference is that, if the player isn't back to preinjury performance level, the team will likely move on.  

 

But let's say you matched with an uninjured cohort by age, position and number of snaps before injury.  The null hypothesis is that the dropoff in performance for the injured player is no different than the noninjured controls?  That the injured players were as likely to have a dropoff if they were not injured?

 

While randomized controlled studies are "gold standard", not all queries lend themselves to this type of study.  And certainly uncontrolled "before and after" studies are well known and accepted in the medical literature.  Faulting the editorial review of this paper by this journal (some have here) therefore, is not appropriate.  They understand the limitations of such a study--but they obviously see it as valid and congruent with other published similar studies on the same topic (referenced)

[Orlando Tim]

As pointed out a few times the study would be more useful if it delineated between a few factors such as age, starter vs backup, and position. These three elements could make all the difference.

[Rubes]

2 hours ago, Mr. WEO said:

 

Sham surgical trials aren't done for obvious ethical reasons, for the most part.

 

The benzo test wasn't asking how they affect performance of the skills games, but the effect on the games as it relates to the concurrent MRI images.  Comparing to a cohort not given benzos would have not been meaningful, as that was also me before I was drugged.

 

Certainly being injured leads to missed games and losing starting roles and teams moving on.  That would be reflected in the decreased performance metrics they listed.  The inference is that, if the player isn't back to preinjury performance level, the team will likely move on.  

 

But let's say you matched with an uninjured cohort by age, position and number of snaps before injury.  The null hypothesis is that the dropoff in performance for the injured player is no different than the noninjured controls?  That the injured players were as likely to have a dropoff if they were not injured?

 

While randomized controlled studies are "gold standard", not all queries lend themselves to this type of study.  And certainly uncontrolled "before and after" studies are well known and accepted in the medical literature.  Faulting the editorial review of this paper by this journal (some have here) therefore, is not appropriate.  They understand the limitations of such a study--but they obviously see it as valid and congruent with other published similar studies on the same topic (referenced)

 

Sham surgeries are most definitely performed, on animals for animal studies, which of course are relevant for our understanding of similar scientific questions in humans. The sham surgeries are done because of the reasons I stated.

 

Of course, not all queries lend themselves to a randomized trial. You can't do a prospective randomized trial of ACL injuries, for instance. Studies like that are best done as controlled observational studies the way I described. You can certainly do an uncontrolled before-after study, and lots of people publish those, but by no means are those studies considered to be high quality evidence. The main criticism of an uncontrolled before-after study is that the results are untrustworthy—you have no idea if the observed effects are truly significant or not. In many cases it's very difficult to identify a control group for a before-after study, and that's okay, you can't always have what you want. But by accepting that and publishing an uncontrolled before-after study, you're basically admitting that your results, while interesting, may or may not have real-world significance.

 

You choose a study design based on the question you're trying to answer. If the question is: what is the impact of an ACL injury on an NFL player's career? then you know you'll be doing an observational study, but the real question you're trying to answer is: how does what happened to those injured players compare to what would have happened if they had never been injured? Since you can't do that directly, you do the next best thing—compare what happened to those injured players to what happened to a similar group of non-injured players.

 

Imagine if the main outcome you were trying to test is a player's maximum running speed. So the main question is: what is the impact of an ACL injury and repair on a player's maximum running speed? Let's say you have all of the data on player's maximum speeds from the NFL combine, and now you identify players who had ACL injuries during their NFL career, so you test them again for their max running speed. You could just do a simple before-after study and compare their running speeds now vs. their running speeds then, and you'd probably see a decent difference. You could, for instance, say that those with an ACL injury saw an average loss of 1MPH in their max running speed. Is that a valid conclusion? Not really.

 

Of course, the reason is that everyone slows down as they age, so what you'd really want to do is compare the average speed loss in those who are injured with the average speed loss in those who were never injured. Then you'd know the impact of the ACL injury on max running speed. You need the control group to know how significant the loss of speed observed is.

 

And yes, the null hypothesis would be that the ACL injury had no effect on max running speed. And that may very well be the case. Or, there may be an effect, but it's not statistically significant. Or, the loss may be statistically significant, but it may not be significant in the real world. For instance, you could show a statistically significant loss of 0.1MPH to their speed, but how that impacts play in real games may not be a big deal.

 

But you still need the control group to understand whether the observed differences pre- and post-injury are statistically above and beyond what you would have seen, on average, in the absence of the injury.

 

[Mr. WEO]

17 minutes ago, Rubes said:

 

Sham surgeries are most definitely performed, on animals for animal studies, which of course are relevant for our understanding of similar scientific questions in humans. The sham surgeries are done because of the reasons I stated.

 

Of course, not all queries lend themselves to a randomized trial. You can't do a prospective randomized trial of ACL injuries, for instance. Studies like that are best done as controlled observational studies the way I described. You can certainly do an uncontrolled before-after study, and lots of people publish those, but by no means are those studies considered to be high quality evidence. The main criticism of an uncontrolled before-after study is that the results are untrustworthy—you have no idea if the observed effects are truly significant or not. In many cases it's very difficult to identify a control group for a before-after study, and that's okay, you can't always have what you want. But by accepting that and publishing an uncontrolled before-after study, you're basically admitting that your results, while interesting, may or may not have real-world significance.

 

You choose a study design based on the question you're trying to answer. If the question is: what is the impact of an ACL injury on an NFL player's career? then you know you'll be doing an observational study, but the real question you're trying to answer is: how does what happened to those injured players compare to what would have happened if they had never been injured? Since you can't do that directly, you do the next best thing—compare what happened to those injured players to what happened to a similar group of non-injured players.

 

Imagine if the main outcome you were trying to test is a player's maximum running speed. So the main question is: what is the impact of an ACL injury and repair on a player's maximum running speed? Let's say you have all of the data on player's maximum speeds from the NFL combine, and now you identify players who had ACL injuries during their NFL career, so you test them again for their max running speed. You could just do a simple before-after study and compare their running speeds now vs. their running speeds then, and you'd probably see a decent difference. You could, for instance, say that those with an ACL injury saw an average loss of 1MPH in their max running speed. Is that a valid conclusion? Not really.

 

Of course, the reason is that everyone slows down as they age, so what you'd really want to do is compare the average speed loss in those who are injured with the average speed loss in those who were never injured. Then you'd know the impact of the ACL injury on max running speed. You need the control group to know how significant the loss of speed observed is.

 

And yes, the null hypothesis would be that the ACL injury had no effect on max running speed. And that may very well be the case. Or, there may be an effect, but it's not statistically significant. Or, the loss may be statistically significant, but it may not be significant in the real world. For instance, you could show a statistically significant loss of 0.1MPH to their speed, but how that impacts play in real games may not be a big deal.

 

But you still need the control group to understand whether the observed differences pre- and post-injury are statistically above and beyond what you would have seen, on average, in the absence of the injury.

 

 

I wouldn't have thought you referring to  animal studies.

 

 

You state that non controlled studies results "may or may not have real world significance" and yet also state that control matched studies with statistically significant results which "may also not be significant in the real world".  This is the basic truth of most of what is published all the time.  It doesn't make them untrustworthy results.

[Rubes]

15 minutes ago, Mr. WEO said:

You state that non controlled studies results "may or may not have real world significance" and yet also state that control matched studies with statistically significant results which "may also not be significant in the real world".  This is the basic truth of most of what is published all the time.  It doesn't make them untrustworthy results.

 

Poorly worded on my part. Non-controlled studies may produce results of undetermined meaning—you may show a statistical difference before and after the intervention, but without a control group you can't say if that result would have happened anyway, in the absence of the intervention.

 

Controlled studies can produce results with more confidence and meaning in their statistical significance—but the statistically significant value may not have much meaning in the real world, like if the study of max running speed had a large enough sample size to detect a tiny difference of 0.01MPH. You could conclude that ACLs had a significant impact on speed, but most people would look at 0.01MPH and say, "Who cares?"

 

The former is less trustworthy, because you can't draw a valid conclusion with much confidence. The latter is more trustworthy, because it was designed well, even though the more valid conclusion may not ultimately mean much.

 

[BADOLBILZ]

On 4/6/2022 at 12:37 PM, FireChans said:

Most players are probably out of the league after an ACL because teams don’t want to keep a roster spot for a rehabbing player who isn’t an impact player.

 

If Dane Jackson tore his ACL instead of Tre, instead of “keeping his roster spot warm for him” we would just find a replacement player.

 

 

Agree...........Justin Zimmer is a good example of this...........he was a playmaker and seemed like he had a good chance of being in Buffalo for a couple more seasons.  

 

Injured his knee.......

 

[wppete]

If Jameson Williams is still there at #25 we run to the podium.

 

 

[JakeFrommStateFarm]

What if Jameson Williams loses 3/10s of a second on his 40 ?

[In Summary]

So I ask,

1. Will Tre play by October, and if so, what is his performance ceiling this year - 90% of last year?
2. Is drafting Jameson Williams worth the risk?  Does he fully recover and do we need him playing early this year as much as next?

With Diggs locked up, and a need for a good outside #2-3 WR, I'd say the Bills lean towards a WR who can play early this year rather than go with Williams for the future - especially with this appearing to be an "all in" year.

[Mr. WEO]

3 hours ago, BADOLBILZ said:

 

 

Agree...........Justin Zimmer is a good example of this...........he was a playmaker and seemed like he had a good chance of being in Buffalo for a couple more seasons.  

 

Injured his knee.......

 

 

Hard to say if they would have kept him.  He played on only 1/3 of the snaps in the 12 appearances in 2020.  In the 6 games before he was injured last season he didn't do much.  He went on IR so no one knows what they would have done.  He has no post-injury data. He's a free agent. 

[FireChans]

23 hours ago, Mr. WEO said:

 

Hard to say if they would have kept him.  He played on only 1/3 of the snaps in the 12 appearances in 2020.  In the 6 games before he was injured last season he didn't do much.  He went on IR so no one knows what they would have done.  He has no post-injury data. He's a free agent. 

I think the point is he may have gotten some kind of a contract here or elsewhere had he not been injured. 
 

Instead, his entire career is on life support. The impact of the injury on performance doesn’t even matter.
 

 

[Mr. WEO]

13 minutes ago, FireChans said:

I think the point is he may have gotten some kind of a contract here or elsewhere had he not been injured. 
 

Instead, his entire career is on life support. The impact of the injury on performance doesn’t even matter.
 

 

 

Hard to say-pretty fringe player.  He wouldn't have been included in the study cited by the OP.

[FireChans]

8 minutes ago, Mr. WEO said:

 

Hard to say-pretty fringe player.  He wouldn't have been included in the study cited by the OP.

No he wouldn't but he would be included in this part:

 

"Only 55.4% (n = 173/312) of players returned to play after ACLR."

 

I wouldn't be surprised if the 45% of players who didn't return were fringe players a la Zimmer. Best ability is availability and all that. And if a player like Zimmer was re-signed, we would probably be bringing in replacement level players who could easily take his job.

 

Put another way, if Zack Moss tore his ACL, we would likely sign another RB who would get his snaps.  If that RB was even mediocre, he would probably result in Moss losing snaps and AV. It's no surprise that positions that have a lot of "filler players" like RB, DL and LB experience the most drop off after injury.  There's a million of STers and 4th DLmen in the NFL.  Not so many QB's,

Edited April 8, 2022 by FireChans

[Mr. WEO]

2 hours ago, FireChans said:

No he wouldn't but he would be included in this part:

 

"Only 55.4% (n = 173/312) of players returned to play after ACLR."

 

I wouldn't be surprised if the 45% of players who didn't return were fringe players a la Zimmer. Best ability is availability and all that. And if a player like Zimmer was re-signed, we would probably be bringing in replacement level players who could easily take his job.

 

Put another way, if Zack Moss tore his ACL, we would likely sign another RB who would get his snaps.  If that RB was even mediocre, he would probably result in Moss losing snaps and AV. It's no surprise that positions that have a lot of "filler players" like RB, DL and LB experience the most drop off after injury.  There's a million of STers and 4th DLmen in the NFL.  Not so many QB's,

 

 

59 of the original 135 included in the study were still in the league at least 3 years after the injury.   The positions with the highest % of post injury players in that group were QB, OL, FS, DE.

 

Another study (cited in this one) showed skill players drafted in highest rounds had the worst outcomes after injury.

 

 

All of the limitations of this study brought up in this thread were clearly discussed by the authors in this paper.