Orton's Arm

An excellent post. There are a lot of young players on the team who, for whatever reason, have had little or no playing time. The Bills need to play those guys against the Ravens. That way, we'll have a better idea as to what our needs are in free agency and the draft.

I agree it's gotten out of hand. What gets under my skin is that some people make jokes about regression toward the mean despite the fact that a) those jokes have been told dozens of times before, b) schools such as Stanford, Berkeley, and the University of Chicago have indicated that those who obtain extreme scores on imperfect tests tend to score closer to the population's mean upon being retested. The utter stupidity of those regression toward the mean jokes gets under my skin maybe a little more than it should.

Yes, it's ridiculous that the town of Hazelton actually cares about upholding U.S. immigration law. What's this world coming to?

Your post is wrong from start to finish. For one thing, I cannot recollect a single instance in this discussion when you've done any math. Not once. Secondly, I have done math, both by setting up my Monte Carlo simulation, as well as in the examples I've sometimes given. Third, you've given contradictory answers to the question I've asked, which is why I'm asking it again. When I first announced that people who get high I.Q. scores tend to score somewhat lower upon being retested, you ridiculed both the phenomenon itself, and me for proposing it. Later, you told Wraith that you weren't disputing the existence of this phenomenon. So I want you to come out and give a definitive statement about what you do or don't believe regarding this phenomenon. I've asked you for clarity on this issue before, only to have you dodge the question.

The math is actually fairly simple and straightforward. But I can see why you'd want to make it seem harder than it really is as an excuse to once again dodge my question.

The one topic I've repeatedly asked for specifics on is one which you've consistently avoided. Suppose you have a population that's normally distributed with respect to I.Q., and an I.Q. test which has measurement error. The measurement error is normally distributed with a mean of zero. Suppose you were to give everyone in the population an I.Q. test; and were to retest those who obtained, say, a 140 the first time they took the test. Would people retaking the test tend to score closer to the population's mean upon being retested?

Once again, you're trying to convince people I must have somehow misunderstood something or another, while providing no specifics on what this something might actually be. Quite frankly, if you try to provide specifics, you'll once again fall on your face. I don't blame you for remaining intentionally vague. As far as the math angle goes, I've provided a lot more math to this discussion than you. You just didn't understand it.

It sounds like you've got a firm grasp of this issue. You're right to say there are circumstances where measurement error would not be associated with regression towards the population's mean. But the example that started the whole debate was an example of I.Q. test scores. I argued that the average person who scores a 140 on an I.Q. test will, on average, obtain a somewhat lower score upon being retested. Bungee Jumper called me an idiot who doesn't understand the first thing about statistics, and the debate was on. Happily, I've found sources like Stanford, the University of Chicago, and others which state that those who obtain very high or very low scores on their first tests tend to regress somewhat towards the population's mean upon being retested. An intelligent, unbiased person paying attention to this debate from start to finish would clearly realize I've long since won. Bungee Jumper's notion of "regression toward the mean of error" can seem seductive. It's true that whether a person gets lucky or unlucky on the first test, that person is expected to be luck-neutral upon being retested. However, the group of people who scored above the population mean on the first test will typically contain more lucky people than unlucky. Likewise, the group of people who scored below the population's mean will typically contain more unlucky people than lucky people. (I'm assuming a normally distributed population, and a normally distributed measurement error term.) If you were to retest everyone who scored above the population's mean, or any given subset of that group, you would find that those whom you retested would see their socres regress somewhat toward the population's mean. It's the population mean that's being referred to in the phrase, "regression toward the mean," and not the mean of the error.

I've been trying to explain math to a pinhead for at least the last 50 pages. It's been less than fun. You're right in saying that "inexactness" has led to this discussion becoming exceptionally long. To say," you're too stupid to know the difference between error and variance," translates into, "I think the difference between error and variance somehow undermines some portion of what you're saying. But because I'm an inconsiderate jerk who isn't afraid of a little board pollution, I won't bother giving specifics about which portions of Arm's examples are supposed to be incorrect. In fact, specifics aren't that important to me anyway. If I say that Arm is an idiot who doesn't understand statistics often enough, people will believe me. This, even if they don't understand the underlying debate. Especially if they don't understand the underlying debate." The bottom line is that you've mocked me for saying the same thing people at Stanford and the University of Chicago are saying. This is a reflection on your credibility, not mine.

Spikes probably will never be the same player he once was. That said, the Bills have plenty of salary cap space going into next year. There's no particular reason to mess with his contract right now. If he hasn't returned to form by the end of next season, then they should think about upgrading his position.

It looks like we're in agreement on this one.

No, not always. But under normal, real life circumstances, an error-prone test is generally associated with regression toward the population's mean in test/retest situations.

In real world examples, (that is, normally distributed population, normally distributed measurement error with a mean of zero) regression toward the mean applies to individuals in a certain sense. That is, if someone got a 140 on an I.Q. test, that person is more likely to get a lower score the second time around, than he or she is to get the same score or better. This doesn't mean that all the people who got 140s the first time around will get lower scores upon being retested, just that most will. To address your example, the most likely outcome for the person's retest is 140, because that is this person's true I.Q. If you're saying that the test only allows someone to score a 120 or 160; then either outcome is equally likely both for the initial test score and for the retest. I can't really answer your second question without knowing more. Assuming the person to which you're referring has a true I.Q. of 140; that person's expected value on the retest is 140 (50% chance of getting a 135; 50% chance of getting 145). But based on the way the I.Q. test is set up, it's impossible for any specific person to do anything other than score exactly five points away from the population's mean I.Q. of 140.

Thanks for that "expected value" link. Here's an interesting quote from it: To return to the discussion at hand: clearly in your example, it's impossible for any one person to regress towards the population's mean. But as a group, those who score high or low on the first test regress toward the mean on being retested.

Reread what I wrote about his example. I said that someone who scored a 145 on the first test would, on average, score a 140 the second time around. Obviously, 50% of the time those retaking the test would get a 145 again, and the other 50% they'd get a 135. That works out to an average outcome of 140.

The regression toward the mean thread is a cancer that nearly conquered the PPP board, and seems to be spreading onto the main board. Hopefully future debate will remain confined to the regression toward the mean thread. (BTW, that thread's now at 21 pages.) In any case, your explanation as to which team to root for in the Denver/Bengals game was an excellent one. While I don't have complete confidence in the Dolphins' ability to beat the Jets, I think a Dolphins win is more likely than some other scenarios might be. So I'll still be rooting for Denver. Not that it matters a whole lot--it's not like the Broncos are going to read this thread, see there's one extra guy rooting for them, and decide to play that much harder.

You pretty much took the words out of my mouth. If you select, say, 100 people who scored a 145 the first time around, that group's average on being retested will be around 140.

In his example, the people being tested had an average true I.Q. of 140. Hence, someone who scored above or below a 140 is, on average, expected to regress towards the population's mean I.Q. upon being retested.

Yes, that's about the shape of things.

Amen to that. Countering Soviet expansionism simply wasn't a priority for FDR.

That's your opinion, and you're welcome to it. Others see the issue differently: Hmmm . . . should I take the word of jzmack, or should I take the combined word of Stanford, Tufts, Berkeley, the Environmental Protection Agency, the University of Chicago, and the University of Washington? Hmmm, let me think about this . . .

Under the circumstances you've outlined, someone who scored a 135 on the first test would, on average, score a 140 on retaking the test. Someone who scored a 145 on the first test would score a 140 upon being retested. In both cases, people who scored above or below the population's mean I.Q. of 140 are expected to regress toward the population's mean upon being retested.

You seem to be describing the following: 1. Every member of the population has a real I.Q. of 140. 2. Due to a bias in the I.Q. test, the average person taking the test scores 5 points too high. 3. Therefore, the apparent population mean is 145, and not the real 140. I'll agree that under these circumstances, a test/retest situation would cause I.Q. scores to regress towards the population's measured mean of 145, and not its true mean of 140. In this case, you seem to be describing a situation where every member of the population has a true I.Q. of 140. However, due to measurement error, 10% of the population scored a 137 on the relevant I.Q. test, 20% scored a 138, etc. Assuming this interpretation of your example is correct, those who scored a 137 on the test the first time around should expect to score a 140 on retaking the test. (This is an average expectation, as 10% of them will score a 137, 20% a 138, etc.) As a group, those who scored a 137 the first time around will regress toward the population's mean I.Q. of 140. Look at those who scored a 143 on that I.Q. test. The mean I.Q. for that group is also 140 (if I understand your example correctly). On being retested, 10% of them will score a 137, 20% of them will score a 138, etc. As a group, those who scored 143 the first time around will, on average, regress towards the population's mean I.Q. of 140 upon being retested.

Wrong. If you look at those who scored, say, a 140 on an I.Q. test, they're going to be more lucky than average. Retest that particular group, and on average they'll score closer to the population's mean I.Q. Conversely, the group of people who scored 60s on the I.Q. test will contain more people who got unlucky than people who got lucky. Retest the 60s; and they'll score closer to the population's mean I.Q the second time around.

There's more to it than that. Suppose someone was to flip a coin 100 times. You have a group of people trying to predict the outcome of that coin flip. Let's say the most accurate person in the group was able to predict it successfully 65% of the time. This person is now asked to predict the outcome of another 100 coin flips. Odds are this person will only succeed 50% of the time. That extra 15% of success was due entirely to luck; and that luck is expected to disappear when the person gets retested. Now consider something that's based on something innate--such as height. Height measurements aren't expected to change from one trial to the next. Suppose you're dealing with something that's half innate, and half luck. You take the people who had above-average scores. The typical person who had an above-average score obtained half of his or her success to innate ability, and the other half was through luck. When these people are retested, the good luck is expected to go away, but the innate stuff is expected to remain. So they're expected to move halfway toward the population's mean. To, um, relate this to sports, consider a group of highly successful rookies. In selecting that group, you selected players who (on average) were disproportionately lucky. The lucky ones should expect that luck to disappear in their second years, leading to the so-called "sophomore jinx."