Orton's Arm

I just went back through the play-by-plays from 2005, to look at the drives some people want to eliminate from Losman's second stint by using a 15 or 20 yard threshold. Below are the descriptions of the drives. I'll leave it to the reader to decide whether Losman did enough on those drives to deserve some credit for the points they produced. I personally feel he did. Drive 1 (against San Diego): This drive started on the San Diego 47, and moved to the San Diego 35 for a FG. Losman passed for 11 yards, McGahee rushed for 6, and the Bills lost 5 due to a penalty. Drive 2 (against San Diego): Losman completed a 9 yard pass to Reed on 2nd and 10. He completed a 3 yard TD pass to Shelton on 1st and goal from the 3. He also had a 30 yard run. Drive 3 (against Miami): This drive started on the Miami 26, so I deducted 3 points because the Bills were already in FG range. McGahee had 9 rushing yards to Losman's 17 passing yards, including Losman's 4 yard TD pass to Evans. Using a 20 yard threshold, Losman gets no credit for this particular drive. People are accusing me of dishonesty and manipulating statistics because I gave Losman credit for those three drives.

It's funny you should bring up my Tony Romo comment the day after he went 24/36, for 270 yards, 1 TD and 1 INT. In last week's game, I saw a player who could seriously contribute to his team if he could cut back on his interceptions. That's what Romo did last night. As for the Losman stats, I had to draw the line somewhere. I made the ten yard decision before looking at any data at all; basically for the same reason you decide an appropriate alpha level before looking at any data. I chose a 10 yard cutoff because I felt that getting enough yards to move the chains constituted a meaningful contribution to a drive. Should I have tested the data at 15 and 20 yard cutoff points? In a PhD program, yes. For a paper intended to be published, yes. But for a discussion board post?

And your reason for keeping this flame war open is . . . ?

I'd welcome a law like that, if applied properly. But there's danger here. For example, take some deeply controversial issue, such as the conflict between the Israelis and the Palestinians. If the issue is controversial enough, people are often willing to distort things to get their way. Suppose, therefore, that a politician makes an assertion about that conflict, which he honestly believes is true. Suppose further that those on the other side of the issue choose to call him a liar. They'd take him to court, and they'd do what they could to blur the line between their opinions, and actual facts. If you were just looking at actual facts, you'd probably see that there's no way of knowing whether the politician's assertion was true. But if you look at the accusations leveled against him by his ideological opponents, he'd seem to be one of the worst liars in the world. Whether the politician gets punished will ultimately be left up to a jury, which is little better than leaving it up to random chance. (Witness the O.J. trial.) You'd have to put controls on the system to make sure it wasn't used to stifle legitimate intellectual debate. You'd also have to make sure those who distorted facts in an effort to incriminate a politician were themselves subjected to very harsh penalties. But I agree with the implication of your post: namely, that the abundance of lying that takes place in politics indicates that the rewards for telling a lie are often too high, and the penalties for getting caught are too low.

Looks like this thread is still a flame war. Before I once again shut this thing down, I want to thank everyone who contributed to the discussion about statistics. A number of intelligent observations were made. It's too bad this meaningful discussion got crowded out by the other stuff.

Wow! You've actually written a complete sentence without mentioning the casting for America's Stupidest Woman. Color me impressed.

Are you always this big a jerk? Because it's really annoying when people make false assumptions about my actions. If you care to find out whether there's any truth to your assumptions, go back through my older posts about Losman. You'll see that the first time I used the adjusted points per game method, I was comparing Losman's second stint to Holcomb's performance in 2005. My assertion was that even in his second stint, Losman didn't do as well as Holcomb. Oh, and by the way, I used a 10 yard threshold. Had I used a 15 or 20 yard threshold, people would have accused me of manipulating the system to throw out too many of Losman's contributions from his second stint. The Senator wrote that my adjusted points per game system was too confusing, and on that basis accused me of manipulating stats to make Losman's second stint seem worse than it actually was. Now I'm once again being accused of manipulating stats, but this time to show that Losman's second stint was better than it actually was. And it's been the same system both times.

Prove? Prove! You're actually trying to suggest that a 7% increase in adjusted PPG (at a 15 yard threshold) proves that Losman is doing better this year than in his second stint last year? Shouldn't a stats guy like you be a little more cautious when using words like "proof"?

If your FG kicker goes 6/6 and your team loses, do you bench him? If your DE gets four sacks and a bunch of tackles and your team loses, do you bench the DE? I for one believe a QB should be evaluated based on his own contributions, and not on whether the team wins or loses.

You may well be right about this year's record. On the other hand, you could make the case this year's record doesn't matter much, because this year's Bills team isn't destined for the Super Bowl anyway. In building for the future, would Leinart or Cutler represent a significant upgrade over Losman? It's way too early to tell, but fun to speculate.

This thread is no longer about statistics. It's degenerated into a flame war; so I'm closing it.

And your evidence for this is what exactly? The fact that you used a lame excuse to chicken out of answering the question? Or is it the fact that when you did participate, the objections you raised demonstrated your own shortcomings? I wasn't the one who couldn't predict the result of a fairly simple Monte Carlo simulation.

To those who aren't familiar with what syhuang is bringing up, the argument in question is one about whether Losman has played better this year than in his second stint last year. He embarrassed himself with comments like Syhuang apparently thinks comparing Losman's numbers from last year with those from this year is a ridiculous way of determining whether Losman has improved! Not content with having put his foot in his mouth, he proceeded to shove in the whole leg: Other QBs' stats are irrelevant to determining whether Losman has improved, or whether his improvement is statistically significant. So syhuang, what on earth was going through your mind when you provided the link to that thread?

I'll give you that the offensive line played pretty well for the second half of 2004. In the first half it was . . . well, let's just call it a work in progress. I do take issue with your statement that "most of those years we've had at least an average OL." Take Fina for example. I remember hearing about how the Bills used a rating service for its own players. Fina's rating was mentioned, which I think was like 20th or 24th best LT in the league. Somewhere in there. Maybe he was average or better in his prime, but that was a long time ago. You mention Brown, and that's fair enough. But he was only one guy. If you look at the Bills' line from center on over, you had players like Ostrosky, Lacina, Nails, Louchiey. You had them starting. So the Bills' line generally had a below average LT, a good RG, and lots of backup-quality players and turnover at the other three positions. Not exactly a recipe for success.

Hilarious. Great find.

I wasn't trying to say that most Losman supporters are like this. And I suppose it's easy to overestimate the number of unreasonable Losman supporters, because they're often the ones who post the most.

When you were coming up with this stuff, didn't any warning bells go off in your head? That maybe, just maybe, this might not be a good idea? It wasn't. I've been very patient with your unfounded insults and condescending tone. But there are limits to my patience, which you've managed to exceed. Congratulations. I didn't want to do this to you, but you've changed my mind. The above post demonstrates your own lack of awareness of statistics. Had you been more familiar with the topic, you'd know there are several types of variables: metric, ordinal, categorical and binary (with binary being a subset of categorical). In this case, we're dealing with an ordinal variable (the 1 - 5 scale) and a categorical/binary variable (interested/uninterested). Based on the wording of your objection, you don't believe a dependent categorical variable (interested/uninterested) can be driven by an independent ordinal variable (the 1 - 5 scale). Or maybe you were trying to say that interested/uninterested was the independent variable, and the 1-5 scale was the dependent variable. But it doesn't matter which variable you thought was dependent, and which independent. Your objection is wrong, and wrong in a way which indicates you are unfamiliar with this portion of statistics. Variants of logistical regression analysis can be used when your dependent variable is categorical, and your independent variables metric or ordinal. Conjoint analysis can be used when your dependent variable is ordinal, and your independent variables are categorical. If you care to find out "what the [expletive deleted] kind of result" such tests would give, I suggest you read up on them. I don't like having to attack you in this way, because I honestly believe you're a smart guy who generally knows what he's talking about. Generally. But not in this case. If I was playing by your rules, I'd use the above example to try to say you don't know anything about statistics. But that would be an unfounded accusation, just as your statements about my knowledge of statistics are unfounded. Let's show a little maturity here, bury the hatchet, and confine any future criticism to ideas. There's no need to make things personal. Oh, and to answer your question, I wasn't the one who made up the survey.

My post did, in fact, say that we need to look at whatever Losman's done by the end of this season in deciding whether to take a QB in the draft. Mostly, I was addressing the tactic of some Losman supporters to say things like "You're a moron. Can't you see how bad the offensive line is? Why on earth do you think the quarterback is the problem with this offense when the line can't run block or pass protect?" My response to this is to say, yes, I see the Bills have a bad line. We've had a bad line for about the last ten years. We've also had bad QBs for years: Todd Collins, Billy Joe Hobart, Rob Johnson, Alex Van Pelt, Drew Bledsoe, and now Losman. Will Losman become more than a bad QB by the end of the season? I guess we'll see.

I've taken enough statistics classes, at a vigorous enough level, to have a solid understanding of the fundamentals. I'm not trying to teach you, or Ramius, or Coli, anything about the subject. I'll agree that method 2 was flawed in the sense Ramius pointed out--in a perfect world, you'd like to throw more money at the problem. But given the constraints imposed by the survey-only nature of the chess club's budget, and given the additional constraints imposed by the flawed design of the survey, method 2 is probably about as good as you can make things. If you have a better way of salvaging something from the survey data, I encourage you to share it.

The professor I mentioned earlier. Only he told me the wrong answer. I had to figure out the right answer by myself.

This thread's been open for 24 hours now, so it's time to give out the correct answer. Thanks to everyone who's contributed their insight into the question at hand. What makes this problem interesting is the method of getting the correct answer. Suppose you had a calculator, and you wanted to see if it worked. You could ask it a question to which you already know the answer. If a calculator tells you 2+2=5, you shouldn't trust it to add larger numbers. So let's ask methods 1 and 2 a question to which we already know the answer, to see what they tell us. Suppose the chess club were to offer people $1 million for each year of membership. We can feel safe in assuming this would be a very strong inducement for people to join. If a given method tells us a $1 million check would be a weak inducement to join, we can conclude the method is flawed. How would the two methods interpret the $1 million question? Method 1 looks at differences in preferences between Interested and Uninterested. In the case of a $1 million check, both groups' preference would be very close to 5. Because there would be little or no statistically significant difference between the two groups' preference for $1 million, method 1 would tell us that offering a $1 million incentive wouldn't significantly affect someone's decision to join the chess club. In other words, it's telling us that 2+2=5. Method 2, on the other hand, would correctly interpret the data from the hypothetical question. Method 2 is clearly superior to method 1. Method 2 assumes that a significant percentage of the people who seriously considered joining the chess club did not in fact do so. It further assumes the chess club should be focusing the most attention on those who were interested but didn't join. If, instead, one were to assume that nearly all the people who seriously considered joining the chess club went on to do so, you'd need to make the changes to method 2 that some of the posters have suggested. The need to assume things one way or the other could have been avoided through a better-designed survey.

My next question is this: how current is the 2006 chart? If you were to exclude his last game or two, the 2006 numbers would probably look a lot better.

Are the numbers you cited for Losman's performance in 2005 for the whole year, or just for his second stint? I'm perfectly willing to agree that he's playing better than he did in his first stint of 2005. If you've got first stint 2005 data lumped into the totals you cited, he'll obviously look better in 2006. But the real question is: is he playing better now than he was in his second stint of 2005?

This is about how many points Losman helped the offense score in an average game last year, versus how many he helped it score in an average game this year. In other words, we're dealing with two measured means, and we're testing to see whether the difference in those two measured means is statistically significant. The correct tool to test for statistical significance in differences between measured means is the t-distribution.

The question was whether Losman's 7% improvement in adjusted points per game at a 15 yard threshold is statistically significant. Bringing the results of other players into the picture won't help answer that question, because the question is about whether what Losman is doing this year is better than what he was doing last year. The fact of the matter is that the aforementioned 7% improvement is not statistically significant. But you can't read too much into that either. It's possible that Losman has in fact made improvements, but that these improvements have been masked by the noise factors I mentioned earlier. What you're looking at is a null hypothesis that Losman hasn't improved. Then you're trying to disprove the null by looking for a statistically significant improvement in his adjusted points per game total for this year versus his second stint last year. In this case, you're going to fail to disprove the null. But failing to disprove the null is not the same as proving the null. It's possible that you couldn't disprove the null because the null was true. It's also possible the null is false, but that you couldn't disprove it because there weren't enough data points for you to do so.