Lott changes his story again

In Lott's latest response, he changes his story again. He originally told Lindgren that hadn't discussed the survey with anyone at the time. Now he has recalled the name of an economist he discussed it with at the time. Unfortunately, when Julian Sanchez contacted this economist, he was unable to recall those discussions. He also originally told Lindgren that the survey was conducted by "several University of Chicago undergraduate volunteers" (a survey of the size he claims to have conducted would have required at least ten students, and much more than that if almost everything was done in one month as he is now claiming). Now he seems to be saying that only two students from U of Chicago were involved, and the rest were from other places. This latest story change from Lott is truly disturbing because it appears to be designed to stymie Lindgren's plan to settle the issue. This was Lindgren's plan:

"If someone were to email the 1997-98 Chicago college alumni twice, I remain virtually certain that at least one of those who did the study for Lott would come forward, if the study were actually done. If no one came forward after two attempts emailing the entire class, the evidence would tend to point strongly to the probability that the 1997 study was never done."

If there only two U of Chicago students involved, then, given the frequency at which people change email addresses, it would not be at all surprising to get no response from the mass email. Lott has changed his story and changed his story until he has ended up with one that cannot be proven or disproven.

Julian also has some comments where Lott claims that the difference between the 2% shooting figure he got and the 21% to 67% shooting numbers from all nine published surveys is not statistically significant. He mentions that in his current survey he found 13 defensive gun uses, and in only one of them (8%) was the gun fired. We can calculate a 95% confidence interval for the percentage (via Wilson's method for those interested in the technical details). It's 1.4%-33%. (If you're not sure what a 95% confidence interval means, it's the same thing people are talking about when they say that a survey is plus or minus 3%.) Since the interval includes 2%, this new survey is consistent with what he alleges he found in his first survey. Trouble is, since it also includes 21%, 24%, 27% and 28% it is also consistent with most of the big published surveys. (These had hundreds of defensive gun uses in the sample.) The trouble is that the sample size of 13 defensive gun uses is much too small to say anything meaningful about the percentage who fire their weapons. Most statisticians would not even print a percentage based on a sample size of 13, instead they would print an asterisk, with a footnote stating that the sample size was too small for the number to be meaningful. Lott's claim that his new survey confirms his previous one is nonsense.

As for Lott's original survey, he hasn't told us how many defensive gun uses were found in that survey (maybe he has forgotten?) so we can't determine if his 2% shooting number was significantly different from the numbers in the other surveys, but there are only two possibilities:

  1. it wasn't, in which case he should not have repeated the statistic 50 times when there were numbers from other, vastly more statistically reliable surveys available
  2. it was, in which case Lott's latest explanation is wrong, and he still has to explain why his survey gave such a radically different number.
Tags

More like this

Julian Sanchez finds evidence that Lott lost data because of a computer crash. I'm afraid that he hasn't discovered anything new---his time would have been better spent reading Lindgren's report: "I talked with one of Lott's co-authors on another paper, Bill Landes, and received emails…
After discussion has simmered along via email, Usenet and mailing list, Marie Gryphon has posted a nice summary on her blog. Several blogs have picked up on this: Julian Sanchez, Jim Henley, Jane Galt, Kevin Drum and Thomas M. Spencer. Clayton Cramer has posted a letter from John Lott on his…
A few days ago I observed that Lott had changed his story from his original, unworkable, claim that he had used 1836 categories (sex, race, age and state) to weight his data to the claim that he had used just six (sex and race). If this is indeed the scheme he used then two…
Lott's reply to Duncan's article raises some disturbing questions about Lott's honesty. See also James Lindgren's report on his attempt to find some evidence that Lott actually conducted a DGU survey. Where did that 98 percent come from? 98 percent claims before 1997 Way back in 1993 in…