In Lott's latest response, he changes his story again. He originally told Lindgren that hadn't discussed the survey with anyone at the time. Now he has recalled the name of an economist he discussed it with at the time. Unfortunately, when Julian Sanchez contacted this economist, he was unable to recall those discussions. He also originally told Lindgren that the survey was conducted by "several University of Chicago undergraduate volunteers" (a survey of the size he claims to have conducted would have required at least ten students, and much more than that if almost everything was done in one month as he is now claiming). Now he seems to be saying that only two students from U of Chicago were involved, and the rest were from other places. This latest story change from Lott is truly disturbing because it appears to be designed to stymie Lindgren's plan to settle the issue. This was Lindgren's plan:
"If someone were to email the 1997-98 Chicago college alumni twice, I remain virtually certain that at least one of those who did the study for Lott would come forward, if the study were actually done. If no one came forward after two attempts emailing the entire class, the evidence would tend to point strongly to the probability that the 1997 study was never done."
If there only two U of Chicago students involved, then, given the frequency at which people change email addresses, it would not be at all surprising to get no response from the mass email. Lott has changed his story and changed his story until he has ended up with one that cannot be proven or disproven.
Julian also has some comments where Lott claims that the difference between the 2% shooting figure he got and the 21% to 67% shooting numbers from all nine published surveys is not statistically significant. He mentions that in his current survey he found 13 defensive gun uses, and in only one of them (8%) was the gun fired. We can calculate a 95% confidence interval for the percentage (via Wilson's method for those interested in the technical details). It's 1.4%-33%. (If you're not sure what a 95% confidence interval means, it's the same thing people are talking about when they say that a survey is plus or minus 3%.) Since the interval includes 2%, this new survey is consistent with what he alleges he found in his first survey. Trouble is, since it also includes 21%, 24%, 27% and 28% it is also consistent with most of the big published surveys. (These had hundreds of defensive gun uses in the sample.) The trouble is that the sample size of 13 defensive gun uses is much too small to say anything meaningful about the percentage who fire their weapons. Most statisticians would not even print a percentage based on a sample size of 13, instead they would print an asterisk, with a footnote stating that the sample size was too small for the number to be meaningful. Lott's claim that his new survey confirms his previous one is nonsense.
As for Lott's original survey, he hasn't told us how many defensive gun uses were found in that survey (maybe he has forgotten?) so we can't determine if his 2% shooting number was significantly different from the numbers in the other surveys, but there are only two possibilities:
- it wasn't, in which case he should not have repeated the statistic 50 times when there were numbers from other, vastly more statistically reliable surveys available
- it was, in which case Lott's latest explanation is wrong, and he still has to explain why his survey gave such a radically different number.