After discussion has simmered along via email, Usenet and mailing list, Marie Gryphon has posted a nice summary on her blog.
Several blogs have picked up on this: Julian Sanchez, Jim Henley, Jane Galt, Kevin Drum and Thomas M. Spencer.
Clayton Cramer has posted a letter from John Lott on his blog. Some highlights:
"The overwhelming majority of the survey work was done at the beginning of the period over which the survey was done. It has obviously been a while, but my recollection is that the small number of people surveyed after the first four or five weeks (mainly January 1997) did not include any more defensive gun uses."
In an email to Lindgren Lott wrote "I am willing to bet that I don't start mentioning this [98%] figure until the spring of 1997. If I use it before I said that I did the survey, I will say that they nailed me." Lott used the figure on Feb 6, 1997 and got caught. Does he say "they nailed me", like he promised? No, he changes his story yet again. Instead of three months, the survey is now supposed to have taken one month. This is the fourth version (1. "national surveys", 2. "Kleck", 3. "three month survey", 4. "one month survey") of the source of the 98% figure Lott has now given.
"I did another survey over 10 days this past fall and it will be discussed in a book coming out in a couple of months. The results of the survey are very similar to those previously reported."
It is surprising that with nine published surveys giving numbers ranging from 21%-67% for the percentage firing, Lott could conduct two surveys that give far lower figures. The questions and methodology of this second survey need to be examined carefully to seek an explanation. [There used to be some comments here about the results of Lott's new survey. Clayton had posted that Lott's new survey found a "bit more than 4%" firing. He informs me that he removed this because it was a mistake and could not post the correct figure because Lott does not want the results made public. Consequently I have deleted my comments on this figure.]
"I am not sure that I understand why things should be weighted by household size since I was asking questions about individual experiences."
Because the sampling unit was the household. Let me demonstrate with a small example. Suppose that half the household have two adults and half have one adult. Suppose further that adults in small and large households are equally likely to experience a defensive gun use (DGU), but adults in large households are more likely to shoot, 30% (large) vs 12% (small), say (because they are defending a spouse and children as well as themselves).
OK, now, since 2/3 of people and hence 2/3 of DGUs are in large households, it follows that 2/3*30+1/3*12 = 24% of DGUs involve shooting. However, since Lott is sampling by household, he will find half of his DGUs in small households and half in large households and hence 1/2*30+1/2*12 = 21% of his DGUs involve shooting.
Notice, however, that even in this contrived example the difference is small, so this can't possibly explain Lott's anomalous 2% shooting figure. I also did an analysis on the 1992-2001 NCVS data set, and it gives 21% shooting whether you weight by household or don't weight by household.
Next, Clayton gives me too much credit. Otis Dudley Duncan deserves the credit for uncovering this mess. As part of my critique of Lott I listed his 98% claim in a long list of errors in More Guns, Less Crime, but without expressing any doubts that he actually conducted a survey. Duncan contacted me, suggesting that the survey might never have been carried out. While I was initially sceptical, Duncan was able to persuade me that there was a real issue here. (The key was Lott's failure to respond to a reasonable request from Duncan for evidence that the survey had actually been carried out.) In September 2002, I raised the issue in Eugene Volokh's firearmsregprof mailing list. The ensuing discussion involving Clayton, myself, Glenn Reynolds and several others prompted James Lindgren's investigation that he reports on here.
Finally, I should comment on the overall significance of this question. Lott's 98% claim takes up just one sentence of his book. Whether or not it's true, it doesn't affect his main argument, which is about alleged benefits of concealed carry laws. I don't think any fuss would have been made if Lott hadn't repeated the claim numerous times on TV shows, on radio shows, and in opinion pieces. I have a list containing 52 examples. By suggesting to millions of people that all you have to do defend yourself is wave a gun around when all the evidence suggests that more serious action is often required, Lott could cause people to act in ways that could get them injured or killed.