John R. Lott - General Discussion on 1997 and 2002 surveys

[Note: This is a copy of a document found at this link on John Lott's website on May 13, 2003. I have added critical commentry, written in italics like this.
Tim Lambert ]

With some recent attacks on me in a variety of places from the Washington Post to the Chicago Tribune to numerous other places, I thought that I should send out some responses for those who might be interested.

A) There have been many claims that I didn't conduct a survey in 1997 that was reported in one sentence on page 3 of my book, More Guns, Less Crime. In the attached MSWord file, I present the complete statements from the people listed below.

1) Professor David Mustard confirms that I discussed doing the survey together with him many times during 1996; that he likely knew that I had completed the survey by myself in 1997; and that he personally spent a long time in August and September 1997 helping me replace some of the data lost in the July 1997 hard disk crash. [Mustard is only sure that Lott told him about the survey in 1999, which was after questions were raised about the source of the 98% statistic.]

2) A statement by John Whitley, who at the time was a graduate student at the University of Chicago and is now at Adelaide University in Australia, saying that he believes that he met the students that conducted the survey. He also confirms my hard disk crash. [All Whitley can say is that he met some students who might have been doing a project with Lott. There is nothing to suggest that the students were working on a survey.]

3) David Gross, a former assistant city prosecutor from Minneapolis, provides evidence that he was interviewed in my survey in 1997. There were only two other defensive gun surveys after 1995 and neither was done in 1997. Both those other surveys were done by Hemenway and were extremely different from the survey that I gave. [Gross was one of the prime movers behind the Minnesota concealed carry law, devoting four years and $1M in lost income towards getting it passed. It is too much of coincidence for someone with this much of a motive for preserving Lott's credibility to have been randomly chosen in a national survey.]

4) Geoff Huck, the editor at the University of Chicago Press who handled More Guns, Less Crime, remembers that I lost the computer file for my book in the computer crash and that part of the book was permanently lost, though it has been six years and he can't remember what part that was. While he no longer works for the Press and does not have his work e-mails, he does have one e-mail on his home computer from the end of July 1997 that helps verify the loss of material for my book. [While Lott has plenty of people who can confirm that he lost some data, he has noone who can see that he lost data from a survey.]

5) Multiple academics also confirm my hard disk crash. Many of these academics were involved in co-authoring research with me and themselves suffered from this loss because it affected our joint research. They don't remember all the other data that was lost, but they can confirm that the hard disk crash was catastrophic and that I lost all the data that I had. All these statements are backed up by memos in the attached file. Eight academics wrote letters to the Post, but I could have gotten many others.

One point should be made clear. While I lost all my data for my book, David Mustard as well as others to a lesser extent helped me replace all the crime data so that many dozens of academics around the country have been able to replicate every single regression in the book. With respect to the survey data that was lost, I obviously could not go to a primary source to replace that data, [Lott claimed to have the data on tally sheets, so it could have been reentered in a day or two.] but last year, as part of my new book that should be out in a couple of weeks (The Bias Against Guns), I replicated the survey and obtained fairly similar results. [Lott claimed that the new survey gave a figure of 95%, but it does not.] That survey data has been available to any academics who have asked for it. The Washington Post's statement (2/11) that "a 1997 survey the researcher purportedly did to support claims in his provocative book, 'More Guns, Less Crime,'" is incorrect. We are talking about one number in one sentence. [However the claim was repeated over 60 times.]

Because of claims that no one would finance this research themselves, I have made my business and personal income tax forms from 1997 available to some other academics. Many statements have also been made about how long it would have taken me to do the survey. I designed the survey so that about 90 percent of those who took the survey only had to answer three very short questions (one question about whether they have been threatened by violence during the last year and then two short demographic questions). For those people, the survey would often take only 30 seconds. Only one percent of those surveyed were asked a maximum of seven question and even then the questioning lasted just a few minutes. [Such a short survey is pointless---it does not provide any information that has not already been obtained in other surveys. Lott has not explained what research goal this survey was supposed to serve.]

B) Did I Attribute the 98 Percent Brandishing Number to Others? No

Apparently, some credence is being given to the claim that I have attributed the 98 percent brandishing estimate to others instead of myself. Some are taking this as evidence that I never conducted the survey. Yet, the fact is I never attributed this number to anyone else other than myself. It is claimed that I attributed this number to Gary Kleck on one occasion as well as the Los Angeles Times, Gallup, and Peter Hart during a couple of other times. Attachments entitled "Did I Attribute the 98 Percent Brandishing Number to Others?" and "Cases where 98% number was used" provide a detailed discussion of these issues. However, two brief statements are useful here:

The Independence Institute web site posted a piece that contained the one reference to Gary Kleck's work.

------ Forwarded Message From: "Dave Kopel" <david@xxxx.xxx>

Date: Sun, 9 Mar 2003 13:07:49 -0700

To: <cut>

Subject: Re: FW: A quick question. John Lott

I've got no specific recollection of editing the piece, but the evidence seems to indicate that attributing the 98% figure to Kleck was an error by the Independence Institute, rather than an error on the author's part.

Dave Kopel

------ End of Forwarded Message

The second case of false attribution is that I claimed that the Los Angeles Times, Gallup and Peter Hart were the source of the 98 percent number. Where people get the claim is by combining two sentences in a Chicago Tribune op-ed that I had in 1998 (the same op-ed was also republished, for example, in the Washington Times).

The relevant passage from the op-ed reads:

"Other research shows that guns clearly deter criminals. Polls by the Los Angeles Times, Gallup and Peter Hart Research Associates show that there are at least 760,000, and possibly as many as 3.6 million, defensive uses of guns per year. In 98 percent of the cases, such polls show, people simply brandish the weapon to stop an attack."

If the reference in the second sentence had been to "these" polls and not "such" polls, I would think that the critics would have a much better argument. Instead, I view "such polls" as merely referring back to this type of polls and not those specific polls. Still there is admittedly an error in using the plural. The most plausible explanation is that I was describing what findings had been generated by the polls, in other words I was viewing them in general as a body of research. [I don't think that this is the most plausible explanation, but even if it is true, Lott has contradicted his claim that he "never attributed this number to anyone else other than [him]self". Here he says that the 98% came from the polls as a general body of research. At best, only one of those polls was his. ]

C) As to other claims by Al Hunt on CNN that I have not given out data for the work that I did on the Florida election and for concealed handgun research are plain false. A list of just some of the critics to whom I have given those data is also provided in the attached collection of memos.

D) The Chicago Tribune story a couple of weeks ago adds to the list and references an academic who "wonders" whether I had really replicated the survey again in 2002. I have provided a list of student names who did the survey, provided the survey questions and methodology, shown the telephone records, and provided the data. If anyone would like to see any of this material, please e-mail jknowles@xxxx.xxx. In addition, James Knowles, a research assistant at AEI, sent the Tribune a letter, though it wasn't published:

From: "James Knowles" <jknowles@xxxx.xxx>

Date: Mon Feb 17, 2003 7:12:38 PM US/Eastern

To: <TribLetter@tribune.com> Cc: "James Knowles" <JKnowles@xxxx.xxx>

Subject: Re: *Another Misfire in the Academic Shootout on Guns* (Feb. 14)

To the Editor:

Ron Grossman closes Friday's news article "Another Misfire in the Academic Shootout on Guns" (Feb. 14) casting doubt as to whether John Lott Jr.'s 2002 survey on defensive gun use actually occurred. Grossman quotes Florida State criminologist Gary Kleck saying the Lott staff would have needed "500 evenings to do 2,000 interviews". Kleck's calculation assumes that any one staff member might conduct two surveys an evening. As supervising research assistant for the Lott survey, I believe that Messrs. Kleck and Grossman have misunderstood the nature of our work. [Indeed, Kleck thought that the survey would be to meet some research goal, not to bolster claims of having conducted a survey in 1997.] Unlike Kleck's previous surveys, the Lott survey consisted of a maximum of seven questions asked of any respondent. With an average of five callers a night completing about 25 surveys each, we ended up with just over a thousand respondents in two weeks of calls. We have produced telephone records and disclosed the names and email addresses of surveyors to support the integrity of the survey. With that said, I believe the debate would be most intelligently furthered by moving past questions of whether these results were fabricated.

JAMES KNOWLES

American Enterprise Institute

Washington, D.C., Feb. 17, 2003

(202) 862-7183

E) With respect to the charges in the Washington Post that David Mustard now believes that the evidence from the 1990s overturns our original work, David wrote the following to the Post though like all the other letters, it was not published:

Dear Letters Editor:

In his evaluation of the book "Evaluating Gun Policy", Saul Cornell stated, "John J. Donohue challenges economist John Lott's contention that states issuing permits for carrying concealed weapons have seen significant reductions in violent crime. In a commentary on this essay, David B. Mustard, Lott's original co-author, quibbles with Donohue but concedes that data from the 1990s show a rise in crime in those states that adopted concealed-carry laws."

I would like to correct this misrepresentation of my analysis, in which I write, "Unfortunately, many of [Donohue's] criticisms have already been addressed extensively in the literature" and that Donohue's own data show that "the passage of the law [that allows law-abiding citizens to carry concealed weapons] is associated with sharp decreases in murder, rape and robbery." [Mustard misrepresents his own writing here. The quote does not appear in the book at all, since it was not a reference to what Donohue's data showed, but to what Lott's data showed. The passage it comes from was deleted from the book when Lott's data was found to have serious coding errors.]

--

David B. Mustard

Terry College of Business

University of Georgia

Athens, GA 30602

Phone: (706) 542-3624

Other misleading charges about David's changing views on our research have also been made in other publications. If you would like a discussion of those, it can be provided. Unfortunately, absolutely none of the various supportive letters sent to the various newspapers have been published.

F) Response to the New York Times attack:

------ Forwarded Message

From: John Lott <johnlott@xxxx.xxx>

Date: Tue, 04 Mar 2003 21:01:39 -0500

To: Letters New York Times <letters@nytimes.com>

Subject: Pity the Poor Gun, 3/2/03, Section 7, Page 31 (earlier version of letter transposed numbers in date)

Dear Letters Editor:

Your comic strip would be funny if it didn't contain so many errors (Pity the Poor Gun, 3/2, Section 7, Page 31). 1) My hard disk crash is extremely well documented. 2) There is evidence of the survey's existence from both a couple of professors and from a survey participant. [The only real evidence is from someone who claims he was surveyed. And that someone just happens a person who Neal Knox says singlehandly got the concealed carry law in Minnesota passed.] 3) I never attributed the 98% figure to the Los Angeles Times, Gallup, or Peter Hart. [Most people would interpret his sentence as attributing the claim to such polls.] 4) The claims you raise are over one number in one sentence in my book More Guns, Less Crime. ALL the crime data in the book was replaced after the hard disk crash and has been given to dozens of academic researchers. Every single regression measuring the impact of guns on crime has been redone by others.

I should also mention that I replicated my lost 1997 survey in 2002 and got similar results--to be presented in my new book, The Bias Against Guns, when it is released in a couple weeks. Incidentally, only two other surveys have been done on this question since 1990 and one found that in 92% of all defensive gun uses either brandishing or firing a warning shot was sufficient to scare off the attacker. [The 92% number comes from Kleck's survey, but it is not the number for brandishing and warning shots. That number is 84%, but why doesn't Lott report the brandishing number from Kleck's survey? That is 76%, very different from Lott's 98%.]

Sincerely,

John R. Lott, Jr.

Resident Scholar

American Enterprise Institute

1150 17th Street, NW

Washington, DC 20036

(202) 862-4884

http://www.nytimes.com/imagepages/2003/03/02/books/20030302stamaty.html

------ End of Forwarded Message

This letter was never published.

Finally, a response to the claims about the use of the pseudonym in internet chat rooms is available at the very end of the attached "memos" statement.

Memos regarding 97&02 surv

Did I attribute 98% to others

Cases where 98% number was used

Tags

More like this