[Note: This is a copy of a document found at this link on John Lott’s website on April 25, 2005. I have added critical commentary, written in italics like this. Tim Lambert ]
1) Did I Attribute the 98 Percent Brandishing Number to Others? No
Apparently, some credence is being given to the claim that I have attributed the 98 percent brandishing estimate to others instead of myself. Some are taking this as evidence that I never conducted the survey. Yet, the fact is I never attributed this number to anyone else other than myself. It is claimed that I attributed this number to Gary Kleck on one occasion as well as the Los Angeles Times, Gallup, and Peter Hart during a couple of other times. Attachments entitled “Did I Attribute the 98 Percent Brandishing Number to Others?” and “Cases where 98% number was used” provide a detailed discussion of these issues. However, two brief statements are useful here:
The Independence Institute web site posted a piece that contained the one reference to Gary Kleck’s work.
—— Forwarded Message
From: “Dave Kopel”
Date: Sun, 9 Mar 2003 13:07:49 -0700
Subject: Re: FW: A quick question. John Lott
I’ve got no specific recollection of editing the piece, but the evidence seems to indicate that attributing the 98% figure to Kleck was an error by the Independence Institute, rather than an error on the author’s part.
See more recently:
—– Original Message —–
From: “Dave Kopel”
To: “Julian Sanchez”
Sent: Tuesday, April 01, 2003 9:37 PM
Subject: Re: Sorting out the Lott piece
. . .
Re-discovering Kleck’s 98% figure from Social Problems (an article I thought about a lot in, say, 1994, but which I have rarely thought about recently) makes me believe that I added Kleck to the 98% sentence in the belief that the 98% figure came from him.
The fact that I added Kleck is, of course, also supported by the Rocky Mountain News version of the Lott article (a version I did not edit) which contains no reference to Kleck.
—— End of Forwarded Message
[The attribution to Kleck was probably added by Kopel. Discussion is here.]
The second case of false attribution is that I claimed that the Los Angeles Times, Gallup and Peter Hart were the source of the 98 percent number. Where people get the claim is by combining two sentences in a Chicago Tribune op-ed that I had in 1998 (the same op-ed was also republished, for example, in the Washington Times).
The relevant passage from the op-ed reads:
“Other research shows that guns clearly deter criminals. Polls by the Los Angeles Times, Gallup and Peter Hart Research Associates show that there are at least 760,000, and possibly as many as 3.6 million, defensive uses of guns per year. In 98 percent of the cases, such polls show, people simply brandish the weapon to stop an attack.”
Webster’s Unabridged Dictionary defines the word “such” as “like or similar,” but not as the same. “Such” is merely referring back to this type of polls and not those specific polls. If the reference in the second sentence had been to “these” polls and not “such” polls, I would think that the critics would have an argument.
[Even if Lott was referring to polls similar to the three he mentioned, his statement is false. The polls that are similar to those he mentions show that the number is markedly less than 98%.]
Statements by those who have made general statements about the 1997 survey
2) Statements by others documenting the loss of my computer hard disk in July 1997
From: Dan Kahan
Date: Thu Feb 13, 2003 12:49:32 AM US/Eastern
Cc: John Lott
Subject: Feb. 11, “A Fabricated Fan and Many Doubts”
A column appearing in the Post yesterday (Feb. 11, “A Fabricated Fan and Many Doubts”) implies that economist John Lott made up the claim that a computer malfunction destroyed data from his research on gun control. At the time Lott was engaged in this research, we were colleagues at the University of Chicago Law School. I clearly recall John relating the computer data-loss incident to me then — many years before the current controversy about his work arose.
Just so you know, I’m not relating this information to you because I support Lott’s position on guns (I don’t). I’m relating it to you because I think journalists — even the ones you employ to write political gossip columns like this one — should live up to their professional obligation to check out the facts before they make claims harmful to an individual’s reputation.
Dan M. Kahan
[The column in the Post did not imply that Lott made up the claim about the computer malfunction. It questioned whether Lott lost his survey data. On this question Kahan has no information. Kahan should have checked out the facts himself before he wrote his letter in support of Lott.]
Dan M. Kahan
Professor of Law
Yale Law School
PO Box 208215
New Haven, CT 06520 (regular mail)
127 Wall Street
New Haven, CT 06511 (courier)
(815) 366-1458 (fax)
Date: Tue Feb 11, 2003 2:50:46 PM US/Eastern
Subject: Article about John Lott in today’s Post
The Washington Post unfairly casts doubt about whether John Lott suffered a hard disk crash on his computer in 1997 ( A Fabricated Fan and Many Doubts, February 11). I was co-authoring a paper with him at the time and I was affected by some data that were lost. We lost a very large data set that had been used to estimate the wage premium paid to workers exposed to long-term latent hazards in the workplace. The loss prevented us from performing additional research and significantly delayed publication.
[The column in the Post did not question whether Lott had a disk crash. It questioned whether Lott lost his survey data. On that question Manning has no information.]
Richard L Manning, PhD
203 Putnam Road
New Canaan, CT 06840
From: Lawrence Kenny
Date: Tue Feb 11, 2003 2:21:58 PM US/Eastern
Subject: Wash Post letter
This is what I sent to the Washington Post.
John Lott and I worked together on a project examining the impact on government spending of women being granted voting privileges. Some of this research, utilizing older census data, was published in the Journal of Political Economy in December 1999. But the publication of other research utilizing recent survey data was set back when the basic data was lost in 1997 when John’s hard disk crashed. Thus, assertions that John fabricated the story of his disk crashing are incorrect.
[The Post did not assert that Lott fabricated the story of his disk crash. It questioned whether Lott lost his survey data. On that question Kenny has no information.]
Lawrence W. Kenny
Professor of Economics
University of Florida
Date: Thu, 13 Feb 2003 23:55:13 -0800 (PST)
From: Jonathan Karpoff
Subject: John Lott
A column the Post published this week implies that John Lott fabricated a story that a computer crash destroyed some data related to his gun research. I have collaborated with Lott on two research projects — neither related to guns — and remember him talking about the crash several years ago. The crash indirectly affected one of our projects, as Lott had to divert much time to re-create his lost databases. I recall him telling me how some of his philosophical opponents refused to help him by returning a copy of some of his data, despite the fact that the only reason they had the data in the first place was that Lott had given the data to them!
[The column in the Post did not imply that Lott fabricated the story of the disk crash. It questioned whether Lott lost his survey data. On that question Karpoff has no information.]
During our collaborations, John Lott has been an exemplar of integrity in academic research. It is not always easy to work with John, as we sometimes have disagreed over how best to conduct our tests and write up our results. But always, Lott has been honest, insightful, and willing to consider arguments and accept data that do not agree with his prior beliefs. He is an excellent social scientist.
It is time to put to bed any rumors that question Lott’s credibility or seriousness as a researcher. Give him credit for taking unpopular positions, sticking to those positions in the face of vitriolic personal attacks, and sharing his data and exposing his research to scrutiny more openly than his opponents. You — and I — might not like like all of his conclusions. But that makes him all the more important to engage seriously in policy debates.
[Academic misconduct is a serious matter. If Karpoff wants to put to bed the questions about Lott’s credibility he should call for an investigation to clear Lott’s name, instead of dismissing them as “rumors” without even taking the trouble to find out what those questions are.]
Jonathan M. Karpoff
Jonathan M. Karpoff
Norman J. Metcalfe Professor of Finance
Managing Editor, Journal of Financial and Quantitative Analysis
University of Washington School of Business
Seattle, WA 98195
Another letter by Karpoff:
———- Forwarded message ———-
Date: Mon, 9 Jun 2003 15:18:49 -0700 (PDT)
From: Jonathan Karpoff
Subject: John Lott editorial
I have worked with Lott on three research projects (none related to guns). The computer crash at which you snicker indirectly affected one of our projects, as Lott had to divert much time to re-create his lost databases. Although Lott got most of his data back, he really did lose some of it.
[Karpoff has no evidence that Lott conducted a survey or that Lott lost survey data in the crash.]
Lott re-created his lost data with a new survey. It is difficult to believe that, in a book filled with empirical tests that have been replicated by others, Lott would fabricate a single number, and that his made-up number would just happen to be statistically indistinguishable from the results of the new survey.
[I think that the 98% number came from an old estimate of Kleck’s that included shots that missed. Several other people have also made that mistake. (Details here.) Lott cited the 98% number many times in 1997 and 1998. It was only when Otis Dudley Duncan pointed out that “national surveys” gave a very different number that Lott claimed that the number came from his own, previously unmentioned, survey. It seems likely that Lott invented the survey to avoid having to admit to making an error.
The difference between the 98% number and the result from the new survey is not statistically significant, but that is because the new survey was so small that the 95% confidence interval is very large: 55%–99%. This includes 98% but it also includes the estimates from the other surveys. Lott claimed that the point estimate from the new survey was 95%, which is close to 98%, but if you check his calculations, you will find that the 95% number is incorrect. After I pointed out this error, Lott switched to saying that his new survey gave an estimate of 90%, but after using this corrected estimate for a while, he went back to claiming that the number was 95%.]
If you are concerned about research integrity, your aim is off. Throughout the controversy over gun research, Lott has shared his data with all comers. In contrast, some of Lott’s critics refuse to share the data they use to attack Lott’s findings. Which seems more suspicious to you?
[Who are these critics that won’t share their data? Lott insinuates that Ayres and Donohue won’t, but their data is available right here.]
You — and I — may not like or agree with all of Lott’s conclusions. But no serious researcher can question Lott’s integrity or skill as a social scientist. The whole point of Kennedy’s editorial is about the importance of integrity in the conduct of research. I would hope that Science has higher verification standards than this piece shows.
[Unfortunately, it seems that Karpoff has taken Lott’s claims at face value, rather than verifying them himself. He is in no position to criticise Science on verification standards.]
Jonathan M. Karpoff
Jonathan M. Karpoff
Norman J. Metcalfe Professor of Finance
University of Washington School of Business
Seattle, WA 98195
From: “David B. Mustard”
Date: Wed Feb 12, 2003 12:04:50 PM US/Eastern
To: John Lott
Subject: Re: Sample letters that were sent in to the Washington Post
This is a copy of my letter, which I sent to the Post yesterday morning shortly after we talked.
On February 11, 2003, in the article “A Fabricated Fan and Many Doubts”, the Washington Post incorrectly questioned whether John Lott experienced a computer crash in the summer of 1997.
[The Post did not assert that Lott fabricated the story of his disk crash. It questioned whether Lott lost his survey data. On that question Mustard has no information.]
I can testify that Mr. Lott’s computer crashed at that time and that he lost everything he had on his hard drive. He and I discussed this crash many times. John and I were co-authors on a project that ended prior to the crash. Because he lost all the data from our project, I replaced as much of the data as I could.
David B. Mustard
Terry College of Business
528 Brooks Hall
University of Georgia
Athens, GA 30602
Phone: (706) 542-3624
Fax: (706) 542-3376
From: John Whitley
Date: Tue Feb 11, 2003 5:50:16 PM US/Eastern
Subject: [Fwd: Letter to the Editor]
Here is the letter I sent to the Washington Post.
——– Original Message ——–
Subject: Letter to the Editor
Date: Wed, 12 Feb 2003 09:19:41 +1030
From: John Whitley
Organization: The University of Adelaide
CC: John Whitley
I believe you draw an inaccurate comparison between the issue of Dr. Lott’s survey and the Dr. Bellesiles case in your article “A Fabricated Fan and Many Doubts” of February 11. I am not an expert on the Dr. Bellesiles case, but my understanding is that there was little or no contemporaneous corroborating evidence of the flood that he reported as destroying his records and that his results were not reproducible by other scholars in the field. Whatever the case is with Dr. Bellesiles, these are definitely not true in Dr. Lott’s case. [In fact, the flood at Emory is well documented. Several scholars had books destroyed. There is however, no evidence that Bellesiles’ notes were destroyed in the flood, just as there is no evidence that Lott’s survey data was lost in his disk crash.]
There are numerous contemporaneous witnesses to Dr. Lott’s hard-drive crash – I am one of them. I began working for Dr. Lott as a research assistant shortly after the hard-drive crash in 1997 and I distinctly remember him mentioning it to me at the time. More importantly, Dr. Lott’s general results have been reproduced in many peer reviewed academic journals and he has a strong reputation among academics for distributing his data and publishing reproducible research.[Lott’s survey results have never been obtained by any other researcher. Just about all Lott’s results on guns and other issues has been criticized by academics for serious flaws.]
Dr. Lott himself has now reproduced the survey and released the names of all people who worked on it and the phone records from the calls. The results are largely in line with his previous results and no one has questioned the integrity of the new survey. [The results of his new survey are quite different from the results of the survey he claimed to have done in 1997. Lott has simply misrepresented the results of his new survey.]
Dr. Lott has a strong academic reputation for remaining faithful to the scientific protocol. Your comparison was unwarranted and inaccurate.
Ashton, SA 5005 Australia
61 (0)8 8390 0504
School of Economics
Adelaide University, AUSTRALIA 5005
Ph : +61 8 8303 5500
Fax : +61 8 8223 1460
3) Statements from David Mustard, John Whitley, and Geoff Huck
Statement on in response to Jim Lindgren’s summary of my comments relating to John R. Lott’s Survey Work on Self-Defensive Uses of Guns by David B. Mustard, Thursday 16 October 2003
In the summer and fall of 1996, I had numerous conversations with John Lott about how he was going to extend the original paper (JLS January 1997); a survey to learn more about defensive uses was one of those extensions.
To summarize, I did not back off my claims of being “likely” or “fairly confident” that I heard about the survey in 1997.
Statement on John Lott’s Survey Work on Self-Defensive Uses of Guns by David B. Mustard Monday 10 February 2003
John and I started working on our concealed carry paper in the fall of 1995. I was finishing my Ph.D. in Economics at the University of Chicago, and John was a faculty member. We worked on our paper intensively from about February 1996 to September 1996. We presented it at the American Law and Economics meetings in Chicago and at some workshops in the spring of 1996. We finished the JLS proofs in August or September. We then completed our work together and interacted much less frequently. In the summer of 1996 Lott talked about working on other projects related to guns. Once our paper was finished he started to work on a number of extensions. I did not work with John on any of these other projects because I had to finish my job market paper and send out my job applications (most due by 1 December 1996). In the rest of December 1996 I worked on my job market paper and practiced interviews. In January 1997 I went to the American Economic Association annual meetings and interviewed. I had campus visits through early February and signed my contract with the University of Georgia around the second or third week of February in 1997. From March to June I finished my dissertation and taught an undergraduate class at Chicago. In July I defended and traveled a lot, and my wife moved to Georgia. I moved to Georgia in early August.
Do I have direct first-hand evidence about John Lott’s survey?
I did not co-author the work on the survey with John, I did not work for John as a paid employee, and I do not know anyone who worked with John on the survey. I have not seen any survey instruments or primary data from the original survey.
I have copies of both the survey questions and the survey data from his 2002 survey that he did to replicate the original survey. I did not work with John on the follow-up survey, but was given the questions and data once the survey was completed. These data are qualitatively consistent with John’s statements about his first survey. [In other words, Mustard has no first-hand evidence.]
Did John have a computer crash in the summer of 1997?
I believe beyond a reasonable doubt that John had a computer crash during the summer of 1997. On Friday 8 August 1997 my wife and I closed on our home in Athens, Georgia. While we were unloading our belongings in our new home a phone rang, which surprised us because we had not yet unpacked or plugged in our phones. We learned that the house had a phone in the garage, which we had not noticed. John Lott was on the phone. I believe that he told me at this time about his computer crash and having lost all his data. During the next few weeks I had additional conversations with John about his hard drive collapse and his lost data, and I agreed to provide him with all my data. My office at Georgia was not set up until late August (we were on quarters at that time and did not start until fairly late). I had a severe infection for a number of days in mid-late August. I returned to Chicago for about 5 days in late August (about August 27-1 September) for my graduation (on Friday 29 August) and to close out our affairs in Chicago. My grandfather died in early September and I returned to my home in Buffalo, NY for a number of days at that time. When I returned to Athens we had orientations and I had to prepare for classes. I likely provided John with the data from our concealed carry paper in mid-September to October. Did John specifically mention that he lost his survey data in his computer crash? John told me that he had lost all his data in the crash. He specifically told me that he lost all data related to our paper, which I later restored to the best of my ability. He also mentioned how he lost many things related to his book, which set him back in completing the book and forced him to eliminate some things he intended to include in the book. [Lott keeps proving over and over again that he had a disk crash, because he has no evidence that he conducted a survey in 1997.]
Was their evidence of Lott’s intending to do a survey?
As we worked on the concealed carry paper, John talked about pursuing other projects to extend our work on concealed carry and guns. We talked extensively about self-defensive uses of guns and how we knew how frequently guns were used for self-defense and in what contexts they were used. John articulated a desire to learn more about self-defensive uses through a survey, just like he articulated an interest in many other extensions, like using city-level data (which we did not use in the 1997 paper), decomposing the effects more by time and place (which we started to do in the preliminary paper), decomposing the effect by different types of robbery classifications, and determining whether surrounding areas experienced increases in crime after a concealed carry law was passed. (This is a partial and not complete list of the extensions of the gun-related research). John generally followed through on these extensions. He further decomposed the effects by time and place and used city-level data in his book More Guns, Less Crime. He analyzed the effect on surrounding areas with Steve Bronars in “Criminal Deterrence, Geographic Spillovers, and the Right to Carry Concealed Handguns.” American Economic Review. Vol. 88, n2 (May): 475-479. He was unable to decompose the effect by robbery type because he was unable to find robbery data that allowed him to do so.
[But the survey Lott claims to have conducted does not provide more information about self-defensive gun uses. If it really was conducted, the number and scope of the questions were much less than in previously conducted research. If Lott wanted to learn “in what contexts [guns] were used” defensively, why were there no questions about this in his survey?]
When do I first remember talking with John Lott about the survey?
I do not remember the first time John Lott and I talked about the survey. At the time there was nothing exceptional about the survey for me to associate with it and help me remember when I first learned about it.
I believe it likely that John informed me of the completed survey in 1997.
I think it highly probable that John told me he had completed the survey at the time of my talk at the Academics for the Second Amendment conference in Washington, DC in November 1998.
[An earlier version of this statement included this: “I know beyond a reasonable doubt that John and I talked about the completed survey before I testified to the Maryland House of Delegates Judiciary Committee on 20 October 1999 about Maryland House Bill 736 to allow citizens to carry concealed weapons.”
The only date that Mustard is sure of is that he was told before October 1999. This is consistent with the other dates when Lott told people about the survey—the May 13 letter and May 21 phone call to Duncan, the May 25 letter to the Wall Street Journal, and the June 23 email to me. It seems likely that Lott told Mustard about the survey around the same time and this is why Mustard recalls that he had already heard about it in October 1999.]
Comment on John Lott’s data distribution policies
John has always impressed me with his willingness to give out his data. To my knowledge John has always released his data to anyone who asks for it. In fact, we gave out our data about 4 months before the article even came out in print. We have now given our data to about 75 people from around the world; perhaps more. This behavior sets John apart from many scholars who will not release their data even after multiple requests to do so. [However, Lott would not answer reasonable questions about his survey from Otis Dudley Duncan.]
Another statement by David Mustard
Date: Tue Jun 3, 2003 4:04:39 PM US/Eastern
To Whom It May Concern,
. . . you allege that Mr. Lott did not have a severe computer problem in which he lost a substantial amount of data. [In fact, the article Mustard is responding to, made no such allegation, but rather raised questions about whether Lott had conducted a survey in 1997.] I believe beyond a reasonable doubt that Mr. Lott had a computer crash during the summer of 1997. During the summer of 1997, my wife and I moved from Chicago to Athens, GA. On Friday, 8 August 1997, we closed on our new home. John called the evening we moved into our home. I believe he told me about his computer crash and that he lost all his data at this time.
During the next few weeks I had additional conversations with John about his computer crash and lost data. He specifically mentioned that he had lost all of the data related to our JLS paper and many things related to his book More Guns, Less Crime, which delayed the completion of the book and forced him to eliminate some things he intended to include in the book. I agreed to help John with all the data from our Journal of Legal Studies paper (Jan. 1997). I helped John with these data in the fall of 1997. [None of this is evidence that Lott conducted a survey.]
David B. Mustard
Terry College of Business
528 Brooks Hall
University of Georgia
Athens, GA 30602
Phone: (706) 542-3624
Fax: (706) 542-3376
evidence that part of original book being lost
From: Geoffrey Huck
CC: John Lott
Date: Wednesday – January 15, 2003 9:57 AM
Subject: Re: John Lott Survey
Attachments Mime.822 (3600 bytes) [View] [Save As]
Dear Mr. Sanchez:
Since I no longer work for the University of Chicago Press, I don’t have access to the Press’s files. Nevertheless, I am pretty certain that there is nothing in those files that would help you. The Press generally did not keep any draft manuscripts, tables of contents, etc., after publication – there was simply not enough space to accommodate all that that would entail. I have purged all my Press files from my home computer, and I would doubt that there are any electronic files at the Press that would answer your questions. That leaves my archived home e-mail files (some of which I still have) and my memory (which isn’t all that reliable). As to the latter, I have a vague recollection of a chapter or a section or sections of a chapter that had to be scrapped because of the computer crash, but I don’t at this stage remember the subject of it (or them). At the time, we were talking about a variety of things John could do (e.g., including a chapter on mass public shootings). As to my e-mail archives, there are a couple of brief mentions in John’s and my exchanges about the crash and loss of data, though I have found nothing explicitly about the defensive use of handguns in them (which doesn’t mean anything in itself, since we were mostly talking on the phone and there must have been all kinds of things that were lost in the crash that we didn’t discuss in the archived e-mails I still have). There is one message from July of 1997 that gives some sense of the magnitude of the problem we were dealing with, though. We fortunately did have an earlier draft of the manuscript in hard copy at the time of the crash and a disk with some – but not all – of the tables. My assistant Theresa let me know what was missing. I don’t at this point know what these tables are – you might want to ask John about that.
[Lott keeps proving over and over again that he had a disk crash, because he has no evidence that he conducted a survey in 1997.]
Date: Fri, 25 Jul 1997 16:13:03 +0000
From: Geoffrey Huck
Subject: More about tables
John: Theresa tells me that she has on the disk you gave her all the tables from 1 to 30 except 2, 5, 16, 21, 24, and 25, and 2 appendix tables. I asked her to e-mail you the contents of the disk. Hope this helps.
There is also one e-mail message I have from February 1997 which contains an attachment that I’m not at this point able to decode. If that file turns out to be decodable and has anything of relevance to add to this question, I (or John) will let you know.
I hope you find this information useful.
—— End of Forwarded Message
—– Forwarded message from John Whitley
John Lott has kept me informed on some of the highlights of your investigation into the survey he conducted in the mid-90’s. In particular, he indicated you were having trouble finding corroborating evidence to his hard drive crash at the time and overall evidence that he conducted the survey. I can definitely vouch for the hard drive crash and I do have some recollections which may be from the survey, but I am not very sure of those.
[Lott keeps proving over and over again that he had a disk crash, because he has no evidence that he conducted a survey in 1997.]
I started graduate school in economics at the University of Chicago in 1996. I met John in my first year and began working for him as an RA the summer of 1997 when my core exams were finished (I don’t know the exact dates off the top of my head, but I can look them up if you would like). That work was on city police departments for his paper on affirmative action in policing and we then went on to collaborate on a paper on safe-storage gun laws and have worked together several times since.
As I said above, I can definitely corroborate the hard drive crash. He mentioned the crash to me in at least one conversation we had in the months leading up to my starting as an RA. It came up because he was telling me he was telling me he was very far behind in his work because he was having to dig out from a hard drive crash. I think it came up in the context that he couldn’t yet tell me what I was going to be working on when I started as his RA because he was so far behind, but I can’t remember for sure. His specific comment was worded as a complaint about how “you think you have these things backed up and it never works” or something similar (it could have been that he wished he had backed it up or something, I don’t really remember the specifics). Although I remember very little from those early conversations I had with John, that particular comment stuck out and I remembered it because, although I had had a computer as an undergraduate, I was really just then becoming computer literate as a serious user. I hadn’t really thought about backing up hard drives and the like at the time and John’s comment opened a range of issues I had never thought of. It has stuck in my head ever since because I have been more careful since and I remember it every time I change or use my backup systems.
Unfortunately, I can’t directly corroborate the survey, but I do have one memory which may be related. I remember stopping by John’s office one time I think during my first year and there were some undergraduates in the office. John was finishing up with them and my recollection is that he introduced me to them and then they left. I think he introduced their names (which I don’t remember) and said that they had recently worked for him (although I don’t remember if he said on what), they then left and I met with John to talk about working as an RA for him. I am pretty sure they were undergraduates because I seem to recall them being impressed when I said I was an econ graduate student (anything that inflates your ego during the first year of graduate school at the University of Chicago is a big deal at the time).
In that situation, what I really remember most is the scene and not the words. I don’t know when exactly it was, but I can remember the room. It was in John’s old office, when he was in the middle of the back wall of the Chicago Law School library (before he moved over near the stairwell to the smaller office). I can remember him sitting at his desk and the students (I am pretty sure there were two students, but not 100% positive) were standing between me at the door and John at his desk against the far wall. I think one was taller than me (I am 5’6) and had lighter hair while the other was shorter and had darker hair (the heights I am pretty sure of, the hair color I am less sure of). The taller one was closer to me and seemed to be more of the leader. For some reason, that image sticks in my head.
Unfortunately it is possible that I am mixing this scene up in my head with other events, but it is fairly clear in my head so I am at least reasonably confident in it. If my recollection is correct, it is entirely possible (very likely, in fact) that these were some of the students who had worked on the survey (I do have some vague recollection that they had coordinated something and that others may have been involved). Unfortunately that is all I can really remember on that one right now, sorry I can’t be more specific.
[In other words, Lott once had some students in his office. This is not evidence that he had students conduct a survey.]
I hope this helps. Please let me know if you need any more information or if I can be of any further assistance.
— John Whitley School of Economics Adelaide University, AUSTRALIA 5005 Ph : +61 8 8303 5500 Fax : +61 8 8223 1460 e-mail: firstname.lastname@example.org
—– End forwarded message —–
4) Email from Joseph E. Olson
From: Joseph E. Olson
Subject: Jim Lindgren’s “report”
[Note the use of scare quotes. Look at Lindgren’s report and ask yourself why Olson tries to dismiss it instead of addressing the content. Olson is a partisan Lott supporter who tried to shut down the discussion on firearmsreg about Lott’s survey in 2002. Neal Knox wrote about him:
Prof. Joe Olson and Atty. David Gross, who wrote and almost single-handedly pushed the [Minnesota concealed-carry] law into existence
Yes, that’s the same David Gross who claims to have been surveyed by Lott.]
Date: November 17, 2003 4:12:50 PM EST
To: Glenn Reynolds
Jim Lindgren concludes that David Gross took a 1996 survey by a professional polling organization done as part of a study by Hemenway and Azrael at Harvard rather than a 1997 survey taken by students as part of a study John Lott of Chicago was conducting.
Mr. Gross disagrees.
What follows is a compliation I’ve put together of a number of David Gross’ e-mails to me discussing this situation.
I don’t know where James Lindgren gets off stating things as he does, for he is wrong about my improving memory. I expressly closed the issue out quickly. Please, see my email of January 20, 2003 at 4:25 P.M. I spent a total of less than 24 hours involved in the discussion, start to finish. I don’t believe that I said that the survey concerned the laws; I said that I was familiar with the carry laws. I think James Lindgren has that incorrectly in any notes he took. That’s his confusion; not mine.
I sent my first message concerning my awareness of the issue at 4:26 P.M. on January 19, 2003. I talked with Mr. Lindgren once that evening, [shortly before] 10:46 P.M when I sent a message referring to his call. Lindgren was impressed that I remembered the timing of the talk at the Minneapolis Athletic Club (some four years earlier) to within a couple of weeks, and that, when I said “a couple of years before” to John Lott, I meant “a couple of years” and not 1 3/4 years or 2 1/4 years. THAT, alone, excludes the 1996 Harvard study, because it WAS early 1997 when I answered the short survey, NOT 1996.
[This does not make sense. I have never heard of anyone using “a couple of years” to mean “exactly two years” before. And why would he be trying to nail down exactly when the survey was done to the month when he talked to Lott?]
I believe that in my emails to you and in my conversation with James Lindgren, I have stimulated just about all of my memories and impressions that I’m going to have concerning the survey that I took, without having any substitutions and additions by any possible suggestion, express or implied. I do not wish to be seen as a partisan in this matter, for I am not. I also am unwilling to speculate. My recollections and impressions of the survey that I took are now closed to prevent any accusations that I am having “convenient” recollections. I simply refuse to get in the middle of this any more than I already have. I am a witness who has come forward; I don’t have a dog in this fight.
[In fact, at the time that he wrote this email, Gross most definitely did have a dog in the fight, since he was actively trying to get CCW laws passed in Minnesota, and if Lott’s credibility was destroyed, Gross’ campaign would have been damaged. ]
The bottom line is that listening to the audiotape [of Lott’s 1999 speech], today, has confirmed the recollections that I had of the survey and the timing.
It goes beyond any statistical possibility that I was surveyed by anyone else, given the timing, the subject matter, its brevity, it’s precise focus, etc.. The clincher, for me, was my recollection, last night, in my e-mail, that there was a question concerning warning shots as a distinct category of discharge as opposed to a shot directed/fired AT the criminal. From that e-mail, before I listened to the tape of his talk, today: “I don’t recall whether the survey was limited to permits to carry and usage, or whether the subject matter was the broader “defensive use” to include use on
private property. I think it was the broader question, because I know that I mentioned those two significant incidents in my life as defensive use. I seem to recall, at this point, some question about a “warning shot,” concerning discharge of a firearm, but I’m not sure.” I had not listened to the tape, when I wrote that note, because I hadn’t even begun to look for it yet. I had just concluded my conversation with James Lindgren and was responding to an earlier email reporting that I had been contacted by James Lindgren, already; and amplifying my comments on what I remembered; as well as obligating myself to try to find the audiotape, something I had recalled since I was first contacted, and which I had mentioned during my conversation with James Lindgren.
Yet, as clear as day, in the tape of the talk, as I listened this morning, was the distinction between warning shots as a discharge (a distinct category, or part of “brandishing”) and shots directed/fired AT the criminal. Bingo! That was the sound of a synapse firing, ladies and gentlemen. 🙂
The 98% figure is not only set out in his talk, but further, briefly explored/defined. He does not, anywhere that I could find in the tape, say that he was the one who had developed the figure and its nuances. But it was clear to me, then, whoever did conduct the survey, that I had been a respondent in that survey; and I mentioned it to John Lott in passing [after his speech]. Now, I’m sure that it was John’s survey. I’ll bet the house on it. I have no reasonable doubts based on logic, evidence, and common sense.
I am certainly willing to write a letter that says, “Dear Jim, if you had only suggested the possibility of Harvard/Cambridge, I could have easily dispelled your confusion, because . . .. .”
1) Wrong year by a whole year!
2) Wrong source: I very clearly recalled the “Chicago” connection, and as both my older brothers (one of whom used to live in Highland Park at 461 Lakeside Place and worked in the loop [and I currently have a friend who lives at 471 Lakeside Place; and, by the way, my brother’s in-laws lived at 827 Bob-O-Link Road in Highland Park] WENT TO HARVARD (Cambridge, MA), I would have recalled any such connection, there, as significant, in that context and with that association. THAT didn’t happen. Interestingly, this comparison (Cambridge vs. Chicago) has never surfaced before or been brought to my attention in any way, shape, or form. I could have easily and quickly dispelled any confusion in that regard, if only because of my demonstrated ability to remember associative detail as a mnemonic aide. If Lindgren had asked me about this, I would have given him an answer that clarified HIS confusion, based on the above. I have no confusion; not possible; no way; no how. The Chicago connection is something that I CLEARLY and IMMEDIATELY and INITIALLY recalled and discussed; and the lack of any Cambridge/Harvard connection is equally as firm and solid: negative. There was no doubt about that at any time. There is no doubt, now. This is nuts.
A. And who said that a relatively locquatious (under the circumstances) respondent (me) would be cut off by the interviewer, because the last DGU was just over 24 months earlier, instead of listening, politely, and coding the information properly on the facts of the response.
3) There were no 30 questions asked; no way. It was fewer than a dozen or so. Not many at all. SHORT; VERY SHORT. I was very clear on that, always.
4) Warning shots issue. As developed as the “clincher” (in my mind) in earlier communications. End of discussion. I kept myself completely “pure” in my communications with you and in my discussions with Lindgren, I was uncoached in any fashion; unled by anyone; and undeterred by any criticism of my motives, because I had none, except the pursuit of the truth of the matter. Perhaps I was too honest, forthright, and forthcoming.
Again, there’s no doubt in my mind, based on my reconstruction of the source, timing, and number and content of the questions, that it was John Lott’s survey. I’m only sorry that I was forced to “work backwards” to the conclusion from significant intervening, associative markers so many years later. It’s how I remember things and put them into context. Even Lindgren remarked on the accuracy and detail of my memory and its associations. The truth is like that.
I’m sure that the survey to which I responded was NOT the long, 1996, Harvard survey; and any inference from the circumstances that Lindgren draws in that regard utterly fails to exclude clearly the very short 1997 study from Chicago that had a warning shots question. The circumstances of my recollection lead directly to Chicago and away from Cambridge (proper use of circumstantial evidence: he can not support even probable cause for inquiry (a search warrant) on what he has done.
Professor Joseph Olson Hamline University School of Law
tel. (651) 523-2142 St. Paul, Minnesota 55104-1284
fax. (651) 523-2236
Statement by David Gross about taking the 1997 survey, with follow-up comments by me
From: David Gross [mailto:email@example.com] Sent: Tuesday, February 11, 2003 3:00 AM To: John Lott Subject: RE: Question
Statement from David Gross:
The answer to your question is a factual one that emphasizes three things: 1) that I was the subject of a telephone survey in early 1997, the only one that I have been a respondent in; 2) that during your talk for the Center of the American Experiment in Minneapolis on January 27, 1999, you mentioned the “98% survey” of DGUs (not attributing it to yourself, at that time) in a manner that caused me to approach you and to comment that I had been a respondent in the “98% survey” that you had mentioned in your talk “a couple of years ago;” and 3) when I became aware of the controversy made over the issue of whether the “98% survey” had been, in fact conducted, I re-reminded you of my comment to you and wondering whether the survey I had responded to was the same one whose existence was, now, being questioned.
Both of those events preceded any controversy and are clear in my memory, the first event having been refreshed in 1999 and the second event being a “recording” of the first. I was not sure, at first, that the 1997 survey was in fact connected to the current controversy, but was willing to discuss my recollections and the circumstances with anyone who was willing to inquire.
James Lindgren contacted me the same evening of the day that I emailed you with my suspicions and probed my recollections in detail and in general. I was able to place the date of your talk in Minneapolis within a week of its occurrence (subsequently documented) and I gave the “details” of what I recalled to James Lindgren without the benefit of any “coaching” by you or any format or details of the challenged survey or not. I kept myself “pure” and untutored so that others could judge whether or not my recollections were sufficient to verify that it was your survey. I wanted to remain a neutral witness to the facts and not be able to be accused of manufacturing any memories. I think I accomplished that, for good or ill, along with the fact that I cannot claim a perfect, super-human memory, just a good one.
I’m still not absolutely sure of ALL of the details of the survey, but my clear recollection DOES include:
1. Its brevity: the fact that it was focused, short, and involved DGUs and something about permits to carry;
2. That it was in the evening, from a student in the Chicago area (for some reason, Northwestern University plays a part, an impression, in the memory that I have and that the student explained to me that Northwestern was in the Chicago area; but I knew that.);
3. After my conversation with James Lindgren and his questions, I recalled that it contained a questions as to whether the incident(s) had been reported and, I vaguely recalled, something about “warning shots.” The whole conversation with James Lindgren had stimulated me to think hard about what it was that I recalled, to let it come back to me as best it could. James Lindgren did not “feed” me any information on the survey, if only because I refused to accept any implicit or explicit assumptions of fact that I could not recall. The conversation with James Lindgren had already stimulated the memory of the tape, whether or not I could find it or locate a substitute.
4. My clear recollection of the comment to you about having participated in the 1997 survey (“a couple of years ago”). Even I wanted to know, to recall what revelation it was that had startled me and had motivated me to make a comment to you about having been a respondent in that survey, even though I didn’t know it was yours at the time.
5. It was the only phone survey on DGUs and such in which I have ever participated.
The rest is all about the PROCESS of association, recollection, corroboration, and verification of the circumstances.
During my conversation with James Lindgren, I recalled purchasing the tape of the talk from the Center (I mailed in a check and the tape was sent by mail) and obligated myself to attempt to find it. I know that I had not listened to it and expected to find it in its mailing envelope. Failing that, I could contact the Center to see if they had an archival copy, or the master. I was, also, absolutely clear that I had approached you, as you were leaving the talk in order to go the Minnesota Legislature and to testify on the issue. I was with Joe Olson and he can verify the fact that I did approach you, but he cannot verify the conversation “in passing.” Frankly, you were preoccupied, and I felt a little foolish making the comment, “as if you cared.” I did not attend the committee hearing. I know that I made the comment to you about my having been a participant in the survey you mentioned in your talk “a couple of years ago;” but during my conversation with James Lindgren, I was unable to “put my finger” on exactly what it was that caused me to reach that conclusion and motivate me to make the comment to you as you were being ushered to the committee hearing in St. Paul. It was some unique characteristic that you mentioned that made me sure that I had been a respondent in THAT survey you had referenced in the talk; and I wanted to know/remember what it was. The tape would either refresh my memory or it would fail to do so.
I found the tape and listened to it, making notes. When the talk got to the point where the survey you mentioned contained a distinction concerning “warning shots” as distinct from shots AT the attacker, my memory, the circumstances, and the motivation for my comment to you all “clicked.” It was the warning shot issue that caused me, in 1999, to recall the uniqueness of the survey in 1997. My memory of the survey had been refreshed/jogged in 1999, and the incident including the comment to you, in my mind, serves as a “recording” of the clear recollection I had then. My memory was much clearer, then, in 1999, being only two years removed from the survey, instead of six years; but my memory of my emotional/intellectual reaction to the sophisticated handling of the warning shot issue (Aha!), my then certainty as to the uniqueness of the survey I had taken in 1997, and the physical action and effort to make the comment to you is, today, crystal clear, even though we are four years removed from that event. Most importantly, my vague recollection of the survey’s question of the warning shots prior to listening to the tape, the talk’s drawing a distinction between warning shots and shots AT the attacker, and I remembered what it was that caused me to react and to comment with certainty, then.
I can’t help but to analyze “evidentiary” matters in the legal context, not only because that is my training and professional experience, but also because the legal context has such real, practical value as the embodiment of the human condition and the search for the “truth.” These evidentiary rules are the distillation of human experience and have real value to real people in their ordinary affairs of daily life as well as the important ones. They help one to think clearly and to guide analysis of real situations. They work well in the context of real-life, human society.
I became sure that the survey to which I had responded was yours because my January 27, 1999 reaction and comment to you serve as a “recording” of my memory of the survey from 1997, at a minimum on the point of the warning shots aspect of the survey. I recalled enough “details” independently (short, Chicago, student, DGU, carry laws/permit, reported, and vaguely, warning shots) that the timing (which fact I supplied in 1999 and which was not mentioned in the talk) and the warning shots issue nail it down dramatically. In the absence of another survey going on with similar features, the rule of circumstantial evidence allows one to draw only one conclusion, which I unabashedly do, especially when combined with my direct memory: circumstantial evidence supports a finding of fact when it directly points to/allows the inference of the existence of a fact and, at the same time, reasonably excludes the existence of a contrary inference or conclusion.
Where is that other survey I could have been a respondent in? Even James Lindgren suggested to me that, in the absence of another survey with which I could have confused the two, it was clear that I had been a respondent in your survey.
If you look at my initial email to you, you’ll see that my reaction and comment to you in 1999 were a main component of my willingness to come forward and possibly to shed some light on the matter. All the rest has been confirmation of the unique details, of most of which I was unaware until after I had exhausted my ability to remember/recall.
Now, some people have suggested that I have come forward in order to “cover” for you and that I am biased and willing to lie for you. I mean no offense; but John, I don’t know you that well. I don’t believe that I know anybody that well. Even though I follow the rules of evidence (and common sense) in my daily life in evaluating the credibility of people with whom I come into contact, it has become apparent to me that there is a dearth of common sense, analysis, or respect for others’ integrity in the “politically correct” world out there. Even academia, where proven, valuable standards are supposed to be rigorously maintained by those trained and experienced in them, appears not to be immune from political pressures and the consequent lowering of standards. This is how thought control and fascism gain a foothold. That will be the community’s loss comparable to the loss of the library at Alexandria.
The fact that I confirm the survey in 1997 has nothing to do with the validity of the conclusions you drew from it, the propriety of your use of it, under the circumstances, or its sufficiency. Those matters are pretty much beyond my expertise, although what I recall of it from the talk in Minneapolis seemed powerfully and intuitively reasonable and accurate, to me, based upon my understanding and experience, as well as from other research.
Note from John Lott: I know of no other self-defense surveys on guns that were done in 1997. Hemenway et. al. did a survey in the Spring of 1996 and Spring of 1999, but both of those surveys are not only in different years but also quite different than the survey described by Mr. Gross. All the other surveys have been done by professional polling services, not students. Hemenway et. al.’s polling firm was based in Albany, New York, while my survey was based out of Chicago. It is not clear how many questions were asked by the Hemenway et. al.’s survey since it is not stated how many questions were asked before the respondents discussed if they had used a gun defensively. After two questions identifying who has used a gun, up to an additional 30 open ended questions were asked. It is not possible for that survey to have been completed in the short period of time discussed by Gross. Also, after the 1994 National Survey of Private Ownership of Firearms National Survey of Private Ownership of Firearms, there were no other surveys that specifically asked respondents about “warning shots.”
5) Supplying my tax returns
I have also provided my tax returns for 1997 to several academics. While we did not retain the itemized list of deductions past the IRS three year requirement, the returns indicate that I had spent a large amount of money on RAs that year. John Whitley was the only other RA that I had hired that year and the amount that I paid him was just a tiny fraction of the total amount that I spent that year on RAs.
A) The point of using the survey in More Guns, Less Crime was to deal with the issue of media bias (it is exactly the same point raised in The Bias Against Guns). The point was that the lack of coverage of defense gun uses is understandable if most uses simply involve brandishing where no one is harmed, no shots fired, no dead bodies on the ground, no crime actually committed. If others believe that the actual rate of brandishing is lower and I had used the results of Kleck, it becomes MORE difficult to explain the lack of news coverage of defensive gun uses. The two short discussions that I have on this issue in my two books thus choose results that are BIASED AGAINST the overall point that I am making, that the media is biased against guns.
[It isn’t more difficult to explain the lack of news coverage with a lower brandishing number. A lower brandishing number merely suggests that defensive gun use isn’t as common as Lott claims it is.]
B) Compared to the 98 percent number there was an earlier survey by Kleck that found 92 percent of defensive gun uses involved brandishing and warning shots and because the survey was asking people about events that occurred over a long period of time it is likely that it over emphasized more dramatic responses. (My number that is directly comparable to the 92 percent estimate is about 99 percent.) My point in the book was that defensive gun use rarely involves more “newsworthy” events where the attacker is killed and either survey would have made the general point. I have a discussion that explains why his surveys get lower results here.
[Lott fails to mention that the number from Kleck’s survey that is comparable to his 98% is 76%. That means that people in Kleck’s survey fired their guns ten times as often. Nor does the time frame for Kleck’s survey make a difference. If you just look at DGUs within the past year in Kleck’s survey, you also get 76% brandishing. And the NCVS which uses a six month recall frame gets an even lower brandishing precentage than Kleck.]
C) The point of science is replication. I redid the survey in 2002 and got similar results.