survey
tlambert | April 25, 2003Donald Kennedy, Editor-in-Chief of Science has an editorial (subscription required) in the April 18 edition entitled "Research Fraud and Public Policy". Here is some of it: Michael Bellesiles, of Emory University, supported the gun control case with a book called Arming America. Part of his argument was that guns were rare at much earlier times in U.S. history. Challenged on that claim, he failed to produce the data, claiming that an office flood had destroyed his records. Emory empaneled a committee of scholars to investigate, and its report questioned Bellesiles "scholarly…
tlambert | April 25, 2003A few days ago I observed that Lott had changed his story from his original, unworkable, claim that he had used 1836 categories (sex, race, age and state) to weight his data to the claim that he had used just six (sex and race). If this is indeed the scheme he used then two things follow:
He has incorrectly calculated the brandishing number for his 2002 survey.
It is impossible for him to have obtained a 98% brandishing number for his 1997 survey. Here are the details:
In his new book, Lott tells us the brandishing number he gets after weighting:"the survey I…
tlambert | April 22, 2003Kevin Drum comments on Lott's weighting scheme. He also links to a January posting which has this explanation from Lott explaining how he might have got a weight of 1/8 from his weighting procedure (my emphasis). Whether it is possible depends upon how finely you do the weighting. If you do something as simple as national weighting, you are right, it would not be likely. But if you are willing to put in the effort to break things down into enough categories it becomes quite likely. I just looked up some different numbers from 2000 to give you a rough idea.…
tlambert | April 21, 2003 Last September Lott told Lindgren that he "weighted his respondents by demographic information taken from his main national study in More Guns, Less Crime"
On January 14 he provided more details: I did not weight the sample by household size but used the state level age, race, and sex data that I had used in the rest of my book. There where 36 categories by state. Lindgren hypotheses why you can get such small weights for some people and I think that this fine of a breakdown easily explains it. I don't remember who answered what after all these years, but suppose someone who fired a gun…
tlambert | April 19, 2003Otis Dudley Duncan has written an excellent article on Lott and defensive gun use surveys. I'll quote from the conclusion, but you really should read the whole thing:
Investigators are obliged to tell the truth about what they take from the work of other investigators and to provide verifiable evidence and complete documentation for statements made in reports on their own research. They are responsible for telling the “whole truth” about it, to use the legal phraseology, and for enabling others to confirm or falsify their results. As far as his claim about the evidence on gun…
tlambert | April 7, 2003After I concluded yesterday that Kopel had probably added the attribution to Kleck in one Lott op-ed, Lott has weighed in, contradicting Kopel's story. In this posting Lott writes:"My vague recollection of what happened is that David Kopel (Research Director at the Independence Institute) called me up asking for more information on who had done self-defense surveys and I mentioned that among them was Gary Kleck."
This is contradicted by Kopel's account of what he thinks happened (see yesterday for a summary). It is ridiculous to suppose that Kopel would have needed to ask Lott…
tlambert | April 6, 2003So, was the attribution of the 98% to Kleck's study in the Lott quote below made by Lott, or did Dave Kopel add it? "Guns clearly deter criminals, with Americans using guns defensively over 2 million times each year---five times more frequently than the 430,000 times guns were used to commit crimes in 1997, according to research by Florida State University criminologist Gary Kleck. Kleck's study of defensive gun uses found that ninety-eight percent of the time simply brandishing the weapon is sufficient to stop an attack."
Our first piece of evidence is Kopel's recollections…
tlambert | April 3, 2003Julian Sanchez suggests that if Lott really got the 98% from his survey, then by marrying the 2.5 million Kleck DGU estimate with the 98% brandishing number, Lott is indulging in cherry-picking the numbers most favourable to his position from different surveys. Well, in this case I don't think that Lott is cherry picking. In statements before May 1999, Lott would say that there were 2.5 million DGUs (Kleck's DGU number) with 98% of them involving brandishing. After May 1999 (which was when he first claimed that the 98% came from his own survey) Lott switched to…
tlambert | April 2, 2003Julian Sanchez has another thoughtful post on the question of whether it was Lott or Kopel who attributed the 98% to Kleck. I'm still trying to collect my thoughts on this one, but I should correct one statement he makes. Even if the attribution is established to be Kopel's it does not follow that Lott did not get the 98% by misreading Kleck. Lott got the defensive gun use numbers 2.5 million, 760,000, and 3.6 million,from Kleck but never attributed them to him in his public statements.
tlambert | April 1, 2003Pro-gun activist Neal Knox has leaped to Lott's defence. He claims that Lott is in trouble for getting the same result in his survey as Kleck when in fact concerns were raised because Lott's brandishing number was so very different from Kleck's. I was going to do a detailed dissection of Knox's claims, but gzuckier beat me to it.
Mac Diva does not believe the story about the Independence Institute editing Lott's article. I will post more on this tomorrow.
tlambert | March 29, 2003One feature of Lott's behaviour in this affair is his refusal to admit that he attributed the 98% figure to "national surveys" and to Gary Kleck. Instead, he told Slate "A lot of those discussions could have been written more clearly."
However, in on-line publications by the Independence Institute and the Heartland Institute he wrote: "Kleck's study of defensive gun uses found that ninety-eight percent of the time simply brandishing the weapon is sufficient to stop an attack."
This isn't the sightest bit unclear. He is attributing the 98% to Kleck. How…
tlambert | March 28, 2003Meanwhile, CNSNews.com seems to be blissfully unaware of the Lott affair, with this story reporting: "Surveys Lott conducted in January 1997 indicated that guns are used more than two million times a year in self-defense, either by threatening to use a gun, brandishing it, firing a warning shot or actually shooting a criminal."
Mitch Berg defends Lott against Laura Billing's criticism:
For instance, critics of his have long wondered where he came across a "national survey" cited in his book claiming that "98 percent of the time people use guns…
tlambert | March 23, 2003The Washington Post has printed a letter from Lott responding to two Washington Post articles, one about his survey, and one about Mary Rosh. Lott makes several false claims in his letter:
that the Post did not print a letter from "an academic who wanted to correct a statement attributed to him that was the opposite of what he had written." You can check the two articles and see for yourself that the only statements attributed to an academic were those attributed to Lott, and he has not disputed those ones.
"Academics have confirmed ... discussions that I had back in 1996 and…
tlambert | March 18, 2003 ArchPundit has a post and another post on Lott's new survey. He argues that Lott has replicated his previous survey---by replicating a worthless survey with another worthless survey.
tlambert | March 13, 2003Lott's new book has been published. This means that I can disclose the results of his 2002 survey. In that survey, 7 people said that they had used a gun for self defence. Of those 7 people, only one reported firing the gun (in fact, that person reported wounding the attacker). This means that a 95% confidence interval for the percentage firing is 3%–50% (calculated using Wilson's method). This confirms what I suggested earlier---the number of defensive gun users in Lott's 2002 survey is far too small for this survey to give any useful information about…
tlambert | March 8, 2003David Mustard has made a statement giving his recollections about when Lott told him about the survey: I do not remember the first time John Lott and I talked about the survey. At the time there was nothing exceptional about the survey for me to associate with it and help me remember when I first learned about it.
I believe it likely that John informed me of the completed survey in 1997. I think it highly probable that John told me he had completed the survey at the time of my talk at the Academics for the Second Amendment conference in Washington, DC in November 1998.
I know beyond a…
tlambert | March 2, 2003Lott makes it into the Sunday comics. (Thanks to Julian Sanchez for the link.)
tlambert | March 1, 2003Glenn Reynolds has an update with comments from Dan Polsby who writes: Numerous of Lott's opponents (John Donohue, Ian Ayres, Phil Cook, Jens Ludwig, and many others) use the Lott-Mustard numbers, subsequently updated by Lott, in their work because they have to
However, because of Lott's dishonesty, it will now be necessary to check all of Lott's data for accuracy.
tlambert | February 28, 2003Glenn Reynolds links to a Richmond Times-Dispatch editorial that is very critical of Lott.