Lott calls me "extremely dishonest"

In an email to a poster at The High Road forum Lott writes

"The actual data has been available on one of my websites at www.johnlott.org since February 2003. The Appendix of my book, The Bias Against Guns, goes through and discusses the data in depth. I talk about how the survey was done, the questions used in the survey, who did the survey, how it was weighted, etc. there. The www.johnlott.org website also has some downloads discussing the survey debate in general.

On this last point, Lambert has been extremely dishonest. For example, he has a long list of surveys but he lists the date for them as the mid 1990s when that was just when a particular paper cites them as opposed to when virtually all of them were done primarily in the early 1980s or earlier.

Let's look at the list:

Survey Percent firing Source
Kleck 24 Kleck 1995
NSPOF 27 Duncan 2000<.phpa>
NCVS 1987-1990 28 Duncan 2000
NCVS 1987-1992 38 Rand 1994
NCVS 1992-2001 21 NCVS online analysis system
Field 34 Kleck 1995
Cambridge Reports 67 Kleck 1995
DMIa 40 Kleck 1995
Ohio 40 Kleck 1995

Apparently Lott is arguing that I gave the cite for "NCVS 1987-1992" as "Rand 1994" in order to trick people into believing that NCVS 1987-1992 was conducted in 1994. Well, no, that was not my intent. I gave the publication year because that is the convention when citing sources.

Furthermore it is not true that "virtually all of them were done primarily in the early 1980s or earlier." Only the last four of the nine in the table date date from the 80s or earlier. I don't think that it is correct to call four out of nine "virtually all".

This is, incidently, the first time that Lott has admitted the existence of this list that shows how much his 2% (and now 5%) firing figure differs from all other results. Presumably his remark about virtually all of the surveys being from the 80s or earlier is an attempt to dismiss them as being out of date. However, the ones from the 90s, such as NSPOF, Kleck and NCVS all contradict the numbers he claims come from his surveys. To this day, Lott continues to advance his 95% brandishing number, never admitting to the existence of contradictory research.

Tags

More like this

I am not sure why you spend so much time trying to correct people like Gunstar1, GEM, and others like them. Their position has blinded him to facts and nothing will change it. Like Clayton Cramer, they will support Lott no matter what and regardless of facts. I think the Lott/Bellesiles comparison is interesting here. In the case of Bellesiles, people who supported his research unequivocally condemned his behaviour when the fraud was evident. Lott's supporters continue to support him even after multiple instances of him engaging in behaviour that amounts to academic fraud.

By Carl Jarrett (not verified) on 08 Dec 2004 #permalink

Research Misconduct Defined

Research misconduct is defined as fabrication, falsification, plagiarism, or other serious deviation from accepted practices in proposing, performing, or reviewing research, or in reporting research results.

Fabrication is making up data or results and recording or reporting them.

Falsification is manipulating research materials, equipment, or processes, or changing or omitting data or results such that the research is not accurately represented in the research record.

http://www.upenn.edu/almanac/v49pdf/030506/misconduct.pdf

By Carl Jarrett (not verified) on 08 Dec 2004 #permalink

And he also says

".....actually read what I wrote on the brandishing issue. Here is the quote from my book:

Even though the survey I conducted during the fall 2002 indicates that simply brandishing a gun successfully stops crimes 95 percent of the time that guns are used defensively and other surveys have also found high rates, it is very rare to see such a story. No conspiracy explanation is really needed to explain why an editor finds a dead body
on the ground very newsworthy (particularly if it is a sympathetic person like a victim). Take a story in which a woman brandishes a gun and a criminal flees, no shots are fired, no crime is committed, and no one is even sure what crime would have been committed had a weapon not been drawn. Nothing bad actually happened. It is not emotionally gripping enough to make the story "newsworthy."
____________
The point here is a simple one.
I want to show that the media is biased. Therefore a lower percent of brandishing would make my case stronger. Instead with the 95 percent figure I was providing an explanation for why the media doesn't cover a lot of cases. If he believes that the 95 percent number is too high, the results are BIASED against my claim."

This seems not simple but incoherent. It's certainly interesting that he says he wants to show media bias, rather than wanting to establish whether or not there was bias, but his argument is actually that
(a) brandishing is underreported
(b) because of the inherently incident-loving nature of news reporting.

To this end, a finding that (say) 60% of gun incidents (those with only brandishing) made up only 5% of news reports would mean that shooting incidents were overreported by 95/40 or about 2 1/2 times; a finding that 95% of gun incidents made up only 5% of reports would mean that shooting incidents were overreported by 95/5 or about 19 times, which is surely more bias and thus more in favour of what he wants to prove.

By Chris Borthwick (not verified) on 08 Dec 2004 #permalink

So Tim, are you planning to apply equal rigour to the 'public health approach' exemplified by Professor Ozanne-Smith et al. 2004 'Firearm Related Deaths: the Impact of Regulatory Reform'?

I am encouraged by your blog to think your ethic is to encourage quality thinking and research, debunking work that tips toward propaganda. It is a high standard. Do you only take on the enemies of the pc consensus?

Name one country in the post WWII era where guns were banned or made more difficult for law abiding citizens to obtain that violent crime went down? That crime went down?
What about characteristic about about a gun, handgun or rifle, that makes an other wise law abiding citizen want to commit a most serious crime that is not a characteristic of a knife or a baseball bat or any other inanimate object that can be used as a weapon?

By Terry Josiah (not verified) on 08 Dec 2004 #permalink

When Lott misrepresents 12/13 as 95%, he is not 'just rounding up', and he and his supporters are being grossly mendacious on this point. The difference between 12/13 and 95% is effectively the difference between 1/13 and 1/20. Which is about 50%. He's overstating his result by more than 50%, and relying on an innumerate audience not to figure this out.

That's before we get into whether his 1/13 result has much statistical significance (which of course it doesn't). I surveyed the names of people who work for my company, and only two out of six have a name other than "Nick" or "Richard".

By Nick Barnes (not verified) on 08 Dec 2004 #permalink

Terry, that's a whole mess o' misrepresentation you've got going there. First off, (credible) gun-control advocates don't believe getting rid of guns will reduce crimes involving knives, they believe it'll reduce crimes involving guns. You might say that's the whole bloody point.
Secondly, whether the anti-gunners are correct or not, Tim Lambert has repeatedly come out against gun control, so there's no point attacking him with "gun control sucks, ner ner"-type arguments. He's not saying Lott's wrong, just that Lott is a liar, and that that's a Bad Thing. This isn't --- or shouldn't be --- a controversial position.

What Carl and Mark said, with the extra addition that the UK and Australia implemented their first gun control laws in the 1920s and 30s. So looking at the rate of gun crime post-WWII in such countries doesn't tell you anything.

Terry, Tim Lambert is not "anti-gun" and your comment is off-target, as mark noted.

As to your question, try Canada. Violent crime is lower there, in part because of stringent gun control laws. Finland and New Zealand are also examples as well.

As for the characteristics of guns, why do gun advocates want to make it easier to use guns in defensive situations for protection? Because guns are more effective than other weapons. That same characteristic also makes it easier for people to use guns offensively. Making it easier to own guns makes it easier for criminals to use them.

By Carl Jarrett (not verified) on 09 Dec 2004 #permalink

ChrisPer, why don't you tell us what problems you have with the Ozanne-Smith et al. 2004 article? Did the authors of that article report a statistic for which they lack data? Did the authors of that article change their models without reporting it after correcting coding errors?

By Carl Jarrett (not verified) on 09 Dec 2004 #permalink

Well, I wanted Tim to take this on. My own far from humble view:

1) The paper has lots of 'authority wrapping' in titles of authors, institutions and publications. You have a right to trust their word on the face of it.:

2) The paper comes from the 'public health approach' to guns, which has several political implications the most significant of which are (1) that activists dress their stuff in the trappings of authentic scientific research and (2)the responsibility of indiviuals for their actions is downplayed, both in causation and prevention of harms.

3)The maths are a little dense for ordinary people. I trust them and don't quibble, because I don't have the background. From what I can decipher, they are unexceptionable except that they may mislead people to think that the underlying design is just as rigorous.

4) The underlying idea of using inter-state variation for time series control is good. This topic needs innovative measurement instead of the political approach of headlining any supportive stats. However, they totally ignore the case of Western Australia which has essentially been at very high levels of control for many decades.

5) They set themselves up for a pre-conceived win by saying they exclude method substitution, yet working with the statistic 'gun deaths', which are dominated by suicides and which have risen despite less gun suicides. This is so apparent on the most cursory reading of the literature that I must conclude that these people are deliberately setting the design to win PC credit points among their peers, not to do science.

But maybe I am wrong; what do you think?

I wrote a response to Terry above saying more or less what Mark and Carl said. I do think that if you guys are not just 'shooting back', you might note Terry has two points:

1) The places where strong control actions have occurred do not have strong evidence of results; as Tim has previously pointed out, the evidence is that they have little effect.

2) There is indeed an assumption in pro-control thinking that firearms have some causative property. It appears they change the thinking of people to become violent and 'attract' mental defectives who do bad stuff like voting Bush or joining the NRA.

I don't know about you all but this whole antagonistic gun politics thing has made me so sick after eight years.

Chris, the main weakness in the study is that it only considered gun deaths and didn't try to measure substitution effects. The use of a control is a good design. I don't think speculation about the motives of the authors is productive.

I should note that I think that the gun laws here pre-Howard were a good idea and one of the reasons why Australia has a much lower homicide rate than the US. That is not to say that they work if adopted in the US.

And should I presume that a hard look at studies like Mouzos and Peters or Ozanne-Smith et al, which have been publicly used as 'proof' of the value of the confiscations is not of interest to you?

And I clarify that I meant data from non-imaginary surveys only.

Let me get this straight. Kleck didn't do the DMIa or Ohio or Cambridge Surveys and Duncan didn't do the NSPOF survey and these surveys were done many years before the 1995 and 2000 dates listed in the Table. When exactly was the Ohio or DMIa or Cambridge Surveys done? Were they done in 1993 or 1994 or done in the 1970s or 1980s?
Chris Borthwick also apparently misses the point. If you find a result, you want to make sure that it isn't a false positive. If there are biases against that result, that makes the result more believable.

Tim, thanks for your response! I will endeavor to restrict speculation on motives in future. Motives are however part of the picture of why people either falsify or skew research.

Our local gun laws pre-Howard were very diverse in respect of longarms. I come from Western Australia, which has always had strong control and low murder rates. The NT had moderate to strong control and very high murder rates through the same period.

I would like to see a study like the Ozark-Smith one which separated legal environments, controlled for vulnerable populations and included substitution.

I am also interested to work with the contagion effect and its implications for school killings, mass killings and terrorism. I think the media hype around the Australian confiscations may have produced a reverse contagion effect, a possible explanation of the complete absence of mass shooting incidents > 4 victims since 1996.

...and so the debate moves away from Lott, without a sliver of acknowledgement from the pro-gunners that he might actually be a liar, to some other stuff. as for:

"Motives are however part of the picture of why people either falsify or skew research."

I think anyone who has read Lott's work would agree with you on that one ChrisPer.

Agricola, I have read some of Lott's work and Tim's critiques of it. Tim does the job, and as a result I do not cite Lott anymore. The Mary Rosh thing I found quite pathetic, and credit to Tim for the investigation. However you seem to imply that proving he has lied about the survey destroys what his data showed.

I note that the data from Lott's 'More guns less crime' has been dissected (eg by Tim) shows the claimed lessening of crime from CCW fairly doubtful. This is used to overshadow the probably more robust result that there may be no significant negative effect either.

Have you read 'The Bias Against Guns'? It certainly seems to be true that mainstream media filter and distort the facts about gun violence in a very anti-scientific way. Australia's media are more homogeneous than America's on this one; we are a pretty narrow-minded lot here ;-).

So Agricola, are you poisoning the well here? The objective integrity or reputation of the author does not bear on the truth of his assertions, but whether you should rely on them without checking. John Lott's assertions have been so thoroughly checked by the opposition that we are quite able to be confident in which are true or not true.

Sadly, political anti-gun statements are almost never true.

BTW Agricola, does that handle mean you are a fan of "de re metallica"?

ChrisPer, Lott's 'bias against guns' is yet more sloppy research. Did you ever wonder why Lott didn't get any of that published in a reviewed journal? Did it ever occur to you that Lott's 'bias against guns' research would not have survived peer reivew? As for your critique of Ozanne-Smith et al. 2004, your items #1-4 are not valid. Pretty much any reviewed journal research is produced by 'authorities' who mention their credentials. You present no evidence to back up your claim that the research is by activists. It's a research journal, the statistics are supposed to be there. To complain that "The maths are a little dense for ordinary people." is ridiculous. Are you going to complain that Kleck's 'math' or Lott's 'math' is too dense for ordinary people? Your only critique that comes close is #5 but even there you are off. As Tim Lambert mentions, a potential flaw is that they don't go beyond guns and don't look at method substitution but then they don't make claims that go beyond the scope of their research. They don't claim that all violent crime has decreased. Your claim that there is "an assumption in pro-control thinking that firearms have some causative property" is false and is a standard strawman used by the pro-gun side. If you can produce a quote by a gun control advocate where they claim that guns act independently of human intervention, feel free to do so.

By Carl Jarrett (not verified) on 10 Dec 2004 #permalink

Carl, I am not here to give a scholarly critique, I made it plain I do not have the qualifications for that. Nevertheless I have a few things to say. Your points sound very angry; I am not rebutting all their work in detail; I am saying what I dislike and like about it. The fact is they used a cheap shot as the foundation and that discredits the rest.

The work should be re-done, controlloing for method substitution. The parallel control idea is good, but needs to also control for some other items and perhaps work with mechanisms. So much anti-gun scholarship talks about the effect of tightening as though it is a continuum; but a mechanism that restricts one aspect cannot be thought to control independent ones. If it appears to, like the mysterious dearth of mass shootings since 1996, we should be asking why it works, when its proponents denied that would be the effect.

ChrisPer - If you see 'anger' in the response, it is entirely yours. Ozanne-Smith et al. 2004 did not 'use a cheap shot as the foundation' and they don't need to redo it by including method substitution. Your comment about the 'cheap shot' demonstrates that you have failed on your promise to "restrict speculation on motives in future." as does your attempt to project anger into my writings. Had Ozanne-Smith et al. 2004 made claims about overall violent crime while only focusing on gun-related violent crime, then not including method substitution would be a valid critique. But they don't make claims about overall violent crime, the correctly and consistently talk only about gun-related violent crime.

By Carl Jarrett (not verified) on 10 Dec 2004 #permalink

ChrisPer,

If he lies about the survey, then his data are part and parcel of that lie, and are as a result meaningless. Bellisiles wasnt castigated because he lied and was wrong, he was thrown out because he lied. When will the same treatment be accorded Lott?

Carl, I have one peer-reviewed paper long ago and I have a small idea what you mean. Actually, my 'peers' were a lot better than me, being PhD students or profs when I was an Honours student!

By all means call me dishonest; just back it up. I am willing to attenuate my criticism of 'public health researchers' with the possibility that they are acting out of noble motives; no doubt they are all very sincere and concerned people, with just the right ideas. You can say the same of Mr Bellesiles; but the results are still not honest. And when direct personal harms occur because research is misused by media and politicians, I think we are entitled to look into the quality even if we are not in the academic club.

I suggest that the 'public health approach' results in dishonesty, because it puts activist utility of results as a higher value than fairness, because the participants in the debate operate from a very one-sided set of assumptions, and yet the positional authority of the institutions and individuals is misused to validate deliberately politicised research.

Peer review in such circumstances is perhaps like peer review in Lysenkoist genetics or the more current activist-based schools of study; it is weakened to the point of destruction by the narrowness of shared assumptions.

Professor Ozanne-Smith et al. have not covered themselves in glory. I am sure the punctuation on their reference list meets the highest standards of the journal, but their research design is disappointing. If peer review had been adequate the research would not have that flaw. It is likely that peer review pointed it out, but rather than do the work properly (and produce a less-newsworthy bottom line), the good prof just inserted her disclaimer excluding substitution from the scope. Given the ease of re-runnign computer processing, I would not be surprised if they actually did try that control, but dropped it because the results were not meaningful enough to make the publishable point they wanted. If that were true...

The 'false bifurcation', an incomplete specification, I accept. I left out the third possibility that they honestly thought that the statistic was meaningful. If so, without control for substitution in suicide they were wrong in a very obvious way. Why don't you privately contact Jenny Mouzos at the AIC and ask what she thinks of the design?

Agricola, you are correct that data of a fabricated survey are part of the lie and should be rejected. I realised what I had said was wrong in that sense, and added the wee post underneath saying that I was referring to data from non-imaginary surveys.

You assert that "Had Ozanne-Smith et al. 2004 made claims about overall violent crime while only focusing on gun-related violent crime, then not including method substitution would be a valid critique. But they don't make claims about overall violent crime, the correctly and consistently talk only about gun-related violent crime."

Suicide is not violent crime. Suicide is not a crime at all. It is self-harm, and very serious, and it forms from 2/3 to 3/4 of gun-related deaths, and the bulk of the deaths they are talking about are therefore not violent 'crime' at all, by definition.

Let's look at the concept of 'Gun-related deaths'. If we were talking about garrotting murders, would you consider it appropriate that statistics include legal hangings, industrial accidents resulting in strangulation, seat belt strangulation in car accidents and the many, many suicide hangings?

'Gun-related deaths' is term of art, an activist propaganda term which lets activists frame debate as though suicides were deaths caused by access to guns. However, suicides are freely substituted in method. For instance after the 1996 confiscations, gun suicides fell sharply but non-gun suicides rose to compensate. If there were no confounding influences, we could be confident NO LIVES WERE SAVED. Yet Professor Ozanne-Smith et al. used their disingenuous statistic and screwed up what could have been a good study. I LOVE the idea of cross-state control but it seems they either couldn't be bothered designing it carefully, or they deliberately chose to work with a design that started from a propaganda statistic.

Only if they control for substitution is the stat 'gun deaths prevented' in any way meaningful.

Chrisper, your comment "it seems they either couldn't be bothered designing it carefully, or they deliberately chose to work with a design that started from a propaganda statistic" is a dishonest false bifurcation. Ozanne-Smith et al. 2004 did neither and you continue to speculate about motives.

If you have not published anything in a reviewed journal, you have no clue how difficult it is to satisfy the reviewers.

By Carl Jarrett (not verified) on 10 Dec 2004 #permalink

Carl, I think it is wrong for Chrisper to attack the motives of the researchers, but you also should not attack Chrisper's motives (by calling his attack "dishonest").

Aw, Tim, you're no fun some times... I have little patience for scientificaly illiterate gun advocates. The peer review was perfectly acceptable on Ozanne-Smith et al. 2004. BTW, Chris, a reviewed journal article is not a undergrad honors paper being reviewed by grad students and professors.

By Carl Jarrett (not verified) on 10 Dec 2004 #permalink

Carl, thanks for the update! I would be entertained to know what discount on our civil rights should apply for scientificaly(sic) illiterate people. For example, should we be subject to warrantless search and seizure? How about the presumption of innocence, or strict liability? Or the right to fair hearing in the public forum? The right to be judged on our true safety record instead of the record of criminals, filtered through the peer-reviewed fairness of today's media? These seem good things to take from people who don't have refereed papers in scientific journals.

Tim, I have been hogging your thread. I will pull my head in, should write my own blog. Thanks for the hospitality!

Projection, projection. I am not talking about Lott or excusing anything he has done or said; I am also not denying the NEED of peer review - except I deny that it is some magic wand of perfection. I used this thread to try to suggest our host discuss flaws in a piece of propaganda-enhanced research. Tim has done so to the extent that we can say he agrees it has at least one flaw.

Now you're flying off the deep end, Chris. When John Lott publishes his, ahem, "research", he's not sticking up for gun advocates. He's lying through his teeth for political gain.
The whole point about peer review is whether someone's arguments are valid, whether their research is correct, whether their numeracy ought to be questions, and so on. I'm not going to argue whether gun control is right or wrong; the plain fact of the matter is, in this case, it doesn't matter, because Lott has been caught lying repeatedly. What use is it to stamp your foot and say "but, but, but, civil rights!" (I'm guessing you're American?)? How does that change the essential fiction of Lott's work, or the need for peer review?

Nothing really changes, mark. When gun advocates are confronted with errors, inconsistencies, and problems, they act like Chris. They jump up and down and try to point at something else in the hopes that nobody will notice their errors. Even if the Ozanne-Smith et al. 2004 article had massive flaws (which it does not), it would not justify nor excuse the research fraud committed by Lott.

By Carl Jarrett (not verified) on 13 Dec 2004 #permalink

Yes, Chris, you are projecting. You should read what Tim Lambert wrote about the Ozanne-Smith et al. 2004 article. He does not state that there is a flaw, he notes "the main weakness". YOu also continue to dishonestly (his continued slurs on the study are dishonest, Tim) describe the research. You latest slur of "propaganda-enhanced research" means that you continue to infer motives even after you stated you would not do so. That also is dishonest.

By Carl Jarrett (not verified) on 13 Dec 2004 #permalink

Carl, I realize that ChrisPer sets himself up for attacks on his motives by attacking the motives of the researchers, but please do not call other commenters "dishonest". If you want to make a case that they are dishonest, just present the evidence and let the other readers draw their own conclusions.

After reading comments by ChrisPer elsewhere, I realized that I was wrong about calling his posts dishonest. It assumes a level of intellectual ability that he appears to lack, given the consistent pattern of incorrectly interpreting and spinning what other people write.

By Carl Jarrett (not verified) on 13 Dec 2004 #permalink

Carl, merci du compliment!

Interestingly, Professor Ozanne-Smith has previously had contact with the media using gun statistics for maximum impact, when suicides were confounded with accidents in a news report on her work:

Herald Sun, Friday July 30, 1999
STILL WAY OFF TARGET

I appreciate the letter from Professor Joan Ozanne-Smith (Herald Sun, May 24) regarding the misrepresentation of figures in a report by Monash University Accident Research Centre (Herald Sun, May 6).

Unfortunately, her letter failed to qualify the significant extent of the error. The Accident Research Centre originally claimed there were 632 accidental deaths from the shooting sports in Victoria from 1989 to 1995.

The centre has since admitted the correct figure is just four.

This compares with 106 deaths from bicycles and 274 from trams and trains over the same period.

None of those four deaths occurred on approved target shooting
facilities, so it is clear from the centre's own subsequent admissions
that the target sports are significantly safer than readers were
originally led to believe.

Christopher Hicks (no relation to ChrisPer)
Target Rifle Victoria

Until AEI does an investigation that Jim Lingren called for previously, I do not think it is appropriate to cite Lott's work or to use any of his conclusions. I have a vested interest that his conclusions are supported from the data, but an investigation is the only way the situation can be resolved, one way or another.