In a post on his blog Keith Burgess-Jackson wrote:

First, studies by law professor John Lott and others show that private gun-ownership reduces crime rates. This may be counterintuitive, but it’s true. There would be more crime than there is if guns were banned.

In an attempt to set him straight, I emailed him and pointed out that Lott’s studies had been refuted by better and more extensive work by Ayres and Donohue and gave him a link to my comments on Lott. Instead of responding to any of the points I made, he replied:

You sound like a gun-hater.

I wrote back: “You are mistaken. Do you care whether your claims are true or not?” Burgess-Jackson replied:

I have more faith in John Lott than I do in you, that’s for sure.

In an article called “More Statistics, Less Persuasion: A Cultural Theory of Gun Risk Perceptions”, published in the University of Pennsylvania Law Review, Dan Kahan and Donald Braman argue that the whole debate between Lott and his critics is a waste of time because most people behave like Burgess-Jackson towards empirical evidence on gun issues. Kahan and Braman write:

individuals can be expected to credit or dismiss empirical evidence on gun control risks depending on whether it coheres or conflicts with their cultural values.

The same issue also has five commentaries on their paper and a response from the authors. In those commentaries, Lott and two of his critics (Cook and Ludwig) actually agree on something: that Kahan and Braman are wrong

Anyway, my take on all this: Kahan and Braman are wrong, but so is Lott. For details, read on.

The empirical evidence for Kahan and Braman’s claim is an analysis of GSS data where they find that cultural values (for example, whether a person values individualism more or less) have a statistically significant correlation with whether that person supports gun control. They conclude that all the statistical analysis by Lott and his critics is a waste of time:

Rather than focusing on quantifying the impact of gun control laws on crime, then, academics and others who want to contribute to resolving the gun debate should dedicate themselves to constructing a new expressive idiom that will allow citizens to debate the cultural issues that divide them in an open and constructive way.

I believe that their conclusion is in error—just because cultural values affect attitudes towards gun control it does not follow at all that information about the costs and benefits of gun control would not also affect attitudes. There is also something curiously self-defeating about using statistics to persuade you that statistics can’t persuade you. If you are persuaded by their statistics then that proves them wrong. (Of course, they come back with: “You weren’t persuaded by our statistics. That proves us right!”)

The first commentary, by Cook and Ludwig, pretty much says what I think—that showing that cultural values matter doesn’t show that other things don’t matter as well.

The next one, by Fremling and Lott argues that Kahan and Braman’s results really show that cultural values don’t matter. They observe that adding variables for cultural values in the model increases r2 (What is r2?) by only 1.6 percentage points, and argue:

Thus, Kahan and Braman prove the opposite of what they intended. Instead of demonstrating that people’s views of social order explain a lot of the variation in positions on gun control, they show that these views matter very little. …

It is very interesting that Kahan and Braman find that attitudes on guns are so little explained by attitudes in other areas. …

We think Kahan and Braman should revise their conclusion and take credit for this interesting finding.

I think that Fremling and Lott’s argument makes a fetish out of r2, but let’s accept it and see where it takes us. How much does including the variable for carry laws increase r2 in Lott’s “More Guns, Less Crime” models? In his famous Table 3a Lott reports r2 values for three different models involving carry laws, but he doesn’t tell us what the r2 values are if you don’t include any variable for carry laws. If the amount of increase in r2 is as important as he claims above, it is rather odd that he did not report this interesting statistic. No matter, I have the numbers here. The carry law makes no noticeable change to r2 for almost all models—in a few instances it increases r2 by a tiny 0.01. In any case, the increase is much less than what Kahan and Braman got in their models. So when can we expect to see the following from Fremling and Lott?

Thus, Lott proves the opposite of what he intended. Instead of demonstrating that carry laws make an important difference to crime rates he shows that these laws matter very little. …

It is very interesting that Lott finds that changes in crime rates are so little explained by carry laws.

We think Lott should revise his conclusion and take credit for this interesting finding.

Update: Dan Kahan responds:

We didn’t mean to imply that the empirical debate is a “waste of time,” just that it isn’t suited to promoting a stable resolution of the gun debate through democratic politics. But since enough people misunderstood us on this point (we must not have been careful enough), we wrote the “Cultural Cognition” follow up, which clarifies that issue and which also addresses the Cook/Ludwig/Lambert claim that culture *and* empirics can both matter at the same time. Instead of drafting a new response to your blog, I would rather refer people to the “Cultural Cognition” paper, subject to the proviso that it is a draft (the simulations are being made more complicated and interesting, although we are pretty happy with our narrative account of the social and psychological mechanisms that constrain individuals’ acceptance of factual beliefs that disappoint their cultural commitments).

Comments

  1. #1 raj
    January 9, 2004

    Your correspondent says “I have more faith in John Lott than I do in you”. Query what “faith” has to do with this. Faith evokes images of religion, not rational thought.

  2. #2 Kevin Baker
    January 9, 2004

    We take research largely on faith, our faith in the honesty and rigorousness of the researcher(s). That’s why, for example, Bellesiles’s book was so greatly heralded when it came out – he said things a lot of people wanted to hear, and he was a full professor of a respected university: ergo, we could trust that he had done a thorough, unbiased job.

    But we couldn’t.

    Lott said things a lot of people wanted to hear, and appeared to be a respectable academic, but since his forte is statistics rather than history, proving mendacity is a bit harder. (But it seems pretty apparent now.)

    So, “faith” is involved, because not all of us can go fact-check everything. A lot of people had faith in the Aristotelian view of the universe, until he was fact-checked and found wanting. But there’s a difference between simply being wrong, and intentionally being wrong.

The site is currently under maintenance and will be back shortly. New comments have been disabled during this time, please check back soon.