"Eugene Volokh" writes:
Please, please, let's take special care to be polite in these
exchanges. This is a sensitive subject, but even when we think the
other person is dead wrong, it's better to say this in a subtler way.
OK, I'll do my best to be polite. I won't say anything in reply to the
ill-mannered Frank Warren other than to note that argument ad hominem
is a fallacy, as is arguing from authority ("Kleck is infallible" type
arguments) and as are straw man arguments (falsely claiming that I have
asserted that Kleck is lying or asserted that guns are pointless.) I
think critical analysis of Kleck's estimate is especially important
for those who because of their biases (and we all have our biases)
would like to accept it as true.
On the other hand, there is something odd about the assertion
that guns are used defensively in over 700,000 burglaries and 400,000
robberies. It's particularly interesting that the defenders
perceived that someone "almost certainly would have died" without the
gun use in 15.7% of the cases, or over 300,000 cases total. Surely
that's a remarkable number, one rather inconsistent with our
presumably reliable homicide totals. (I realize, though, that it's
possible that people wildly mis-estimated this but were truthful about
other, more objective, judgments.)
The estimate of 200,000 woundings of criminals in DGUs is also
remarkable. It is 10-20 times higher than Kleck's earlier estimates
of this number derived from multiple independent data sources. It is
double the NEISS derived estimate of the number of ALL gun shot wounds
(criminal, defensive, accidental and attempted suicides) treated in
hospitals.
Kleck also reports that two respondents volunteered that they killed
the perp. Kleck's own earlier estimates of the number of defensive
gun killings indicate that it would be most unlikely to have even a
single such case in a survey of this size.
Another way to estimate DGUs is to ask criminals how often they were
thwarted by armed victims. This is what Wright & Rossi did. 34% of
the criminals that they surveyed admitted this. These criminals had
an average of 10 prior arrests, so if the 34% who had been thwarted
had an average of two thwartings each, it would appear that being
arrested is roughly 15 times as likely for these criminals as being
stopped by an armed victim. Combine this with about 988,000 arrests
per year (this number from Kleck's "Crime Control" paper) and we get a
(very rough) estimate of 70,000 crimes stopped each year with guns.
The uncertainties in this calculation are large: it might be off by a
factor of two or three, but a factor of thirty is implausible.
To put it another way, if Kleck's estimate is correct Wright and
Rossi's criminals should have had an average of 20 encounters with armed
victims each. And yet most said they had never not once encountered an
armed victim. OK, maybe most of these people lied. Or maybe 4% of
Kleck's respondents made stuff up.
Finally, much as one might fault the Crime Victimization Survey, it
seems to me to use methods that are not that far different from
Kleck's. True, its nonanomyous nature and failure to ask the right
questions makes it singularly inapt at measuring defensive gun use.
But I'm not sure it's equally inapt at measuring, say, robbery or
burglary. Am I mistaken on this?
I'm puzzled as to why you say that the NCVS fails to ask the right
questions. Let's focus on the number of DGUs against burglaries, where
the Kleck estimate is 25 times that of the NCVS one. In essence, the
questions asked by Kleck were:
1. Did you use a gun defensively against someone? and
2. (Only asked if answer to question 1 was yes) Was it against someone
attempting a burglary?
While the relevant NCVS questions were in essence:
1. Did someone attempt a burglary?
2. (Only asked if answer to question 1 was yes) Did you use a gun to defend
against that person?
That is, the questions are roughly the same, only the order is
different. It is certainly possible that asking the questions in a
different order will make some small difference to the numbers who
answer "yes" to both questions. It most implausible that the
difference would be large. The NCVS experimented with different
questions in their recent redesign and found that asking several times
about the same crimes in different ways tended to jog their
respondents memories and deliver more "yes" responses, but the
increase was about 20% or so, not factors of 20 or more.
As to the non-anonymous nature of the NCVS: It is true that Kleck's
survey was anonymous, but not in the usual sense of the word, where
the identity of the interviewee is unknown. Since there are not
usually large numbers of adults of a particular sex living in the
average American household, it would be quite easy for someone
conducting such a survey to determine the identity of the interviewee.
Kleck's survey is anonymous in the sense that the identity of the
INTERVIEWER is unknown. Sure he identified himself as a research
assistant working for Gary Kleck, but how does the interviewee know
that he is not really a burglar casing the joint, or a police officer
looking for innocent gun owners to persecute?
Here's what Professor Biderman (author of the book "Understanding
Crime Incidence Statistics: Why the UCR diverges from the NCS") had to
say on this point during a discussion on the NCVS mailing list:
Albert Biderman wrote:
Another issue Weingarten raises relates to respondent fears of
use of (e.g.) her survey responses against her. Respondent
confidentiality, by law and by practice, is better protected in
Census Bureau-conducted surveys, such as the NCVS, than in any
other ones. Fears of uses of information against a respondent
who gives it are groundless. Nobody beats the Census Bureau
either in the level of cooperation and confidence it achieves
from its samples of the public. A few decades of attacks
against government in general, and the Census Bureau, in
particular, appear to have caused some, but limited, damage to
cooperation with federal surveys.
To put things another way: if you want to believe that Kleck got the
right answer and the NCVS the wrong one because people distrust the
NCVS and lied, then you must believe that 96% of the population are
sufficiently paranoid to believe that the Census Bureau lies when it
promises confidentiality AND you must believe that NONE of these
people are sufficiently paranoid to believe that a government agent
would lie to them and claim to be a private researcher conducting
research into DGUs. I find explanations that require 96% of the
population to be complete lunatics somewhat absurd.
Alternatively, 4% of Kleck's respondents may have told some tall
tales. I personally believe that it the explanation "4% of the
population are braggers" to be more plausible than the explanation "96%
of the population are paranoid loons".
To summarize: While Kleck's survey appears to have been properly
carried out, the methodology does not defend against untruthful
responses by interviewees and it only requires a very small fraction
of untruthful responses to generate his enormous estimate for DGUs.
Kleck's estimates are inconsistent (usually by factors of ten or more)
with:
CDC counts of homicides
UCR counts of homicides
Kleck's own, earlier, estimate of defensive woundings
Kleck's own, earlier, estimates of defensive killings
Wright and Rossi's survey of criminals
NEISS counts of gunshot wounds
NCVS counts of burglaries and violent crimes
UCR counts of burglaries and violent crimes
NCVS counts of DGUs
The most parsimonious explanation for all these discrepancies is that
a small fraction of Kleck's interviewees told some tall tales.
Finally, let me note that none of the forgoing proves that the NCVS
estimate is correct, only that the Kleck estimate is probably in
error. It is possible that it underestimates DGUs, maybe even by a
factor of two. There is just no good reason to believe that the same
people who wouldn't tell the NCVS about a defensive gun use would tell
Kleck.