bob (really Edgar Suter?) writes:
From an early DRAFT of Gary Kleck's TARGETING GUNS : FIREARMS AND
THEIR CONTROL scheduled to be published this month:The Medical/Public Health Literature on Guns and Violence
False Citation of Prior Research
One final problem in the medical/public health literature on
guns-violence links is so widespread, serious, and misleading that
it deserves extended attention: the false citation of previous
research as supporting anti-gun/pro-control conclusions, buttressing
the author's current findings, when the studies actually did no such
thing. There is probably no way to persuade readers that large
numbers of supposedly competent and honest scholars would do this,
other than by listing a large number of specific instances.Consider, for example, a widely cited article by Arthur Kellermann
and his colleagues[Kellermann AL, Rivara FP, Rushforth NB et
al. "Gun ownership as a risk factor for homicide in the home." N
Engl J Med. 1993; 329(15): 1084-91. st p. 1090], wherein they claim
that "cohort and interrupted time-series studies have demonstrated a
strong link between the availability of gun and community rates of
homicide," citing four previous studies in support (their cites 2
and 15-17). Naive readers might assume that Kellermann et al. were
citing four "cohort and interrupted time-series studies" that had
empirically documented a guns- homicide association.
Well, those naive readers who just read the sentence Kleck quoted, but
not this one which Kleck forgot to tell us about:
"Previous studies of risk factors for homicide have employed
correlational analysis[15] or retrospective cohort[16] or
time-series[17] designs to link rates of homicide to specific risk
factors."
In fact, the first of the cited publications was not an empirical
study at all, but rather a review of the literature (their cite 2)
which did not support this assertion. The part of this report cited
by the Kellermann et al. (pp. 42-97) did not even address the issue
in question,
Because it was cited in support of the claim that "Homicide rates
declined in the United States during the early 1980s but rebounded
thereafter.[2]"
Since Reiss and Roth was also cited in support of Kellermanns claims
about a guns-homicide link Kellermann should not have made the cite so
specific to that section. This is a pretty minor quibble by Kleck.
while the part that did address it did not conclude that there was a
strong guns-homicide link. Instead, the review authors cited an
earlier review as having indicated that studies "qenerally find that
greater gun availability is associated with ... somewhat greater
rates of felony murder, but do not account for a large fraction of
the variation," and drew an unmistakably "no decision" conclusion,
noting problems of causal order that Kellermann and his medical
colleagues have consistently ignored (Reiss and Roth 1993, p. 268,
emphasis added).
It would seem that Kleck did not read as far as the end of page 268,
where Reiss and Roth write "The strongest empirical evidence on how
variations in firearms availability affect levels of violent crime and
felony homicides can be obtained from carefully controlled evaluations
of changes in laws and especially in enforcment efforts, that reduce
firearms availability." They then go on to summarize these
evaluations in table 6-1. This table shows 5 evaluations of which 2
were found to be fully effective, 2 partially effective, and 1 not
effective. The 2 partially effective interventions were found to have
reduced gun homicides and had no consistent effect on robberies,
assaults and non-gun homicides.
It is incorrect for Kleck to characterize this as "no decision".
Among the remaining three supposedly supportive studies, one did not
even measure "the availability of guns," nor did its authors claim
to have done so, and thus it could not possibly have established any
guns-homicide association, never mind a strong one (their cite 17,
to Loftin et al. 1991). This was an interrupted time-series study,
but neither it nor any such study has ever measured any association
between gun availability and homicide rates.
There would seem to be a major difference of opinion between Kleck and
Reiss-Roth. From the quote above you will note that Reiss-Roth
consider these sorts of studies the strongest kind of study on the
association between guns and homicide. Kleck, on the other hand,
reckons that these studies tell us nothing at all about the link.
Whatever the truth on this matter is (and I believe that R and R are a
lot closer to it than K), it is the height of arrogance for Kleck to
argue that anyone who disagrees with him is dishonest or incompetent.
The third cited study only weakly supported the authors' claim (cite
15, Cook 1979). This study addressed only robbery homicides, which
accounted for only 10% of U.S. homicides the year Kellermann et
al. wrote (U.S. Federal Bureau of Investigation 1994, p. 21), and
apparently confused an effect of robbery murders on rates of gun
ownership with an effect of gun levels on robbery murder rates.
While Kleck is entitled to interpret the results of Cook's study in a
different way from Cook, he is not entitled to impugn the honesty of
those who decide that Cook's interpretation of Cook's study is more
plausible than Kleck's.
Finally, the authors also cited a crude two-city study of their own
(Sloan, Kellermann, Reay, et al. 1988) that simply ignored the
causal order issue, used indirect measures of gun ownership that
turned out to be inaccurate,
I must have missed it when Kleck demonstrated that gun ownership in
Seattle was not higher than gun ownership in Vancouver.
and drew conclusions that ignored pronounced differences between the
cities that were responsible for some or all of the observed
differences in homicide.
So let me get this straight. If study Y clearly comes to conclusion
X, and researcher Z says "there is evidence for X" and cites study Y,
but Kleck feels that there are problems with study Y, then Z is guilty
of falsely citing Y?? What incredible arrogance by Kleck!
Arthur Kellermann and Donald Reay [Kellermann AL. and Reay
DT. "Protection or Peril? An Analysis of Firearms-Related Deaths in
the Home." N Engl J. Med 1986. 314: 1557-60.] cited similarly
nonexistent published statistics to support their assertion that
"less than 2 percent of homicides nationally are considered legally
justifiable" (p. 1559, citing sources 11 and 13). Neither FBI
publication offered any statistics on this matter, never mind a
figure "less than 2 percent."
Nonexistent statistics? Anyone can download a copy of the 1995
UCR and find these
"nonexistent published statistics" on page 31. And yes, they do
comprise less than 2% of homicides nationally. Now, Kellermann's cite
was for the 1983 UCR, so I suppose it is possible that the FBI has
only recently started publishing the figures for justifiable homicides
in the UCR. Even if this is the case, Kellermann's error is minor --
he should have cited the FBI's SHR instead of their UCR. Big deal.
Further, although the FBI had unpublished data pertaining to some
subtypes of "legally justifiable" homicides, these covered only a
small subset of lawful defensive homicides (PB:112).
Which is exactly what Kellermann said in the very next sentence. It
is disingenous for Kleck to take a quotation of Kellerman's out of
context to make it appear that Kellermann was asserting that only 2%
of of homicides were lawful defensive homicides.