You said that they failed to take into account the possibility that
"violent people (gang members for example) are both more likely to get
firearms and are more likely to get themselves killed".
Kellermann et al (in the abstract) "case households more commonly
contained an illicit-drug user, a person with prior arrests, or
someone who had been hit or hurt in a fight in the home. After
controlling for these characteristics, we found that keeping a gun in
the home was strongly and independently associated with an increased
risk of homicide." And this is covered in much greater depth on pages
1087, 1088 and 1089.
Dan Day writes:
Again, my wording should have been something to the effect that they didn't
adequately account for such things.Let's take the prior arrests, for example. Whatever criteria you use
when considering what kind of arrests should be counted as a marker
for violence, if your measure
is "yes" someone in the household was arrested versus "no" no one in
the household was arrested (as Kellermann did), you've woefully failed
to properly capture the very tendencies towards violence you're trying
to measure.So here we find that by using a "yes or no" arrest measure, Kellermann
sadly undermeasured the amount of violence found in the case households.
He assumed that since there were twice as many arrests in case households,
that they were only twice as violent, or twice as likely to have a violent
member in the household, or however you want to describe the relationship.
However, it might actually be the case that only, say, 5% of the control
households had a significantly violent (or criminal, or whatever) person in
the household, whereas say 45% of the households in which a murder took place
might have had such a person. So Kellermann's using a 2:1 factor, when
the reality might be 7:1.
Well, it might, but studies into criminality tend to show that
criminals do not specialize in type of crime. Another question on the
seriousness of the crime arrested for (perhaps by asking if
imprisoned) would have been a good idea.
As such, he not only ended up with results which seriously underestimated
the contribution of violent histories to a risk of home homicide,
Woah there. The "might" in the previous paragraph seems to have
turned into an iron-clad certainty. Well, actually Dan is correct:
"Any household member arrested" does not adequately measure the
propensity to violence. The proof of this can be found by looking at
Kellermann's final model: "household member hit or hurt in a fight in
the home" was independently associated with the risk of homicide.
However, just because one variable does not adequately control for all
confounds, it does not follow that the study does not, since they
measured over two dozen variables.
but since his multivariate model attempts to determine how much the various
possible factors influence each other, he has also underestimated how much
the relationship "violent/criminal tendencies leads to both gun ownership
and murder" contributes to his final results.
No. See above.
The "yes or no" choices on the questions of "is there an illicit drug user
in the household" or "has a member of the household been hit or hurt in a
fight" in the home are equally flawed, for "yes or no" answers do not
properly measure the relative severity of the incidents that are observed in
the case versus control households.
Hmm, I guess they should have also asked a question like "has any
family member required medical attention because of a fight in the
home?". Oh hang on, they DID ask that didn't they? Are you sure you
have read the study?
You claim that they made "wild unfounded conclusions". Here are the
actual conclusions (from the abstract) "The use of illicit drugs and a
history of physical fights in the home are important risk factors for
homicide in the home. Rather than confer protection, guns kept in the
home are associated with an increase in the risk of homicide by a
family member or intimate acquaintance."
The first sentence of the conclusion is justifiable, because if anything
the study undermeasured the effects of such factors. The second sentence
is indeed wild and unfounded, for reasons given above, and given previously.
The short form is that the various flaws in the study all tend to
overemphasize any alleged risk factors for gun ownership,
Not necessarily. Controlling for other risk factors increased the
odds ratio associated with guns in the home from 1.6 to 2.6. If these
factors really were significantly undermeasured as you claim then it
is more probable that the study underestimated the risk factor
associated with gun ownership.
Now, certainly, he made a half-hearted attempt to see whether there
was a criminal record involved somewhere, but surely you're not going
to try to tell us that it was anywhere near rigorously investigated.
Many violent people don't have arrest records,
If you had read table 3 you might have noticed some other relevant
factors considered such as "any household member hit or hurt in a
fight in the home".
See above. The fact that Kellermann considers this an adequate measure
and that the NEJM peer reviewers considered it an adequate measure only
points out the problems of doctors conducting and reviewing matters
better left to the experts in the field, criminologists and perhaps
sociologists.
Epidemiologists do have some expertise at measuring risks associated
with certain factors. Can Dan point to some research by
criminologists on risk factors associated with homicide that he
approves of?
The fact that you consider it an adequate measure only shows that you
haven't examined the study carefully enough. Somehow, I get the
feeling that if this were a study which produced findings favorable
to gun ownership, you'd be the first to notice and point out such
flaws.
Let's consider such a study shall we? In a paper published in Social
Problems 35:1 1-21 (1988) after looking at NCS data Gary Kleck
concluded "Victim resistance with guns is associated with lower rates
of both victim injury and crime completion for robberies and assaults
than any other victim action, including nonresistance."
Here's something I didn't write about Kleck's paper:
Kleck assumes
in his study that if uninjured crime victims used guns more often than
injured crime victims, then that means that using a gun decreased their
risk of injury. Nowhere does he even consider the obvious case
that not only do robbers sometimes injure their victims
to stop them from resisting, but people who prepare themselves to defend
against crime are both more likely to get firearms for defence
and are more likely to be more alert and prevent crime completion.
It's his failure to
take these kinds of complex interactions into account (and the failure
of the sociological journal peer reviewers to understand the problem)
that make this study unworthy of publication in an actual
criminological journal, and which enable him to reach the
kinds of wild unfounded conclusions that he does.
It would have been much more accurate than what you've written about
Kellermann's study, but I haven't written something like the above. I
have pointed out that Kleck failed to consider reverse causation or
confounds, but I haven't called his conclusions wild and unfounded or
suggested that his study should not have been published. Tell me,
Dan, do you consider Kleck's study "unworthy of publication", too? Or
do you have a double standard?
Likewise, we've earlier discussed the problems with
using the kind of mathematical analysis that Kellermann used to
"separate out" the effects of the various factors examined when it
comes to social factors that aren't as easily separable as the model
assumes.
Oh, really? And what is the statistical method of choice?
If you mean how would one separate out the various factors in order to
come up with numbers that could be said to meaningfully capture what
would happen if "all else were equal", then honestly, I'm not sure if
there is one. I don't believe that you can do so. The best that
could be done would be to break all the cases down into somewhat
comparable groups and try to extract some meaning from each one,
the results of which would only be applicable to that subgroup.
Good idea. They should have done stratified analyses with subgroups
like males, domestic homicides, intruder homicides, gun homicides and
so on. Oh hang on, they DID ask that didn't they? Are you sure you
have read the study?
I'm also dying to see you defend the claim that medical doctors can
properly peer-review a paper which concentrates on criminological
and social issues.
The proof of the pudding is in the eating. If a paper that does not
consider reverse causation or control for confounds shows a failure of
peer review then the NEJM passes,
...but not if we expect that proper peer review would result in a paper
that doesn't leave so many things inadequately account for as this one does.
The world is a complicated place. No matter how many factors are
considered in a correlational study (in this study there were 35
factors considered) it is always possible to think of one more. This
does not prove that there has been a failure of peer review. I do not
claim that this study is conclusive or that it does not have
limitations (to find out those limitations you should read the
relevant section of the paper). It does need to be corroborated by
follow-up studies. I think it would be more productive of you to
propose a design for such a study than to assert that the Kellermann
study is "unworthy of publication".