Steve Milloy misfires

Posted by Tim Lambert on September 3, 2004
(22)
More »

Daniel Davies has some criticism of a Steve Milloy Fox News column that purported to debunk a study that found that sugary drinks were linked with weight gain and diabetes. Milloy has a column on Fox News where he regularly disinforms his readers. Today I’m going to look at a Steve Milloy effort titled Gun Control Science Misfires, where he attacks two studies that he clearly has not even read. Milloy writes:

Dr. Kellerman claimed in a 1986 New England Journal of Medicine study that having a firearm in the home is counter-productive. He reported “a gun owner is 43 times more likely to kill a family member than an intruder.”

Kellermann did not report that, nor did the study find that. Kellermann actually reported that:

For every case of self-protection homicide involving a firearm kept in the home, there were 1.3 accidental deaths, 4.6 criminal homicides, and 37 suicides involving firearms.

Note that most of the self-protection homicides were not of intruders.

How did Milloy happen to misquote Kellermann? Well, he didn’t bother to read Kellermann’s paper but relied on this article by Miguel Faria. And Faria didn’t read Kellermann either, but relied on this article by Edgar Suter. The same misquote seems to have been spread far wide, appearing in a law review article, an amicus brief, and a report from the Statistical Assessment Service (apparently written by Iain Murray).

Pro-gun writers often complain about gun-control advocates using this study to make the misleading claim that gun misuse is 43 times as common as defensive use, but if you search you find that just about every such reference is made by pro-gunners objecting to the statistic.

Milloy then attacks another Kellermann study:

In a 1993 New England Journal of Medicine study, Dr. Kellerman again reported guns in the home are a greater risk to the victims than the assailants. In addition to repeating the errors of his prior research, Dr. Kellerman used studies of populations with disproportionately high rates of serious psychosocial dysfunction such as a history of arrest, drug abuse and domestic violence. Moreover, 71 percent of the victims were killed by assailants who didn’t live in the victims’ household, using guns presumably not kept in the home.

This is wrong from beginning to end. Once more it is clear that Milloy did not bother to look at Kellermann’s study, instead relying on Faria who is relying on Suter. The 1993 study used a case-control design, completely different from the 1986 study, so even if there were errors in the that study, the 1993 one does not repeat them. Suter just doesn’t know what a case-control study is. Ironically, Milloy does know what a case-control study is, and if he’d read it, might have been able to come with a coherent critique. For example, Milloy complains that the population studied was disproportionately dysfunctional, but that is not an error. The cases were people who were murdered, and yes they were disproportionately dysfunctional but that is the nature of murder victims. And 71% of the victims were not killed with guns from outside. A glance at the tables in the survey shows that claim to be false.

After dismissing Kellermann’s work as junk science, Milloy gets to John Lott. Instead of citing third hand criticism, he cites Lott’s findings with not a word of criticism:

laws that permit the carrying of concealed weapons are associated with a 69 percent decrease in death rate from public, multiple shootings such as those that occurred in Jonesboro, Arkansas and Columbine High School.

Oddly enough, when writing about a study by Cummings that linked safe-storage laws with reductions in accidental shootings Milloy wrote:

This was an ecologic epidemiology study, meaning the conclusion is based on very “macro” comparisons of groups of people. The study involved no data about individuals, just groups. Traditionally, these studies are only useful for forming hypotheses for further testing, not irrefutable facts.

Now Lott’s study was an ecologic study at the group level and Kellermann’s case-control study was at the individual level, but Milloy uncritically accepted Lott and trashed Kellermann. I wonder why? Furthermore, when I asked him why he had criticized Cummings while posting an uncritical summary of Lott’s work he didn’t defend Lott but tried to pretend he wasn’t endorsing Lott’s findings:

That wasn’t my summary… but quotes from the article.

But here he does endorse Lott’s findings.

Here’s the most telling thing about Milloy—you can tell what his conclusions about a scientific study will be without having to look at the methodology of the study. If he doesn’t like the conclusions he will find some grounds, no matter how specious, for dismissing the study as “junk science”.

Update: SayUncle comments on a draft of this post that I accidently posted:

The problem with Kellerman’s study is he compares self-protection gun deaths to other gun deaths, which discounts the self-protection that does not result in the death of someone.

Kellermann clearly notes this problem in his 1986 study, which also discounts gun misuse that does not result in death. This was one of the reasons why he did the 1993 study which does measure self-protection that does not result in death.

(22)
More »

Comments

  1. #1 ben
    September 3, 2004

    I cannot find the 1986 study anywhere. I only find the abstract. Does anyone know where a copy of the 1986 study can be had?

  2. #2 ben
    September 3, 2004

    Pro-gun writers often complain about gun-control advocates using this study to make the misleading claim that gun misuse is 43 times as common as defensive use, but if you search you find that just about every such reference is made by pro-gunners objecting to the statistic.

    apparently the American Jewish Congress bought the baloney and the brady campaign comes close (22 times more likely) among others.

  3. #3 Tim Lambert
    September 3, 2004

    Ben, as far as I know, the 1986 study is not online — your local university library should have a copy.

    If you found the AJC reference, you must know how rare it is for gun control groups to cite that figure.

  4. #4 Lurker
    September 3, 2004

    It’s hard to believe that some people actually admire Milloy and don’t see him as the unsavory character he is.

  5. #5 ben
    September 3, 2004

    The AJC reference was on the first or second page of search results from Google. I don’t know much about AJC, just sounded official. There were others on the first page or two.

  6. #6 Xrlq
    September 4, 2004

    Tim, you’re wrong. It may well be rare for anti-gunners to trot out the 43-1 canard today, now that it’s been widely discredited, but it was not at all uncommon for them to cite the figure back in the mid-1990s when the story was fresh. They didn’t always get the number quite right; IIRC the Chicago Tribune may have quoted a 42-1 ratio rather than 43, so be sure to keep your Nexis searches flexible. I’m pretty sure the number appears in Stuart Smalley’s first book, “Rush Limbaugh is a Big Fat Idiot,” and is treated as though it were an undisputed fact.

  7. #7 jre
    September 4, 2004

    Tim wrote

    if you search you find that just about every such reference is made by pro-gunners objecting to the statistic.

    and Xrlq replied

    Tim, you’re wrong.

    It seems that Xrlq intended to reply to Tim’s present-tense observation.

    As best I can tell, Tim is right. Before I’d even seen any of these comments, out of curiosity I did a Google search for the phrase “43 times”, and found that 8 of the 10 hits on the first page were, indeed, from gun-rights advocates. Two were unrelated. One hit from the next 10 was related — again, from a gun-rights group. After that, they are all unrelated to the issue. Using Google’s criteria for website popularity seems to lead pretty clearly to the conclusion that the phrase “43 times” is used predominantly by gun-rights advocates writing in opposition to Kellerman.

    I did not turn up the AJC site with that search, but I checked it out. AJC uses the figure without attribution.

    So — the interesting question seems to be this:


    If Kellerman never made the “43 times as likely” claim, who used the phrase first, and in support of what argument?

  8. #8 Kevin Baker
    September 4, 2004

    Given the time frame involved (and my lack of ability to do a Lexis/Nexis search) proving who did what when would be extremely difficult. But what Tim illustrates here is what I’ve protested about since I started writing on the topic – news sources getting the facts wrong, and passing opinion off as irrefutable fact. Now, Milloy’s piece is unquestionably an op-ed, not straight news, but compare and contrast that to CNN’s supposedly unbiased “Just the facts, ma’am” reporting on the Assault Weapons Ban from May of 2003.

    Or, if you want another op-ed, try Jean Hanff Korelitz’s Salon.com piece What a Few Good Women Can Do where she says – without embarrassment – “And what about the more than 4,000 children who die in gun-related accidents each year? That’s 11 kids a day. And we’re not talking about crimes, or intentional shootings. We’re talking — or not talking enough — about accidents.”

    Hey! It’s in print, it MUST be true, right? Or what about the press repackaging Violence Policy Center press releases – often without attribution – as news reports? Say, like the Atlanta Journal Constitution (which did attribute) the VPC’s contention that “Forty-one of the 211 U.S. police officers killed in the line of duty between 1998 and 2001 were murdered with assault rifles, according to a new analysis by the Violence Policy Center.” Except they didn’t check that assertion. But I did.

    I’m not in favor of the media misleading the public, but Mr. Milloy is heavily outnumbered by his polar opposites. That’s not an excuse. It’s an illustration of a much broader problem.

  9. #9 ben
    September 4, 2004

    Just so there’s no more question about it, what Kellermann did report (I went to the library and made a copy of the 1986 paper from the New England Journal of Medicine) was:

    The home can be a dangerous place. We noted 43 suicides, criminal homicides, or accidental gunshot deaths involving a gun kept in the home for every case of homicide for self protection.

    which is followed by

    In the light of these findings, it may reasonably be asked whether keeping firearms in the home increases a family’s protection or places it in greater danger.

    So, the Kellermann study didn’t report exactly what Milloy says it did, but you can certainly infer that a firearm in the home is 43 times more likely to kill criminally or accidentally than for self protection. The imediate mention of family protection after seems to imply (although doesn’t explicitly say) the point of Milloy’s quote. Especially when they say in the body of the paper:

    A majority of [criminal] homicide victims were residents of the house or apartment in which the shooting occured.

    So, to be correct, and assuming that we can call residents of the same house or apartment “family members” (although they might just be close friends, same difference), Milloy ought to have said that the study reported a gun in the home is at least 22 times more likely to kill a family member than to kill in self protection (right, not an “intruder,” but some jerk who presumably had it coming), which is what the Brady Campaign reports.

    Kellermann also claims (and I buy this one) that

    Easy access to firearms may therefore be particularly dangerous in households prone to domestic violence.

    well, duh.

    Getting a little off topic there. Lastly, the first ten hits from a google search (including the quotes) of “43 times more likely” comes up with 4 pro-gun sites quoting the wrong thing (as milloy does) about an “intruder,” 2 that are essentially paraphrased correctly, one that says that gun control advocates report this thing, one that doesn’t complete the whole statement, and 2 that are irrelevent.

  10. #10 jre
    September 4, 2004

    Inquiring minds owe a debt of gratitude to ben for actually getting a copy of the paper in question. Thanks for saving my lazy butt a trip to the library.

    I can add one more data point. Xrlq was quite correct in asserting that “Rush Limbaugh Is a Big Fat Idiot and other observations” (Delacorte 1996) contains a similar misquote of Kellermann. On page 84 we find

    According to a study in the New England Journal of Medicine, guns kept in the home for protection are forty-three times more likely to kill a family member than an assailant.

    I was a bit disappointed, because Al Franken is usually a scrupulously accurate researcher, and in fact had made quite a point of it in the book.

    (I have one more post. Please forgive me in advance).

  11. #11 jre
    September 4, 2004

    Enough of this nit-picking. If Kellermann’s data and methods are sound, do they support the conclusion that guns in the home are more likely to do harm than good? Maybe. Here’s the abstract:

    To study the epidemiology of deaths involving firearms kept in the home, we reviewed all the gunshot deaths that occurred in King County, Washington (population 1,270,000), from 1978 through 1983. The medical examiner’s case files were supplemented by police records or interviews with investigating officers or both, to obtain specific information about the circumstances, the scene of the incident, the type of firearm involved, and the relationship of the suspect to the victim. A total of 743 firearm-related deaths occurred during this six-year period, 398 of which (54 percent) occurred in the residence where the firearm was kept. Only 2 of these 398 deaths (0.5 percent) involved an intruder shot during attempted entry. Seven persons (1.8 percent) were killed in self-defense. For every case of self-protection homicide involving a firearm kept in the home, there were 1.3 accidental deaths, 4.6 criminal homicides, and 37 suicides involving firearms. Hand-guns were used in 70.5 percent of these deaths. The advisability of keeping firearms in the home for protection must be questioned.

    Milloy’s argument against Kellermann relies strongly on the assumption that only 0.1% to 0.2% of defensive firearm uses result in death, a figure he got from Faria, who got it from Suter, who claims he got it from Kleck.

    I couldn’t find the statistic in Kleck, or a primary reference anywhere else. Can anyone help?

  12. #12 Kevin Baker
    September 4, 2004

    First, throw out the suicides. The risk of suicide appears to be unaffected by the availability of method. If someone is serious about killing themselves, then they’ll kill themselves regardless of the availabilty of a firearm. The Japanese, the French, the Finns and many other societies have much higher rates of suicide than the US and don’t use firearms to get there. A study of the effect of the Brady waiting period by (I believe) the NEJM determined that its only effect on suicide was a shift in method by older men. Method, not number.

    Second, we need more demographic information. You’re talking about all gunshot deaths in one county. What criminal records did the perpetrators AND the victims of criminal homicide have? And how representative are they of the total population of King County? Criminal homicide is trends heavily towards a past criminal record. Joe and Jane Average, with nothing more than a speeding ticket on their records, are far less likely to be perpetrators or victims of homicide regardless of how many firearms they possess than Joe Wifebeater and his girlfriend du jour. A study of all deaths by gunshot means a pretty self-selecting statistical pool.

    You need to know data that Kellermann acknowledges he did not have, such as successful defensive gun uses where someone was only wounded, or no one was shot, or even when a shot wasn’t fired. All he’s doing is comparing the number of justifiable homicides to all other deaths by firearm, and then extrapolating how dangerous firearms are to have for protection. Still, one defensive justifiable homicide per 4.6 criminal homicides? That’s a pretty good ratio in my humble opinion. If it starts reaching parity, the number of criminal homicides better be really damned low.

    But the question I want an answer to is “How many criminal deterrences does that represent?” Crimes – I’d like to remind you – that probably would have been completed had the defenders not been armed. We don’t know. Kellermann can’t tell us. All he’s interested in is the death statistics.

    Kellermann does an outstanding job of creating deniable soundbites for other individuals and organizations to fold, twist, spindle, and mutilate and introduce into the background noise of the gun control debate. It’s done by both sides, but it’s an art form for the anti-gun groups.

  13. #13 Jim Norton
    September 5, 2004

    In Milloy’s most recent rant he claims that obesity is not all that bad. Amazing.

  14. #14 Tim Lambert
    September 6, 2004

    jre, the 0.1% of defensive uses does come from an old paper from Kleck. However, it depends on Kleck’s estimate that there are over 2 million DGUs per year and that figure is almost certainly wrong. Unless you believe Lott’s fabricated 98% and 95% numbers for defensive uses with no shots fired, the figure must surely be higher. Real surveys suggest that defensive users fire shots about 25% of the time. If one-third of those result in a wounding, with medical data showing about one in seven gunshot wounds ending in death, it follows that roughly 1% of DGUs result in the death of the attacker.

    The percentage turns out to be roughly similar for offensive gun uses, which is why the 4.6:1 ratio (not the 43:1 ratio that some use) in Kellerman’s study does give an indication of the relative frequency of offensive and defensive gun use.

  15. #15 Kevin Baker
    September 8, 2004

    I’m just a little curious. If you accept Kellermann’s data that almost 18% (1 in 5.6) of homicides in the home (though I’m not sure that in that study all the homicides occurred “in the home”) are defensive, and you’re willing to extrapolate from that the “relative frequency” of (successful) defensive gun useage, then how could anyone conclude that wanting to keep a firearm for self-protection is a bad idea?

    I would prefer a 1-in-6 chance to essentially no chance at all. Wouldn’t you?

  16. #16 Tim Lambert
    September 8, 2004

    If keeping a firearm for protection means you are more likely to end up dead because some other resident uses it against you, then it is a bad idea. Your interpretation of the stats as 1 in 6 vs no chance makes no sense.

  17. #17 Kevin Baker
    September 9, 2004

    Did Kellermann prove that the gun used in the all of the 4.6 criminal homicides was the gun kept by the homeowner? That it was that gun “used by some other resident” against the gun owner?

    I think not.

    But that’s what you’re supposed to assume.

    What Kellermann’s study said was, “If you own a gun you’re more likely to die by firearm homicide than if you don’t.” Not “you’re more likely to die by your own gun in a firearm homicide.” He makes the inference “by your own gun” by noting the accidental deaths and the suicides (which, obviously, would be by the gun in the house.)

    How many of the criminal homicides were committed with the gun in the home, and not a gun from outside the home? Did Kellermann answer that question?

    My problem with Kellermann’s study – as noted by someone elsewhere in your comments – is that Kellermann et al. seem to determine what it is they want to “prove” and then do research to prove just that thing. If you own a gun, is there an increased risk of accidental gunshot? Well, DUH! If you have a gun and a violent spouse, is there an increased risk of homicide by firearm? Well, DUH! But is that risk greater than, say the risk of homicide by battery? Hell if I know. No one’s done that research. In the homes where a legitimate defensive gun use took place, what were the criminal records of the shooter and the deceased? What about in the homes in which a criminal homicide occurred. Don’t know. We don’t have that data. But the FBI says that both perpetrators AND victims of criminal homicide normally have long records with increasingly violent tendencies. They are not “average citizens.”

    Yet “average citizens” own the bulk of the firearms in this country.

    Kellermann’s study is pretty much useless except at producing really good gun-control soundbites.

    That is my objection to Kellermann. He doesn’t lie. He doesn’t need to. He just does essentially useless research. Useless except to people like Al Franken.

  18. #18 Tim Lambert
    September 9, 2004

    You ask: “Did Kellermann prove that the gun used in the all of the 4.6 criminal homicides was the gun kept by the homeowner?”

    I quote from the abstract:

    A total of 743 firearm-related deaths occurred during this six-year period, 398 of which (54 percent) occurred in the residence where the firearm was kept.

    So the answer to your question is “yes”.

    And his 1993 study looked at the risk of non-gun homicide as well as gun homicide. Gun ownership seems to make homicide more likely rather than protecting against it.

  19. #19 ben
    September 9, 2004

    After reading the Kellermann 1993 paper I find some interesting statements (among others). First, we have that

    Guns were not significantly linked to an increased risk of homicide by acquaintances, unidentified intruders, or strangers.

    and

    Efforts to increase home security have largely focused on preventing unwanted entry, but the greatest threat to the lives of household members appears to come from within.

    which seem to imply that if (big if) your family is not a threat to you, a gun in the home is at least benign, and, possibly beneficial (since the study does not analyze the use of a gun to defend when a death does not occur).

    Second,

    Attempted resistance was reported in 184 cases (43.8 percent). In 21 of these (5.0 percent) the victim unsuccessfully attempted to use a gun in self-defense. In 56.2 percent of the cases no specific signs of resistance were noted.

    So in 5% of these homicides, the victim did attempt to use a gun in self-defense. How many times, then, was a gun successfully used in self defense without killing anyone? The main flaw of Kellermann’s work is that he does not address this issue. His work may be usefull in some manner, but his final recommendation that a gun not be kept in the home because it is too risky is invalid for not addressing the non-lethal self defense issue.

    Finally, we have the following.

    we cannot exclude the possibility that the association we observed is due to a third, unidentified factor.

    Interesting.

    Suppose then, that you live in a family that you trust and that is low risk to you for gun homicide. For example, in my humble opinion, the only person in my family at more risk of intentional homicide from anyone else in the household due to the presence of a gun is me (due to age and size/strength). Given how well I know my other family members, I’m not too worried. The rest of them aren’t any more worried either, since, really, if I wished to murder one of them, I could use any of a dozen or more implements (hammer, knife, bat, etc) or even my bare hands to easily do the job. Should I then listen to Kellermann’s advice on the matter?

    What it comes down to is that I trust them, they trust me, we’re all emotionally stable, no drugs, no criminal activity, no alcoholism, etc. etc. The risk factor for homicide by one of us due to the presence of a gun is certainly low. In our neighborhood, the risk of being assaulted by an intruder is also very low, probably lower than the risk of the gun. Really, there is very little risk from any direction. However, the worst scenario for me is to find myself in the unlikely occasion that I need a firearm to defend my family and do not have one. Hence, I have one, end of story, Kellermann be damned.

    On the other hand, if one of my family members, say my son, grows up to be a emotionally unstable or a hot-head or some such thing, I’m responible for recognizing that and re-evaluating the situation. I’m sure it is possible that in some households, a gun in the house is more of a liability than a benefit.

    So it is up to the individuals to assess the risks and live with the consequences of having or not having a gun in the home. Just like we do with having or not having medical insurance or even going for a bike ride down the road etc.

  20. #20 Kevin Baker
    September 9, 2004

    Me:

    “Did Kellermann prove that the gun used in the all of the 4.6 criminal homicides was the gun kept by the homeowner?“

    You:

    I quote from the abstract:

    A total of 743 firearm-related deaths occurred during this six-year period, 398 of which (54 percent) occurred in the residence where the firearm was kept.
    So the answer to your question is “yes”.

    I’m sorry, but No! Fifty-four percent of the deaths occurred IN THE HOME. How many of these were SUICIDES? Remember, the study included suicides as “firearm related deaths” at a ratio of 37 suicides to 4.6 criminal homicides (8:1). Even you dismissed suicides from the list when you conceded the ratio was 4.6:1, not 43:1. If 54% of the “gun related deaths” occurred IN THE HOME, what percentage of these were suicides? Half? Two-thirds? If 46% occurred outside the home, how many of these were criminal homicides committed with firearms not belonging to the homeowner? How many of these victims were engaged in high-risk activities? Note what information was not given by the abstract. Is it in the body of the paper? Hell if I know.

    In addition, Ben reinforces the point I made above: If you don’t live in a violent family (and the overwhelming majority of us do not) then having a firearm for self-protection isn’t all that dangerous. It puts you at risk of accidental gunshot, but that risk is, in actuality, quite small. It does however give you the chance to resist physical attack, and there is a long list of people here in the States who have done so. (If the NCVS is right, some 295 a day. What, that’s not enough?)

    I don’t think they’d like to give up their guns because there’s a possible (maybe) elevated risk of getting shot, when they know they stopped a crime with their gun.

    Damn, but Americans are anti-social people, aren’t we?

  21. #21 Tim Lambert
    September 9, 2004

    Ben, you are right that in a household where there is zero chance of someone killing someone else, adding a gun will not do any harm. After all, tripling a risk of zero, still leaves it zero. However, these are the families where a gun is not needed for defence. In the ones where there is a violent resident, the ones where a gun might be needed for defence, adding a gun doesn’t make things better by enabling self-defence it tends to make things worse by facilitating homicide.

  22. #22 Tim Lambert
    September 9, 2004

    Kevin, the 4.6 criminal homicides were a subset of the 54% that occured in the residence where the gun was kept. So, as I already said Kellermann did show that in those cases the gun was one kept by the homeowner.