The Lancet paper on the Iraq War toll

One of the headlines made by Bob Woodward's new book on the Bush administration, State of Denial, is that the violence in Iraq is much worse than we have been told. Told by the Bush administration, anyway. In fact we have been on notice for two years that the level of violence in Iraq is horrendus. In 2004 researchers at Johns Hopkins School of Hygiene and Public Health, Columbia and in Iraq produced the best evidence up to that time of the death toll from the 2003 US invasion. The study was published in the British medical journal, The Lancet. Its publication was expedited so it could appear before the 2004 November elections and was condemned on that basis as a "political document." It was also subjected to a great deal of misinformed, if not ignorant technical critique. We blogged it many times over on the old Effect Measure site (e.g., here), but by far the best defence of the technical issues was produced by our (now) Science Blogs sibling, Tim Lambert at Deltoid. Tim didn't just knock down the false technical criticisms, he flattened them (an example, here).

Now an update of the 2004 study has been published using the same methodology, with some of the same team of researchers from Johns Hopkins.

It underlines the Woodward thesis in triplicate. The current best estimate of excess deaths from the 2003 invasion stands at just over 650,000. NOt opnly far worse than we have been told. Far worse than anyone imagined. The response from the Republicans will be the same as last time: technical smoke screen and the claim that it's "just politics." We're already seeing both responses. Dear Leader Bush has given his scientific assessment today by saying at a news conference he doesn't consider the work credible (as in, "I choose not to believe it.") (CNN). CNN talking head ("defense analyst") Anthony Cordesman, whose track record on punditry in the Iraq war has been dismal, gave his opinion before the paper was even available for review: it's politics. That was in an early CNN story now replaced by a version without Cordesman's opinion, but we know it will be a prominent version of the Republican response, so let's deal with it first.

Yes, the objective is, in part, political. Voters should have the best possible information about candidates and their policies before voting. The Iraq war is a major issue in the upcoming US midterm elections. So it makes sense to provide the best information possible. You can call that politics if you want. It's not a problem for the science. We'll come back to this at the end of this post.

What about the science? It doesn't take much reflection to see that estimating mortality from an invasion with a disastrous and chaotic aftermath is no simple matter. The Iraqi Ministry of Health is entirely unequipped to make such an estimate. It doesn't have the expertise and cannot operate freely in most of the country. It is also constrained by sectarian politics, as are all Iraqi ministries. Most other estimates are based on hospital and mortuary records or media reports. They are neither systematic nor reliable and are known to underestimate the toll:

The US Department of Defence keeps some records of Iraqi deaths, despite initially denying that they did. Recently, Iraqi casualty data from the Multi-National Corps-Iraq (MNC-I) Significant Activities database were released. These data estimated the civilian casuality rate at 117 deaths per day between May, 2005, and June, 2006, on the basis of deaths that occurred in events to which the coalition responded. There also have been several surveys that assessed the burden of conflict on the population. These surveys have predictably produced substantially higher estimates than the passive surveillance reports. (Burnham et al. The Lancet [.pdf])

Given the security situation, a population based random sample is not feasible, but this still leaves some scientific methods available, one of which, cluster sampling, is a well accepted and conventional technique. In this case it also produced data that is consistent with data from other sources. For example, the paper estimates pre-invasion mortality of 5.5 deaths/1000 population (Crude Mortality Rate), in line with independent estimates from the CIA and the US Census Bureau. I haven't had a chance to read the paper in detail, yet, but the authors are extremely competent and their statistical consultant (Scott Zeger), chair of the Biostatistics Dept. at Hopkins is someone I have been on committees with. He was just elected to the Institute of Medicine, part of the National Academies of Science. These folks are not scientific lightweights. They know what they are doing and The Lancet is one of the world's premier medical journals. There are many reasons to have confidence in these results, even without a detailed examination of the paper (I am still reading it). That confidence is shared by others:

Ronald Waldman, an epidemiologist at Columbia University who worked at the Centers for Disease Control and Prevention for many years, called the survey method "tried and true," and added that "this is the best estimate of mortality we have."

This viewed was echoed by Sarah Leah Whitson, an official of Human Rights Watch in New York, who said, "We have no reason to question the findings or the accuracy" of the survey.

"I expect that people will be surprised by these figures," she said. "I think it is very important that, rather than questioning them, people realize there is very, very little reliable data coming out of Iraq." (WaPo)

And surely, that is the main point. We have little reliable information. This is the best we have, by far. It has a wide confidence interval (400,000 to 800,000), not a surprise. But it would be a bad mistake to consider these estimates flawed. They are telling us something, something terrible. Here are some additional details, from the excellent summary by David Brown in The Washington Post:

The survey was conducted between May 20 and July 10 by eight Iraqi physicians organized through Mustansiriya University in Baghdad. They visited 1,849 randomly selected households that had an average of seven members each. One person in each household was asked about deaths in the 14 months before the invasion and in the period after.

The interviewers asked for death certificates 87 percent of the time; when they did, more than 90 percent of households produced certificates.

According to the survey results, Iraq's mortality rate in the year before the invasion was 5.5 deaths per 1,000 people; in the post-invasion period it was 13.3 deaths per 1,000 people per year. The difference between these rates was used to calculate "excess deaths."

Of the 629 deaths reported, 87 percent occurred after the invasion. A little more than 75 percent of the dead were men, with a greater male preponderance after the invasion. For violent post-invasion deaths, the male-to-female ratio was 10-to-1, with most victims between 15 and 44 years old.

Gunshot wounds caused 56 percent of violent deaths, with car bombs and other explosions causing 14 percent, according to the survey results. Of the violent deaths that occurred after the invasion, 31 percent were caused by coalition forces or airstrikes, the respondents said. (WaPo))

Is this paper political? Of course. If that bothers you, we'll let one of the most famous pathologists of the 19th century explain it:

Medicine is a social science and politics is nothing else but medicine on a large scale. Medicine as a social science, as the science of human beings, has the obligation to point out problems and to attempt their theoretical solution; the politician, the practical anthropologist, must find the means for their actual solution.

Americans will have an opportunity on November 7 to choose politicians they will entrust with cleaning up the lethal mess that is the Iraq war. This paper is information to help them choose. That's politics, for sure.

May they choose wisely.

Addendum: See also excellent posts on this by Cervantes and Lindsay at Majikthise.