How many Iraqi deaths?

The Lancet has published a study of mortality in Iraq, a followup to a similar study from a year ago. In this study, they estimated that over 650,000 more people died in Iraq during the US occupation than would have died otherwise.

The Questionable Authority has some objections. I'll start off by pointing out that he isn't disputing the basic conclusion. Mike writes:

even if I am correct, and all of these errors result in overestimates of the total number of deaths, the number is still going to be much higher than the "official" totals. The population of Iraq is being harmed by this war, and it is being harmed much more than either the Iraqi or US government are willing to admit.

And his critique raises some valid points. Emphasizing mortality caused by coalition forces while not emphasizing sectarian violence is a mistake. It's worth noting that the sectarian violence is itself a symptom of the occupation, and given that the militias are civilians, it's hard to separate sectarian violence from other forms of violent death (the study splits violent death into three categories: coalition-caused, caused by others, or unknown). American forces wear uniforms and drive American military vehicles, so people who report that a violent death was caused by coalition forces have some basis for that assessment.

There are other valid reasons to focus on coalition-caused deaths. We control the coalition, and the coalition is supposed to be helping people. If we aren't better than the terrorists, why are we even bothering? However, those objections are political, not scientific, and that's Mike's point. And he's right.

His methodological critiques are less accurate. He makes four points:

  • The baseline population estimates are different than the previous study.
  • The number of clusters sampled in an area is proportionate to population.
  • The street intersection sampled is chosen randomly.
  • Computing differences between nonviolent mortality rates that are not statistically different.

The last point is right, but he gets a statistical point wrong.

Mike writes:

The number on non-violent excess deaths was obtained by subtracting the pre-invasion non-violent death rate from the post invasion non-violent death rate. I'd have no problem with that approach, if they had shown that the post-invasion increase was significant, but they did not. In fact, when they conducted a significance test, they obtained a p-value of 0.523, which is not at all significant. (If you aren't familiar with p-values, that more or less means that there's a 52% chance that the difference between pre- and post-invasion rates was just due to the luck of the draw.)

The parenthetical is wrong. A p-value of 0.523 does not indicate the probability that the two populations (pre- and post- invasion) are different, it indicates how likely you'd be to get this result by chance alone if there were no difference.

That matters because it actually strengthens his point. A p-value of 0.99 would indicate that the two samples are surprisingly similar, perhaps even more than you'd expect by chance. A p-value of 0.5 says that nothing at all unusual is going on, there's really no difference. And as Mike observes, removing that effect drops the number of excess deaths by 50,000. "Only" 600,000.

The other three points Mike raises seem less valid.

Choosing a random street corner to start with (a random main street intersecting a random residential street) introduces no bias. Mike writes that "towns tend to have more main streets per unit of area than rural areas do," but since they are just choosing one location, that doesn't seem to matter much.

Choosing a higher population estimate does inflate the mortality estimate to some extent, but unless there's some basis for thinking the UNDP estimate is worse than other estimates, there's no basis for claiming that this is a flaw, or even a bias (except in comparisons between this study and the earlier one).

Sampling clusters in proportion to population seems entirely reasonable to me. Sampling populations by choosing a random geographical point (as done in the 2004 study) seems like it would bias you towards rural areas which would tend to deflate your estimate of excess mortality. Sampling proportional to population will tend to sample households (as opposed to geography) at random, which seems more accurate to me.

i-25b6976781c43ada67c46f99669f77b0-200610121603.jpgThis is all borne out by the paper's own internal checks. The graph here shows that this analysis and the previous analysis are very close to identical – compare the green line and the purple one. The authors point out:

Application of the mortality rates reported here to the period of the 2004 survey gives an estimate of 112,000 (69,000–155,000) excess deaths in Iraq in that period. Thus, the data presented here validates our 2004 study, which conservatively estimated an excess mortality of nearly 10,000 as of September, 2004.

They got nearly identical results for identical time periods, which means that the changes to their technique did not unduly alter their analysis.

Categories

More like this

Update: 14 October. I just posted another entry on the topic, responding to some of the comments on this post. My conclusions have changed a bit as the result of some of those comments. As many of you probably know, a study published today in the journal Lancet estimates that just over 650,000…
The Lancet study on deaths in Iraq has been released. Get it here. Here's the summary: Background An excess mortality of nearly 100 000 deaths was reported in Iraq for the period March, 2003-September, 2004, attributed to the invasion of Iraq. Our aim was to update this estimate. Methods Between…
Stephen Soldz has posted his discussion with Jon Pedersen about the new Lancet study: [Pedersen thinks that the] prewar mortality is too low. This would be due to recall issues. ... Pedersen thought that people were likely reporting nonviolent deaths as violent ones. These two have to go together…
The BBC did not publish all of Les Roberts' answers. Here are the rest: It seems the Lancet has been overrun by left-wing sixth formers. The report has a flawed methodology and deceit is shown in the counting process. What is your reaction to that? --Ian, Whitwick, UK Almost every researcher who…

It really doesn't matter to the cons how many people have died as a result of this war; after all, if you listen to them speak candidly, Iraqis really don't count as being human.

If the p-value on excess nonviolent deaths is .52, then Mike is right that we don't have enough evidence to conclude that there were more nonviolent deaths than before the war, but that does not mean that we should subtract the nonviolent deaths out of the total. If that was a valid move, then we could make the whole finding go away just by breaking it up into little pieces (say, deaths that happened in the daytime on the first of the month, etc.), since then each little piece would not be statistically significant on its own (on account of the small sample sizes), and so we could subtract each one off from the total and decide that we couldn't learn anything from the study. Of course that's silly. The 54,000 is our best point estimate of the number of excess nonviolent deaths, which means that if we want to make the best point estimate of the total number of excess deaths we need to include all 54,000. The uncertainty in that number is already accounted for in the confidence interval that they give around the 655,000 figure, and that confidence interval has its minimum close to four hundred thousand excess deaths.

In other words, Mike's criticism is not one that someone who understood statistics would be making.

"Taliban like powers"? Reference please.

Heck, show me a reference for the Iraqi oil revenue. They import their oil these days.

Serious foreign policy and military experts are all agreeing that our presence is increasing the support for terrorists in Iraq. The recently declassified NIE report says "Al-Qa�ida� is exploiting the situation in Iraq to attract new recruits and donors and to maintain its leadership role." The Baker panel is also talking about drawing down troop levels and withdrawing the remaining troops from most of the country.

It may not be your ideology, but it is good policy.

Clarissa: I am more than old enough to have heard "you" say the same thing in Vietnam. If we leave, the world will turn communist. Hmmm.

So this is the situation. According to you, we should trust a government who put us in a situation that no matter what we do -- go or stay -- we lose. Thanks, Dear Leader.

There are already Taliban governments with oil in power in the Middle East (Saudi, etc.). The only difference is that currently they are US allies. It has nothing to do with Taliban.

Out Now is the best, most humane and soundest policy in the short run and the long run.

Clarissa, can you provide any justification whatsoever for your statement? Because I'd really like to see some proof. Otherwise please take the fearmongering elsewhere. As for "Taliban like powers" all you have to do is look at Afghanistan or how about the deal Pakistan made? Oh wait, Pakistan is our ally, they wouldn't make a treaty with terrorists would they?

Correction:

Goldstein, "I HAVE no "thought" at all...sir."

Otherwise, you'd post on topic.

There's something oddly appropriate about quoting a movie about the insanity of the Vietnam war in this context.