Jim Giles talked to epidemiology experts about the Lancet study. (Nature subscription required):
Data from other conflicts show that such sampling is much more accurate than media reports, which usually account for no more than 20% of deaths. “Random counts force you to go to places that aren’t convenient,” says Jana Asher, a researcher with the Science and Human Rights Program of the American Association for the Advancement of Science in Washington DC. “The media don’t wander off to distant locations. It’s a very different type of data collection.”
Death tolls from Iraqi health officials, the source of US government figures, are also suspect. The Johns Hopkins team says that the process for issuing death certificates still works well in Iraq, but the system for monitoring the number of certificates issued does not. Even before the war, note the researchers, the government’s surveillance system captured only one-third of all deaths.
Yet despite the weakness of other measurements, the new figure has still surprised researchers. Perhaps the most significant concern is the baseline rate for pre-invasion death rates used in the new study. The latest survey, which included questions about the situation before the invasion, put this at 5.5 deaths per 1,000 people per year, in line with figures on Iraq from the US Census Bureau. Iran, which has a well-run health system, has a similar rate, but Iraq was at the time suffering from years of sanctions. Some sources, including the United Nations Population Division, list a pre-invasion figure of 9.7.
The discrepancy does not invalidate the new result, and if the researchers underestimated the pre-war death rate, it’s possible that they may have also underestimated the post-war rate. But some researchers say the paper should have addressed the issue. “There should have been more introspection,” says Beth Osborne Daponte, a demographer at Carnegie Mellon University in Pittsburgh, Pennsylvania. “That increased my discomfort.”
Other researchers share that discomfort. Debarati Guha-Sapir is director of the Centre for Research on the Epidemiology of Disasters in Brussels. She has some methodological concerns about the paper, including the use of local people — who might have opposed the occupation — as interviewers. She also points out that the result does not fit with any she has recorded in 15 years of studying conflict zones. Even in Darfur, where armed groups have wiped out whole villages, she says that researchers have not recorded the 500 predominately violent deaths per day that the Johns Hopkins team estimates are occurring in Iraq.
But overall Guha-Sapir says the paper contains the best data yet on the mortality rate in Iraq. And none of the experts contacted by Nature said that their doubts fatally undermined the study. Some, such as Daponte, would have liked the authors to have better assessed their method’s shortcomings before releasing a result with such political impact. But most say the result is a welcome addition to conflict epidemiology, which is now seen as playing a central role in assessing the severity of wars, and in helping states recover from them (see ‘Counting the cost of war’).
Burnham says he would now like to study patterns of migration in Iraq and the state of the health system. He would also like to estimate deaths based on a sample 4-5 times bigger than that used so far. But survey teams are in danger on the streets of Iraqi towns, and Burnham doubts whether the need for more detailed data justifies the risk.
Hat tip: Nexus 6