Bill S.2125 was unanimously passed by the Senate and promotes relief, security and democracy in the Democratic Republic of the Congo. It states:
(5) A mortality study completed in December 2004 by the International
Rescue Committee found that 31,000 people were dying monthly and
3,800,000 people had died in the previous 6 years because of the
conflict in the Democratic Republic of the Congo and resulting
disintegration of the social service infrastructure and that
"improving and maintaining security and increasing simple, proven and
cost-effective interventions such as basic medical care, immunizations
and clean water would save hundreds of thousands of lives in the
Congo. There's no shortage of evidence. It's sustained compassion and
political will that's lacking."
The mortality study used cluster sampling and was published by the Lancet, just like the one on mortality in Iraq that President Bush dismissed because:
"The methodology is pretty well discredited"
Well, I for one don't trust the figures for the Congo either, particularly the headline figure of 4 million which is presumably a difference between an estimated actual death rate and an estimated "what if there wasn't a conflict" base rate.
At least, I am very skeptical. As for President Bush, I presume that he does not equate "study methodology" with cluster sampling, he equates "study methodology" with "plenty of people tell me it's unreliable, even Iraq Body Count does not believe them".
Who cares? The Senate is pretty well discredited, too.
Heiko, the Iraq Body Count people are pretty much discredited by now, so drop it. And Bush only started using the IBC figures when the second Lancet article came out; before that, right-wingers didn't like the IBC.
Barry,
I was speculating what Bush may have been thinking, I somehow doubt he was talking about cluster sampling when he said the survey methodology had been discredited.
The IBC seems to be taking the line that they trust the media and local authorities more than the Lancet to gather data. As those are the sources for their own tally, I suppose they are somewhat biased themselves.
I take a somewhat different line. After reading a lot about mortality estimates, I see so many contradictions that I do in fact believe that cluster sampling to obtain mortality data in developing countries gives rubbishy results aplenty.
How long will it be before David Kane shows up to complain that the International Rescue Committee won't make all the data available? (All credit to him if he does as well as he did with Roberts et al., 2004.)
Heiko Gerhauser:
Some people are paying attention to what you write here and
I am sketical of your skepticism.
You still have not answered where your "factor of 3 either way" comment about the Lancet estimate of pre- and post invasion death rates for Iraq came from.
As for Bush, he has no clue what the word "methodology" means. The guy has demonstrated beyond any reasonable doubt that he is an idiot on scientific (and most other) matters.
I think that a new study is required with an even higher estimate of deaths. Next thing you know, Bush et al will "discredit" it by claiming the Lancet one's more accurate.
Can anybody point to attempts to validate the use of cluster sampling in previous conflicts/war zones? Presumably, there has been some analysis done somewhere by someone to figure out whether it does, in fact, work in practice. Independent replication would help also (can we call on the US or British govts to fund such a study - pigs might fly I suppose...).
Hi JB,
the factor 3 does not specifically refer to the Lancet study.
It describes the rough range where I personally am reasonably satisfied that it contains the real value with near certainty.
This is not exact science, I see the difference between the ILCS and Unicef, and within ILCS between South and Centre, and I can presume that this difference gives a minimum for a range, not a maximum. It's not said that the ILCS value is definitely too low and the Unicef value definitely too high.
When I see unexplained contradictions between different cluster surveys (and even within cluster surveys) that are huge, I just question the accuracy of the method in general, and I ask myself what are reasonable boundaries for the range (of infant mortality and crude death rates)?
Under 5 mortality for example comes in as 125 according to Unicef (in recent years), multiply that by a birth rate of say 40 (which itself is uncertain) and under 5 mortality alone contributes 5 to the death rate (0.125*40). It's not unreasonable to think that the over 5 death rate might then also be 5 in a 1000, giving a total of at least 10.
On the other hand, if under 5 mortality is closer to 30, then that contributes only just over 1 death per 1000 to overall mortality, and with a young population, why shouldn't the total death rate be close to 2.5?
JB, I am not quoting some higher authority on this rough range, I am merely arguing that based on the size of the contradictions between and within the ILCS and Unicef, I am hard pressed to believe that the accuracy of a cluster survey in a country like Iraq is better than that.
Tom Rees,
yes indeed it would be nice to have some validation.
http://www.hsph.harvard.edu/now/feb17/
In the above linked press release about a Lancet study on the Congo, I read:
"The study reviewed death rates from January 2003 to April 2004 and found that there were 600,000 more deaths in the Congo during that period than would normally be expected for the region."
Said huge figure has made a pretty low media impact, presumably because debate about Iraq is so much more divisive than debate about Congo.
But what's the basis for the above figure? Can we trust it?
Frankly, I don't.
And a quick look at death rates for the Democratic Republic of the Congo:
http://www.unicef.org/infobycountry/drcongo_statistics.html
1970 20
1990 19
2004 20
https://www.cia.gov/cia/publications/factbook/geos/cg.html
Or actually:
2006 13.27
http://www.geoplace.com/hottopics/CIAwfb/factbook/geos/cg.html
2004 14.64
The population of the Democratic Republic of the Congo is claimed to be around 58 million.
Multiply that by the difference in death rates between these two sources (20-14.64 for 2004) and that makes 300,000 deaths per year that we don't really know whether they are happening or not.
To get 600,000 means a death rate difference of about 9 per 1000 compared to some base line.
A search on google scholar gave a number of other studies using cluster sampling in war regions to estimate death rates:
War and mortality in Kosovo, 1998-99: an epidemiological testimony
Violence and mortality in West Darfur, Sudan (2003-04): epidemiological evidence from four surveys
Mortality in the Democratic Republic of Congo: a nationwide survey
(these three were all published in the Lancet)
What I can't seem to find is anything coming close to validation, ie where a number of cluster study results would be tested for robustness against more reliable figures (this may of course be because in a war torn country there simply are no reliable figures, on the other hand, a census after a war might be).
For that matter I am not even bothered about it being cluster sampling or completely random sampling (the ILCS uses pretty small clusters, so small that even if we regarded each cluster as one individual there'd still be good coverage).
A great article by Dahr Jamail in ZNet today:
http://www.zmag.org/content/showarticle.cfm?SectionID=15&ItemID=11204
Jamail lives and blogs in the dying country, so I place more on his accounts that on Heiko's, I am afraid. His article suggests that there is quite broad support amongst epidemiologists for both Lancet studies (2004, 2006). For example, 'Ronald Waldman, an epidemiologist at Columbia University who worked at the Center for Disease Control and Prevention for several years, said that the [2004] survey method is "tried and true," and that "this is the best estimate of mortality we have."' A post by Donald Johnson from McLean's news magazine in Canada even reveals that huge numbers of Iraqi dead aren't being counted. Every day piles of bodies are collected from along the Tigris and Euphrates rivers. The IBC counts are a farce.
What I would like to ask Heiko is this: how many excess Iraqi deaths for you constitutes a disaster? Carnage? Slaughter? Or are you like many of the other apologists for US-UK agression here denying 655,000 because it obliterates the most recent lies (democracy promotion) spewed out of Washinton and London justifying the invasion? What gives? How do you explain the fact that 'Tony Blair and Colin Powell have quoted the Congo results time and time again without any question as to the precision or validity', as stated by Bradley Woodruff, a medical epidemiologist at the US Centers for Disease Control and Prevention. Please Heiko, can you explain why the powers-that-be bolster one sets of results and dismiss others by the same group? The answer should be patently obvious.
Jeff,
I certainly am now very suspicious of claimed death rates and excess death figures for the Congo, and as far as Tony Blair is concerned, I'd presume he'll dismiss the Lancet results based on the simple fact that he considers them biased and untrustworthy, while he does trust the official numbers from the Iraqi government. And I'd presume that most people who believe the Lancet numbers do so, because they, in the end, trust them to get their figures more right than the Iraqi government. Tony Blair may not be aware that similar methods are used to obtain figures for the Congo, and even if he is, he may not be too bothered, because the biases of the authors are not as obvious, and to boot diametrically opposed to his own, as is the case for Iraq.
I had come across the article you cite earlier, and yes some epidemiologists in there talk about the method being "tried and true", but what I want is an explanation for obvious failings in these estimates, even when there is no political bias apparent to me whatsoever, and the study seems to be well conducted and large, as is the case for the ILCS.
Your question about the number of excess deaths that would constitute disaster is trying to draw me into a discussion of politics, more precisely, a statement of whether I am with Bush or against him or something of the sort. I don't want to be drawn into a discussion like that at the moment, because I am primarily interested in resolving the scientific questions related to how reliable surveys of mortality are in settings like Iraq over the last 20 years.
I wouldn't say the IBC counts are a farce. They were low, we always knew that. The question is how low. The Lancet study suggests very low indeed - but not out of the ballpark of other similar comparisons.
What interests me is that the two appear to be tracking each other - i.e. the proportionate rises over time are similar. I would've anticipated that, as the death count goes up, the proportion of deaths reported in the media would go down. But that doesn't seem to be happening.
Having had a look at the IBC database myself, I would note that they do count many deaths I myself would not class as civilians.
For total violent deaths, their number is quite close to the official Iraqi government pronouncements.
Also, of course, it's not excess deaths they are counting, but actual deaths, ie they don't subtract a baseline value for violent deaths that would be expected anyway.
Did the Congo study use "cluster clumping" as their 2004 Iraq study? Nope.
I think it's obvious that the Congo study isn't as politically sensitive as the Iraq ones, and thus the scrutiny on it is less, as it may be every bit as exaggerated as the Iraq ones.
But I think it's a bit disingenuous of Mr. Lambert to insinuate that every cluster survey is equal. There are several layers of methodology that are different between the Congo studies and the ones conducted in Iraq.
Oh, and Lambert, I am still waiting (what's it been, over a year?) for you to provide a single example other than the 2004 Lancet study of "cluster clumping" used in a study, much less a mortality study.
Several layers of methodology? Perhaps you can list them and explain why they are different from the Lancet ones? I have to admit having very little statistical knowledge, so if you could make it simple it would be helpful.