The latest pundit to have a go at the Lancet study is Andrew Bolt. Like most of the critics, Bolt just does not have the statistical background to produce a competent critique. In Bolt’s case this is even less excusable, since he had the benefit of the Economist‘s excellent article, but unfortunately Bolt does not seem to have understood it.
Just ask yourself: Have more than 180 Iraqis, mainly women and children, really died every day, on average, for the past 18 months, usually at the hands of the Americans?
If so, where are all the funerals? Where are the pictures? Where are the news reports
The CIA World Factbook reports that the death rate in Australia is 7.38 per 1,000. That’s more than 400 deaths per day. Just ask yourself: Have more than 400 Australians, really died every day, on average, for the past 18 months? If so, where are all the funerals? Where are the pictures? Where are the news reports?
But few of the commentators who seized on the survey bothered to ask such basic questions, or even to heed Human Rights Watch, which warned: “The numbers seem to be inflated.”
Perhaps they should read what Human Rights Watch said later:
“I hate the interview I did for The Washington Post,” he says. “I was on the train, I hadn’t read the report yet [when the Post's reporter called for comment]. In general, I’m not as negative as that [Post] report made me seem. This is raising issues that are not heard of much in the U.S.”
Its researchers interviewed 7868 Iraqis in 988 households in 33 neighbourhoods around Iraq, allegedly chosen randomly, and asked who in the house had died in the 14 months before the invasion and who in the 18 months after.
“allegedly chosen randomly”? The researchers carefully explain, in detail, how the randomization worked. If Bolt wants to accuse them of lying about this, why does he bother with the rest of his attack? Why not just accuse them of making up all the results?
Bolt argues that the pre-war infant death rate that the survey found (29 per 1,000) is wrong because:
2002 figures from UNICEF, which in a much bigger survey of 24,000 households found the infant mortality rate in Iraq before the war was actually a tragic 108 deaths per 1000 infants.
But the 2002 UNICEF figures are based on a survey conducted in 1999 (which meant that it was counting deaths that happened in 1998). And the 1999 survey used similar methodology to the new survey, so the difference can’t be because there was something wrong with the methodology of the new survey—the most likely explanation is that the oil-for-food program lowered the infant mortality rate to between 1998 and 2002.
In March 2002 Matt Welch wrote an article debunking extreme claims of deaths caused by sanctions:
Sanctions critics almost always leave out one other salient fact: The vast majority of the horror stats they quote apply to the period before March 1997, when the oil-for-food program delivered its first boatload of supplies (nearly six years after the U.N. first proposed the idea to a reluctant Iraqi government). …
As the U.N. Office for the Iraqi Program stated in a September 28, 2001 report, “With the improved funding level for the program, the Government of Iraq is indeed in a position to address the nutritional and health concerns of the Iraqi people, particularly the nutritional status of the children.” Even two years earlier, Richard Garfield noted in his survey that “the most severe embargo-related damages [have] already ended.” …
Those who get past the initial frustrations of researching the topic usually end up on Richard Garfield’s doorstep. His 1999 report—which included a logistic regression analysis that re-examined four previously drafted child mortality surveys and added bits from 75 or so other relevant studies—picked apart the faulty methodologies of his predecessors, criticized the bogus claims of the anti-sanctions left, admitted when the data were shaky, and generally used conservative numbers.
So Richard Garfield is the go-to guy on Iraqi infant mortality. Does he think that there is something wrong with the numbers in the Lancet study? Probably not, since he is one of the authors.
Andrew Bolt then demonstrates a lack of understanding of basic statistics. After noting the high death rate in the Falluja cluster he writes:
Truly, these statistics are unbelievable. I suspect the study’s authors thought so, too, which may be why they left the Fallujah figures out—calling them unrepresentative—when they calculated Iraq’s death toll since the invasion.
But the survey techniques they used to give clearly wrong figures in Fallujah are the same ones they used in the other 32 clusters of households that they interviewed elsewhere in Iraq.
An estimate based on Falluja is unreliable because it is based on one cluster, and there is a good chance that that cluster might not be representative. But it does not follow that because an estimate based on one cluster is unreliable that an estimate based on 32 clusters is unreliable. In fact, the size of the sample is the crucial difference—you need a reasonable sample size to get a reliable estimate. If you believe Bolt’s argument, then all surveys can be similarly dismissed. For example, a poll on voting intentions that just surveyed one random person is clearly going to give an unreliable result. Does it follow that a poll of a 1000 people must also give an unreliable result?
Bolt finishes with:
Too many commentators seem too desperate to believe the worst of the Americans and to belittle the liberation of Iraqis from a tyrant.
That desperation means even junk surveys such as this will find many eager believers, ready to hear the very worst. And to recklessly repeat it.
Too many war supporters have a desperate need to deny that the Iraq war has had some bad consequences. This desperation means that even clueless critiques, such as Bolt’s will find credulous believers. This is unfortunate. I believe we need to be clear-eyed about the consequences of the war and face up to reality.
Update: Bolt’s response:
I’m sure you don’t need me to point out the obvious omissions, red herrings, false comparisons of additional deaths to total deaths, and the telling failure even to cite the shocking figure of Iraqi child deaths given even by the “expert” cited as the last word on the issue.
Of course I did cite the infant mortality figures. I don’t know why Bolt would write something so plainly false. And did you notice the clever clever way he used scare quotes to disrespect Richard Garfield?
Update 2: Tim Blair thinks that Bolt’s hopelessly innumerate criticism proves that Peter FitzSimons is stupid for citing the 100,000 figure. Unfortunately, all Blair has proved is that he himself has no understanding of basic statistics.