The defective refutations of the Lancet study just keep on coming. First, we have Gerard Alexander writing in the Weekly standard:
But the study's researchers were sure to survey in Falluja, far and away the most violent city in post-invasion Iraq. Falluja turned out to be such a wild statistical outlier that they offer two estimates, one with Falluja included and one with it kept out. But questions about just how representative the sample sites were go deeper than this. The researchers selected their survey sites households for such unclear reasons that we simply can't extrapolate to the whole country with any confidence. What are the chances that they have over-sampled the most violent parts of the Sunni Triangle and under-sampled the calmest Kurdish and Shia areas? Without better statistics about population and violence, we can't know, and neither can they. The fact that they don't explain their strategy doesn't build confidence in their research design or their conclusions.
Alexander comes from Fumento's don't-bother-to-read-the-study school. If Alexander had bothered to read the Lancet study he could not possibly have missed their lengthy and detailed explanation of how they randomly selected the clusters. After this gross deviation from the standards of scholarship, Alexander then has the hide to accuse the Lancet of jettisoning "their standards of fairness, restraint, objectivity, and integrity."
Next, we have Robert Lichter of STATS who writes:
The crucial assumption is that any increase in deaths after the invasion began on March 19, 2003 is associated with the conflict and subsequent occupation, to the exclusion of any other factor. Specifically, their sample included 46 reported pre-invasion deaths, only one of which was violent, and 89 post-invasion deaths outside Falluja, 21 of which were violent. According to a table that breaks down the causes of death, fewer than half of the "excess deaths" (45 percent) resulted from violence. One in five was accidental, one in six was due to heart attack or stroke, just under one in 10 was caused by infectious disease, and the same proportion consisted of neonatal or infant deaths. Yet all these deaths without exception were attributed to the war and occupation.
It seems pretty unlikely that a doubling in the number of deaths was just a coincidence. If Lichter wants to suggest that something else caused it, he needs to explain what. And I hope it is clear how the disruption to medical services and electricity and water could cause increases in deaths from diseases.
The Economist and Stephen Soldz have discussions of the study that are more balanced than the hatchet jobs by Alexander and Lichter.
Update: I sent my comments to Alexander and he replied:
I appreciate your certainty that I haven't read the study, but am afraid its misplaced. I'm even sorrier that you missed my point: in the absence of better stats, especially on violence, no-one---no matter how lengthily they think they're explaining their methods---can survey a sample in whose representativeness we can have confidence. It also means we can't exclude the study's findings as accurate, which is why I don't dismiss them, though I question them. What I got most specific about was the Lancet's apparent disinterest in their own previous published estimates of child/infant mortality before March 2003. I'm not dismissing those previous findings without explanation; they are.
So Alexander read the extensive explanation of their strategy for selection and then accused the authors of not explaining their strategy. Readers will have to think of the best word to describe Alexander's conduct. Now he claims that it is not possible at all to get a representative sample. He doesn't explain why it is impossible, probably because he cannot. Nor do they dismiss previous findings on infant mortality without explanation. They clearly state:
The Coalition Provisional Authority and the Iraqi Ministry of Health have identified the halving of infant mortality as a major objective. In the absence of any surveys, however, they have relied on Ministry of Health records. These data have indicated a decline in young child mortality since February, 2001, but because only a third of all deaths happen in hospitals, these data might not accurately represent trends. No surveys or census-based estimates of crude mortality have been undertaken in Iraq in more than a decade, and the last estimate of under-five mortality was from a UNICEF sponsored demographic survey from 1999.
In other words, infant mortality has decreased since it was last measured.
Alexander claimed that the Lancet chose to throw away its reputation with the article, but it is Alexander who has thrown his own reputation away with his disgraceful article in the Weekly Standard.
If you've got a weak heart, then very few doctors prescribe exposure to aerial bombardment!
The Economist put it succinctly, and correctly, near the end of their article:
I have read a good deal of the conservative commentary swirling aroung the Lancet's study, and I have not yet come across a single comment suggesting that a better study be done. If anyone knows of a warblogger recommending that we get good numbers on Iraqi mortality, please point me to it.
It seems a reasonable thing to ask, doesn't it? If the Lancet's numbers are wrong, then what are the right numbers, and how can we get them?
Perhaps supporters of the war simply have not yet had time enough to get over their indignation with this study. Maybe, any minute now, the conservative world will rise up as one and call for a better estimate of the number of Iraqis killed by the war.
Or maybe not. Call me cynical, but I suspect they don't want to know, and this show of spectacular outrage over what was a pretty good study, considering the circumstances , is just a way to spread a cloud of doubt over the whole business of estimating mortality.
jre, good point: you can take your observation, pick it up, and put it down in a number of topics that conservatives express outrage over. And you would be correct there too.
Where are the alternative studies that conservatives/warbloggers/whatever fund to empirically back their ideological worldviews?
Nowhere.
D
Indeed. Tim was slightly wrong with his post title below; this Lancet study isn't "fly paper for innumerates" - it's litmus paper for hacks. The divide in my mind between rightwing commentators who I regard as fundamentally honest people and those I regard as worthless hacks has matched up perfectly to those who played the man on the Lancet study and those who expressed some reservations about the number, but admitted it was a quality piece of work.
Coming from the legal side, I look to a simpler principle of evidence; information that you refuse to divulge can generally be taken to be adverse to your argument. It's not believeable that the US has made no estimate of Iraqi deaths, if only in planning for the vote; they refuse to divulge that estimate; ergo, they believe it's at least as shocking as the Lancet figure, and I am entitled to believe the same.
Again, the concept of using the best evidence is surely a keystone of the scientific approach; you can't beat something with nothing, and the Lancet figures (whatever their imputed flaws) must stand until a better alternative is produced.
Neither of these arguments rely on statistical inference, but they'd swing most juries.
Well, Brignell is a crank, rather than a hack, so I wanted a category that vovered him as well.
We also should get R&D to work developing litmus paper for innumerates and flypaper for hacks.
You've all hit the nail on the head. The US and UK have done everything in their power to block even Iraqi efforts to estimate the number of people killed during the past year from allied bombs. Its obvious why - because as long as there is no actual number provided, then there is no grist for the mill. In other words, the number might as well be zero. Most importantly, it makes it easier for the aggressors and occupiers to deny that there was any slaughter at all.
I quite like the idea of litmus flypaper, to tell you whether you're catching wasps or bees (I dimly remember being told that you should put vinegar on a wasp sting but soda on a bee?)
I think d2 is on to something...litmus flypaper for hacks.
Indicators of wasps or bees could be key phrases, for instance. An entire cottage industry could spring up to attract hacks...
D
Or is it the other way round? Obviously the proper thing to do is to get a wasp to sting a bee sting and vice versa. Then you don't have to remember the pH at all.
Now if Tim really wanted to prove to Michael how important he is by blog hits, this is a good point to just put up ilovebess.com.
or ilovebees.com rather
Of course, a weak-minded hack like Fumento would necessarily end up at ad hominems, but to his credit, I believe he's invented an entirely new form of fallacy. Im not sure what to call it when you meet claims of inaccuracy with the counterclaim that you're in wider circulation.
He doesn't even claim that he might be right because he's printed more; it's more along the lines of "my version will be published more, regardless of the merits, ergo it will be 'truer'."
I doubt that the Romans had a categorization for such a Pomo 'fallacy'.
Wu
Carleton:
That is a brilliant analysis and very relevant in the current political climate of the US. The current administration eschews what has been called the "reality-based" community and obviously Fumento adopts this line of reasoning.
"My version is not true but by saying it over and over again it becomes "truer" and by the time you analyze it enough so that it becomes obvious to all, I'll have moved on to another topic."
However you are correct, we need a name for this fallacy. My vote is the VITO VERUS fallacy, which by my very poor Latin means to avoid reality
Others are encourages to add (but nothing with Fumento or Bush please).
Regards
Yelling
How about the Vox latus propaganda fallacy?
D