Tim Blair, whose reaction to the Lancet study was to reject the entire concept of random sampling offers us this:
Among other Lancet critics: Paul Bolton, a professor of international health at Boston University; Stephen Apfelroth, professor of pathology at the Albert Einstein College of Medicine in New York City; and mortality studies expert Richard Garfield.
Of the three, the only one who is an expert in mortality studies is Richard Garfield. What does he say?
The Lancet study cited two sources for the 5.5 pre-war mortality rate: the 2003 CIA Factbook entry for Iraq and a 2002 profile from the U.S. Agency for International Health. (By comparison, the mortality rate for the United States is, 8.3; the United Kingdom, 10.1; Russia, 14.6; and South Africa, 22.) But those sources, according to Bolton, are not rock solid. In the mid-1990s, for instance, the United Nations Population Division reported an Iraqi mortality rate of 10 per 1,000.
"I have no sense as to why they (the Lancet authors) are so keen to believe the data" he said. "We know people had reduced access to food, medication and health care (during the time of United Nations sanctions from 1991-2002). People tried to make mortality estimates, but access to Iraq was so limited."
The last reliable census done in Iraq dates to 1987, says Richard Garfield, a professor of nursing at Columbia University and an expert in mortality studies, adding, "All pre-war estimates are guesses -- it's all projections."
Garfield isn't criticising the Lancet study -- he's defending it. The CIA, USAIH and UN numbers are all guesses. The Lancet study actually measured the pre-war mortality rate. Measurements should usually be prefered to guesses. And in this case it's not even possible for the Lancet study to agree with all of the guesses since they contradict each other, so the best it could do is agree with two out of three. Which it does. And the new Lancet study does agree with the only other measurement of pre-war Iraqi mortality since 1987, the 5.0 rate from the first Lancet study. Garfield is probably aware of this because he was one of the authors of the study.
So much for Bolton's concerns, what about Apelroth?
"Cluster sampling is typically used for something like presidential polls, which involve a list of every household and you can choose your sample truly randomly," said Stephen Apfelroth, professor of pathology at the Albert Einstein College of Medicine in New York City. "But (Iraq) is entirely atypical. Violence occurs in very restricted areas and can vary block to block. The number of clusters sampled is inadequate given the wide geographic variation to be expected with violent death."
Apfelroth admits he is not an expert on cluster sampling, but believes the Lancet's estimate is so extreme it does more to disprove the method than support it. "The statistical community has not come to a consensus on the best method for estimating violent deaths," he said.
"Apelroth admits he is not an expert on cluster sampling". Says it all really, doesn't it? Apelroth is entitled to his opinion, but why is it in this story? Why didn't the reporter stop him there and go and and find an expert on cluster sampling and get an informed opinion on the matter?
Yes, violence tends to cluster, but then so does voting intentions (that's why gerrymandering works). This does affect the number of clusters you need for a good estimate. So if you are designing such a study, it's probably a good idea to consult an expert on cluster sampling, not Apelroth, to see how many you need. Which they did. And if the expert makes a mistake and you don't have enough, it shows up in your results as a confidence interval that is too wide. Which didn't happen.
I was struck by this particular response to the Blair article, #48 in the responses there, from a guy arguing that tthe entire survey was faked, and the surveyers enver actually collected any data:
---
"1. IF they really went door to door as claimed, there'd have been deaths. IF there had been deaths it would have been ALL OVER the media, print, tv, radio, blogs, non stop...why? Because it would have been put forward as proof that the US/Coalition/Iraqi Gov was trying to squash/squelch/eliminate dissenters. That's a fucking given.
The faces, names and gruesome details of the door knockers deaths wasn't front-line news for days and days, therefore I have zero reason to believe door knockers were put into the street."
---
So he is arguing that Lancet are wrong about the 600,000 deaths, becasue ther is such a high risk of death in Iraq that surveyers would have inevitably been killed and we would ahve heard about it - so the high risk of death somehow proves that there could not have been that many deaths.
Got it.
It's pure and simple denial. Nothing for it really except to ignore it.
I enjoyed the other part of TB's posting.
"Labor's Kevin Rudd puts the death toll at 50,000, some 605,000 below the Lancet study's estimate."
Which is technically true. But if you want the whole truth, you might want to say:
"Labor's Kevin Rudd [in an interview before the Lancet study came out] puts the death toll at 50,000, some 605,000 below the Lancet study's estimate."
Hey Lambert,
It looks like you might have some name calling you might want to retract.
http://www.iraqbodycount.org/press/pr14.php
Iraqi body count doesn't believe the study.
"1. IF they really went door to door as claimed, there'd have been deaths. IF there had been deaths it would have been ALL OVER the media, print, tv, radio, blogs, non stop...why? Because it would have been put forward as proof that the US/Coalition/Iraqi Gov was trying to squash/squelch/eliminate dissenters. That's a fucking given.
The faces, names and gruesome details of the door knockers deaths wasn't front-line news for days and days,"
I guess Steve Moore faked those surveys he claimed to have conducted in Iraq in the pasy year and so did the UNDP.
Hey wacki, get back in the time machine. The IBC stuff was dealt with days ago.
heh, that's what i get for drunk posting. Ok I sleep now.
Garfield is mentioned once again in the linked piece, as follows:
Still, the numbers in the Lancet report continue to raise eyebrows.
"I'm shocked by the levels they (the investigators) reached," said Garfield. "Common sense, gut level, says it is hard to believe it could be this high. We don't know how many have died, we just know it's a lot. ... Right now, the only other option is to stay in the dark."
Common sense, gut level...what about facts? Timbob, why not rephrase that? "I don't have any evidence at all, but I don't want to believe this because it is uncomfortable?"
tim, have you seen this?
"More than three decades ago, two psychologists conducted an experiment that was equal parts funny and deadly serious.
They spun a roulette wheel and when it landed on the number 10 they asked some people whether the number of African countries was greater or less than 10 percent of the United Nations. Most people guessed that estimate was too low. Maybe the right answer was 25 percent, they guessed.
The psychologists spun their roulette wheel a second time and when it landed on the number 65, they asked a second group whether African countries made up 65 percent of the United Nations. That figure was too high, everyone agreed. Maybe the correct answer was 45 percent.
The difference in the estimates of the two groups was tied to the original number they were given. It made no difference that the number was meaningless: It came from a roulette wheel. Psychologists Amos Tversky and Daniel Kahneman described the error as caused by a phenomenon known as anchoring -- when you don't know the answer to something, whatever starting point you have plays a powerful role in determining what you think is the right answer."
Exactly, Alex, perfectly put.
But actually, the IBC figure is nearly 50,000, and that is from deaths only reported in the media. The idea that the real figure is ten times higher seems entirely plausible to me.
But actually, the IBC figure is nearly 50,000, and that is from deaths only reported in the media. The idea that the real figure is ten times higher seems entirely plausible to me.
While I have to admit I agree with Mark's statement, I also have to admit that it's a variant of the same misguided critique lodged by most of the Lancet study's critics -- i.e., it's based on what "feels right" or "seems plausible". Let's be honest: The number of people who have a accurate "feel" for what the death rate in a situation like Iraq is, or for how large the discrepancies are between actual and reported numbers (in the media, through governmental channels, etc.), has got to be pretty damn near zero.
And that's why people interested in the actual numbers, like the Lancet authors, go out and measure the data. And that's why, until a better study comes along -- or until someone can actually demonstrate a fatal flaw in the Lancet study, instead of just raising niggling questions -- I'll accept the Lancet results as the best measurement we have.
Marke: The scandalous truth is that JH/Lancet preferred to to rely on the CIA (hardly renowned for its demography brilliance) for validation of its pre-invasion crude mortality rate of 5.5-6.00 rather than the WHO with its unchallenged rate of 9.26 in 2001 derived from life tables, fully consistent with rates in a neighbouring country like Iran not subject to sanctions etc. The Johns Hopkins study is a disgrace because of its failure to admit the existence of the WHO data, despite the WHO surely having more competence in this area than the CIA. Using the WHO crude mortality rate more than halves the JH extrapolation to 650,000 "excess" deaths. Taking into account the Cockburn data (The Independent) on emigration from Iraq since the invasion might well reduce the "excess" by another 50%. All same, the JH "study" by failing to accept formally that most deaths in Iraq since 2003 have been a result of secticide has made no contribution to a constructive solution to (on its count) the worst genocides since WW2.
Alex writes: "Common sense, gut level...what about facts? Timbob, why not rephrase that?"
Those weren't my words to rephrase, Alex. They were the words of Richard Garfield, whom Lambert believes (based on his previous quote) is supportive of the Lancet study. My point in posting that quote was to indicate that Lambert was wrong, at least so far as we're aware from the linked article.
As usual, I look forward to Lambert's correction.
Err, tim, your reading comprehension isn't too good. Garfield is saying the estimate is shocking but that it is the best estimate we have.
And Tim Curtin, the UN number for Iran is 5.5. I'm not sure why you think that proves that rate in Iraq could not be 5.5.
Well, it's not a question of the accuracy of figure, the accurate figure is of the order of 650,000 using evidence-based research. It's a response to the idea that such a huge number "seems" or "feels" too high. My point is that even a layperson sitting in one's loungeroom watching the news from Iraq on tv could deduce that the figure "feels" about right.
Which is still an obscene figure.
If the emigration is of such scale that it can skew the demography of a country to that extent, then it raises the question of why are they leaving? Which brings us back to the real issue, which is that obviously Iraq is a disaster, and the people of this new democratic Iraq which was promised are "voting" with their feet.
No, no, Tim Blair is right. Garfield is all in favour of using the intestines to make deductions. Tim Blair knows how it is. When the gaseous pressure builds up on his sphincter, a blog post is imminent.
Blair, you hopeless yokel, Garfield is saying that 1) you can't depend on your gut for a measurement of this scope and scale, because it truly defies "common sense"; 2) that the study was designed to gauge whether the level of casualities is higher than has been reported, not to provide a precise point estimate and 3) that your only alternative in rejecting the study is to remain in ignorance.
What Garfield said: "Common sense, gut level, says it is hard to believe it could be this high. We don't know how many have died, we just know it's a lot. ... Right now, the only other option is to stay in the dark."
Seems to me this qualifies Garfield as a Lancet critic. Is that why you omitted this quote in the post above?
Seems to me that you don't understand what Garfield is saying.
Oh, and why did you leave out the fact that Rudd's comment was made before the new Lancet study was released?
Re Garfield: why not just add his additional quote to your post?
Re Rudd: 50,000 remains his most recent estimate. He hasn't altered it post the Lancet claim. He hasn't even mentioned the Lancet claim, so far as I'm aware. Nor has Robert Fisk, which might tell you how believable their estimate is.
Oh, I think I'll put the additional quote in a whole new post. I get to use the pinata picture again.
You'd best use that delightful pinata to identify yourself, considering both pro-war and anti-war camps now recognise you as a liar. Let the battering continue, and the candy to rain on down!
I've got a better picture for you, Tim.
So Garfield says that when you are faced between your gut and data, go with the data. Tim Blair thinks this refutes the data. So Tim Blair can neither understand math nor can he read. Exactly where does his qualifications lie?
Tim Blair has excellent proof-reading skills. And he actually knows stuff about cars.
MArk: "But actually, the IBC figure is nearly 50,000, and that is from deaths only reported in the media. The idea that the real figure is ten times higher seems entirely plausible to me."
Posted by: Glenn : "While I have to admit I agree with Mark's statement, I also have to admit that it's a variant of the same misguided critique lodged by most of the Lancet study's critics -- i.e., it's based on what "feels right" or "seems plausible". Let's be honest: The number of people who have a accurate "feel" for what the death rate in a situation like Iraq is, or for how large the discrepancies are between actual and reported numbers (in the media, through governmental channels, etc.), has got to be pretty damn near zero.
And that's why people interested in the actual numbers, like the Lancet authors, go out and measure the data. And that's why, until a better study comes along -- or until someone can actually demonstrate a fatal flaw in the Lancet study, instead of just raising niggling questions -- I'll accept the Lancet results as the best measurement we have."
In the article in the Lancet, the authors reference previous studies, in previous civil wars, that found that a good rule of thumb was that 20% of deaths in civil wars are reported (I imagine that that means in the local media). So there's actually a double-check on the survey:
Iraq Body Count Project tally * 5 = 50K*5 = 250K, just for ciliivans killed by violence.
That provides an order of magnitude level confirmation.
Psychologists Amos Tversky and Daniel Kahneman described the error as caused by a phenomenon known as anchoring -- when you don't know the answer to something, whatever starting point you have plays a powerful role in determining what you think is the right answer."
Watching Rick Steves' travel show last night he had an example of this when he suggested that, before bargaining in a Cairo market, you first get in your mind an idea of how much the item is worth to you before the seller's opening price affects your judgement. It's interesting that this is so easily done as in the example you provided.
SJ,
Completely off topic, but since when do roulette wheels have a number 65?
I suppose it shouldn't be surprising that we witness the enormous amount of shame Tim Blair and his boyz are projecting, when you consider that they by now know that everyone sees them continue to push themselves forward as willing groupies for leaders who, they know themselves, lie brazenly and habitually about everything while totally screwing the pooch. Machiavelli be damned - he'd be sneering at demonstrations of such surpassing stupidity.
Not at all gratifying to see though is it, that tim blair's embarrassed reaction - in the footsteps of the similarly embarrassed and overheated Josh yesterday - to being corrected (again) by Tim and Sven is to call them liars? Would tim's mother be proud of him today?
Abe G - it wasn't a real roulette wheel. It was "a roulette-type wheel labeled with the numbers 1-100".
"...the worst genocides since WW2."
One assumes Tim Curtin is planning to testify for the defence in the upcoming war crimes trials for the surviving Khmer Rouge leaders.