Flypaper for innumerates

They have a companion paper here to counter most of the misinformation.

In it they cover the topic that I brought up a few posts ago that the UN had released a study from ministry of health statistics which showed violent civilian mortality peaking in the summer at little over 100 a day. We should note this study didn't include kurdistan and Al-Anbar reported no deaths. I raised the question of MoH statistics accurately got all deaths that recieved death certificates. They argue:

The Ministry of Health in Iraq has published some numbers from time to time, but these are generally
considered to be unreliable. The registration of deaths in Iraq has been an organized process for many
years. Death certificates have traditionally been obtained for the deaths of all adults and older children.
Death certificates are required for insurance claims, compensation, payment of benefits, and for burial.
Cemeteries do not take bodies for burial without certificates. If deaths occurred outside of hospital, the
bodies would be transported to the general hospital for the certificate to be issued. If there were doubts
about the cause of death, a post-mortem examination would be carried out before issuing a certificate.
Copies of the death certificates would go to the national offices managing vital registration.
This process has continued through the current conflict, with death certificates being required for burial,
and with information from certificates being duly recorded. However, the tabulation of data from registration
of deaths in Iraq has suffered from the chaos of the current conflict. Beyond this, there is also a
suspicion that records of death, particularly related to violent deaths, is being manipulated and only
partially being released for various political reasons.
Even with the death certificate system, only about one-third of deaths were captured by the government's
surveillance system in the years before the current war, according to informed sources in Iraq. At a death
rate of 5/1,000/year, in a population of 24 million, the government should have reported 120,000 deaths
annually. In 2002, the government documented less than 40,000 from all sources. The ministry's numbers
are not likely to be more complete or accurate today."

If thier 1/3 of all deaths are reported then the two would be in the same ballpark. - Andy

One thing you're going to find (you may have already) is something I saw on a forum about science and evolution. This was the claim that ___ had been "debunked" and was defended by changing the plain meaning of "debunked" to "disputed". So if I claim that Tim Lambert is not at a university in Australia, I have "debunked" the claim that he is there, even though my claim is obviously false. I've disputed that he is, but claim I've debunked that he is.

This seems to work quite well with various types of deniers, so expect it.

This seems to work quite well with various types of deniers, so expect it.

I've expected this for some time. It's a standard denialist tactic: "I've given you any ol' answer, therefore I've debunked your claim. Never mind all that fancy fact stuff you spew."

Best,

D

So, will Joe Barton get his team of crack statisticians to investigate the scientific basis for Bush's proclamation that Cluster analysis is a discredited methodology?

Patiently waiting... Mitch

Cluster analysis?

I believe Bush must have been thinking of "cluster fucking" (what he and others have engaged in in post invasion-Iraq) when he said "the methodology has been pretty well discredited".

I think bush was referring to cluster bombing.

Sorry children, but the last study was debunked. Unfortunately once your credibility is shattered, coming back just in time for the elections with another headline grabber that just coincidentally happens to fit into their loudly proclaimed anti-war agenda may raise suspicions. Numbers, no matter how you cook them up, are excellent propaganda, and for the same reason numbers are more often than not the hostage of an agenda. It hardly bears noting the curious capacity of advocacy groups to produce findings that strangely enough support (and support extremely) the very position they just happened to advocate, as if the result was somehow completely fortuitous. If you start out from an staunch anti-war position than exactly how hard has it ever been to fudge the results, to overlook problems, to circumvent reasonable objections, and simply to let the agenda drive the conclusions derived? Lies, damn lies, and statistics.

You know, Ians comment sounds very like the anti-evolutionists comments I am used to reading. It says nothing substantial, just states some broad factoids, (Yes, we know advocacy groups happen to produce information that agrees with their aim) without actually considering the specific case in hand.

So what exactly is wrong with the current study, Ian?

(And I have to get in the obligatory snark about WMD's, and how it seems they dont exist after all)

Ian, do you have a point? What's wrong with the study?

Well done, 'ian', for your substantive critique.

I'm most interested in the extent to which the new survey validates the distribution of the 2004 survey, given the amount of BS poured upon it at the time. Alas, it'll lead to the old 'damned if you do, damned if you don't': had the new study come up with a different number, the denialists would be crowing about its inaccuracy. Now that the two essentially agree, the denialists are pulling out the old canards to claim that the new study is just as flawed.

Perhaps we should offer them a new statistical model, based upon a 'maximum death ickiness': that is, the total number of dead Iraqis that our denialists are prepared to concede. Let's at least systematise their arguments.

Number of empirical field studies the denialists have produced obtaining their own number and/or debunking Lancet studies: zero.

Slope of rate of increase of expected number of studies: zero.

Number of bytes of bandwidth expenditure by denialists "debunking" Lancet studies: bigbig number

Slope of rate of increase of bandwidth expenditure "debunking" Lancet studies as Murrican Election Day approaches: rising asymptotically.

This public service announcement brought to you by: any average thinking person on the street and Michael Moore (TM).

Best,

D

If there were something wrong with the Lancet study, I somehow doubt Ian would be capable of saying what it might be.

Paroting the "Lies, damned lies and statistics" quote reveals that he simply does not understand the nature and purpose of statistics.

Those who know something about science appreciate that statistics play a very important role in the scientifc process and can sometimes reveal information that can not be obtained in any other way.

So we meet again.

I challenge you to a duel, Sir Lambert, one that I am sure you cannot refuse.

Wow. It only took 5 comments for someone to use the disputed means debunked claim I mentioned in my comment.

The report states that 'empty houses or those that refused to participate were passed over until 40 households had been interviewed in all locations.' This is a problem because hundreds of thousands of Iraqi families have fled their homes. Had those families that fled the clusters that were sampled been included in the survey, the estimated death rate would have probably been lower. Also absent from the report is the estimated death rate prior to January, 2002.

As always, the question that counts is "if you don't accept this survey, what numbers do you estimate?" The American government won't make a guess, and the only possible reason for that is that they think it's in their own best interests - i.e. that the actual figures are as high or higher than the lancet figures. If they could make a defensible case that the actual figures were substantially lower, they would.

So, Ian, what is the 'correct' increase in mortality rate in Iraq since the invasion? Don't you want to know? Do you even really care? Would it really change your mind about the righteousness of the invasion if it were proven that it had cost the lives of half-a-million innocent people? Exactly how much death would it take to turn you off of this war? Seriously. I say you should embrace this information, take it in stride, and just stick to "freedom is messy" and "you have to break a few eggs to make an omelet."

You guys are hilarious. So because we can't come up with another number, namely because it's virtually impossible to come up with an accurate one, then anything goes?

Why not just say 10 million and be done with it?

Bush has given a number - 30,000. That was last year. So it might be time to stop pretending that "the American government won't make a guess" - they already did.

As always, the anti-war liberals insert speculation and pure lies to fill a vacuum that cannot be filled with reliable and accurate information. It's like a game.

It's like a game.

For you, perhaps.

So because we can't come up with another number, namely because it's virtually impossible to come up with an accurate one, then anything goes?

Ah, the Pontius Pilate defence. The 'anything goes' approach is yours, Sickson, because no number satisfies your gut. That's the kind of thing one expects from five-year-olds refusing to eat their dinners. Still, it takes a certain something to keep bullshitting for two years straight.

Onto substance:

The report states that 'empty houses or those that refused to participate were passed over until 40 households had been interviewed in all locations.'

As were households in which all members had been killed.

I'm willing to bet cash money that the number of families that have fled their homes is much, much higher than the number of families in which all members had been killed. I honestly don't know how this affects the results of the study, but one would think that if a large percentage of households in sampled clusters were empty because the families fled, then the survey is not truly representative of those clusters. Also I wonder how many households in each cluster refused to participate and why.

As an actuary I can tell you part of the problem you run into with Joe Layman and real statisical analysis is that he has a hard time with any probability that is not 100% or 0%. In my experience, most people see only two possibilities: certainty and cluelessness, the latter a state with which many sadly have great familiarity. They don't know from confidence intervals and logorithmic fits, and the results of such studies constantly conflict with their "gut", and we all know what happens when facts conflict with the gut: the facts be damned. Just start a conversation at your next dinner party, preferably one that you don't care to get invited back to, about the Monte Hall 3 doors problem, and you'll see what I mean.

Since all statistical studies are gibberish to the Ian's of the world, they are all equally worthless, malleable to any desired agenda. Never mind actually showing the agenda exists or had an effect on the work. The mere possibility of a methodological bias is sufficient to reject the conclusions. It is as if students in statistics classes were told "Do whatever you want".

"If thier 1/3 of all deaths are reported then the two would be in the same ballpark. - Andy"

Andy, 1/3 of 650,000 is 216,666. In July of this year the MoH & MLI in Baghdad had documented 50,000.

If their "1/3" is accurate, then there have been about 150,000 deaths, which is about a mile short of 650,000 and even a mile short of the lower bound 426,000.

But their 1/3 is not accurate to begin with. The MoH recorded over 80,000 in 2002 (excluding Kurdistan too), not 40,000. And the "should have reported 120,000" is somebody's uncited estimate of the number of deaths in Iraq in 2002, presented misleadingly as if it were a hard fact.

These are not even in the same city, let alone the same ball park.

Bush has given a number - 30,000...

As I recall, Bush cherrypicked this from the lowest conservative estimate, perhaps even conflating the IBC and UNDP studies. Both of which are now rather dated, and indeed were so when Bush made his "estimate".

I think the design may have some problems if the data exhibit spatial correlation, which seems a reasonable assumption phenomenologically (violent death in a war zone being inherently spatially correlated). Starting with a randomly chosen hh within a cluster and then going to adjacent hhs doesn't seem to me to produce the kind of within-cluster heterogeneity one looks for in a cluster sampling framework. In wildlife surveys (same application, to get counts data) one often moves a minimum distance from one observation point to another within a cluster.

And to anticipate the WTFAY question, I'm an econometrician. Profs. Flinn, Peracchi, Wolpin and Ramsey might still consider me innumerate, but I passed their classes.

I'm not dismissing the study, but if this were someone's job market paper I'd be asking to see everything. And tomorrow I probably will ask for everything...

And Tim, people can be skeptical of the methodology without being a bunch of fundie nutjob "deniers." The zeal to characterize the other side in such condescending terms doesn't wear well on either side of the debate. Sounds like some of your commenters could use a reminder.

One thing I don't quite understand about this survey (and there may be cultural or legal reasons for it), is that the study states that about 92% of all households visited were able to present death certificates. This suggests that if the figure in question is roughly accurate, the Iraqi government should certainly have statistics relating to it, correct? Or do they not gather such data?

If the figure of ca. 600,000 is accurate, has anyone noted that both the absolute number of deaths and the percentage of the population killed (ca. 2.5%) would be eerily similar to the figures for the US Civil War?

I haven't seen the actual study, though the supplementary document provided by Andy Barenberg helped answer my last question. (Thanks, Andy.) Does the study in the Lancet break down cause of deaths by gender? Does it say, for example, what the male/female ratio is for those killed by gunshot or by airstrike?

"Iraqi American" - the estimate of pre-2002 mortality is 5.5 per thousand per year. This figure matches the CIa's own estiamte for that period. It features prominently in the reprot. The 600,000 figure is the estimated INCREASE over the pre-war death rate.

Also, I would have thought that families that lost members would be more likely to flee, suggesting that this would bias the estimate downwards rather than up as you suggest.

At this point I feel like anyone denying the deathtoll in Iraq is well over 600,000 is exactly the same thing morally as denying the holocaust. Worse even for Americans, because it is our government doing it, and that makes us partially responsible

Does the study in the Lancet break down cause of deaths by gender? Does it say, for example, what the male/female ratio is for those killed by gunshot or by airstrike?

Yes for the former, no for the latter. There's a methodological critique at DKos that raises questions about potential oversampling. And I agree with those who would like to see the dataset. But I also think it's important to weed out the hack critiques in a forceful way to prevent them dominating the debate.

Actually, I'll correct myself: Table 2 in the Lancet paper has the raw death totals by sex and cause of death.

DrSteve wrote:

I think the design may have some problems if the data exhibit spatial correlation

Well, that would make the estimates inefficient but why would it systematically bias the estimate upwards? In any event, the authors appear to have been aware of it--see the discussion on the left hand column of page 3 of the paper.

Iraqi American wrote:

Also I wonder how many households in each cluster refused to participate and why.

A total of 15 households of 1849 refused to participate. And, are you saying that the households that fled had lower mortality than those that stayed?

Two things. How many people would have died at the hands of Saddam Hussein if he were still in power? Also, how many of the deaths the Lancet cites are the result of American military action and how many are a result of insurgent attacks and suicide bombings?

The following links may (or may not) fully explain these numbers:

http://www.logictimes.com/body%20count%20Jul%2006%20ed%20note.JPG

From: http://www.logictimes.com/antiwar.htm (I do not necessarily agree with some items on this site, but they do make a few good points here and there.)

and:

"Who's Really Killing Iraqis?

The Real 2006 'Iraq Body Count'

Iraqi civilians killed this year by Islamic Terrorists:

9,812

Iraqi civilians killed collaterally by Americans:

64*

*Source: IraqBodyCount.net (includes civilians caught in crossfire who may have been killed by the terrorists)"

From: http://www.thereligionofpeace.com/ (Caution: contents may offend some viewers.)

Seixon wrote:

So we meet again.

Find the [elephant](http://timlambert.org/2005/10/elephants/) yet?

Ian Gould, the 5.5 per thousand number is for the 14 months prior to the invasion, ie 2002, not pre-2002.

Stephen Berg, the study apparently attributes exactly zero deaths in all the households surveyed to government action or other violent acts in the last 14 months of the Saddam regime, suggesting that, if Saddam was still there, all those people who died violently would still be alive today. A surprising finding, sure, but that is what the survey says.

Two things. How many people would have died at the hands of Saddam Hussein if he were still in power? Also, how many of the deaths the Lancet cites are the result of American military action and how many are a result of insurgent attacks and suicide bombings?

Impossible to say, as we don't know what the trajectory of his policies would have been. At the rate things were going, roughly this estimate fewer, since this is a count of excess mortality

As to #2: Does it matter? The "insurgent attacks and suicide bombings" are part and parcel of the policy trajectory set in motion by the Bush Administration when they invaded. One doesn't have to claim them to be wrong to have done so, although I do, to point this out. One needs to be a fantacist to deny it.

Stephen:

"How many people would have died at the hands of Saddam Hussein if he were still in power?"

&

"Also, how many of the deaths the Lancet cites are the result of American military action and how many are a result of insurgent attacks and suicide bombings?"

Q1: The study estimates the INCREASE in violent deaths since Saddam was overthrown. So your question was already answered by an even cursory read of this post or the Lancet piece itself. Like, just the abstract:)

Q2: American military action has destabilised the country, so both of your categories of violent death are "the result of American military action". Said military action has reduced (to zero) the number of deaths caused by Saddam's regime. This is good. However, the total number of deaths appears to have risen sharply. That is what we are discussing here.

Are you suggesting that this would have happened anyway? That's a pretty crazy assertion.

Ian and his ilk actually don't care how many deaths in excess of the preinvasion death rate have occurred. The point was made with admirable simplicity two years ago by a white House spokesperson-'We don't do Iraqi body counts'. The question to Ian is this-forget about the soft left totalitarian BMA body count guesstimate. How many Iraqi deaths have the CoW noted and accounted for since the invasion-oops-liberation of iraq? If no work has been done on this, why not? Is your position then that Iraqi deaths are not worth recording?

Oops, too slow.

Retreading old ground:

How many people would have died at the hands of Saddam Hussein if he were still in power?

Compared to earlier periods, Saddam wasn't killing that many Iraqis in 2003: that is, he wasn't conducting large-scale repression (the Anfal) or involved in a large-scale war. The distribution of 'died-at-the-hands-of-Saddam' is not rectangular. The best response, courtesy of Daniel Davies, is to note that HRW did not consider invading Iraq a humanitarian intervention -- that is, one which would save more lives under imminent threat than would be lost. Might there have been another Anfal? Very possibly. Would that have created a more credible casus belli? Undoubtedly. But it's fallacious to make a humanitarian case in March 2003 (as opposed to a retributive one) based upon the killings of 1991.

Also, how many of the deaths the Lancet cites are the result of American military action and how many are a result of insurgent attacks and suicide bombings?

Read the companion paper. It's not long. There are piecharts to help you out.

Robert Blendon, director of the Harvard Program on Public Opinion and Health and Social Policy, said interviewing urban dwellers chosen at random was "the best of what you can expect in a war zone."

But he said the number of deaths in the families interviewed -- 547 in the post-invasion period versus 82 in a similar period before the invasion -- was too few to extrapolate up to more than 600,000 deaths across the country.

Donald Berry, chairman of biostatistics at M. D. Anderson Cancer Center in Houston, was even more troubled by the study, which he said had "a tone of accuracy that's just inappropriate."
http://www.nytimes.com/2006/10/11/world/middleeast/11casualties.html

??????????

Blendon was badly misquoted on that one; he was talking about the precision of the point estimate. I'll try to get permission from him to post my email correspondence.

This suggests that if the figure in question is roughly accurate, the Iraqi government should certainly have statistics relating to it, correct? Or do they not gather such data?

See my challenge to Lambert on that, this is an easy way to see how ridiculous the study's conclusions are.

At this point I feel like anyone denying the deathtoll in Iraq is well over 600,000 is exactly the same thing morally as denying the holocaust. Worse even for Americans, because it is our government doing it, and that makes us partially responsible

Here we come full circle in the left-wing circle-jerk. Create a death toll that is completely absurd with a survey that doesn't fit its purpose, then call everyone who denies it akin to a Holocaust-denier.

But go ahead, take me up on my challenge, and let's see who has a problem with facing reality.

Seixon, if you had read the thread, you would have seen the arguments that even in pre-war Iraq, the bureucracy only managed to report a fracttion of _natural_ deaths up to the top.

This is not "an easy way to see how ridiculous the study's conculsions are". Have you ever worked in a bureucracy, or even in an office? Unless you have an extremely well-managed organization, the top will _not_ have an accurate picture of what the folks on the floor see. Reports aren't filed, documents disappear in the "paper mill", happens to the best. It would be nothing short of a miracle if your average district council or local authority in Iraq managed to present accurate data to the central authorities in the present situation - especially when we know that they weren't able to before the war either.

So could you stop saying that? I'm sure you have lots of other great arguments that you can use, there should be no need to use so poor ones.

It seems the Right knows a thing or two about in-depth analysis of scientific studies if the response to the Lancet paper is anything to go by. We in the reality-based community can learn a lot from this. I've summarised here. There is much wisdom. Oh so much wisdom.

"Seixon, if you had read the thread, you would have seen the arguments that even in pre-war Iraq, the bureucracy only managed to report a fracttion of natural deaths up to the top."

Harold, if you had read the thread, you would have seen that the "arguments" you refer to are factually incorrect on one hand and claim to test those (incorrect) facts against an unsourced estimate, which is itself presented misleadingly as an established fact.

I look forward, in any case, to somebody taking up Seixon on his challenge. The persistent evasions of it make me think he may have a point.

Seixon's challenge, for those who can't be bothered clicking through, is to work out from the study the number of death certificates issued post-war. I don't know why he thinks this is a conclusive refutation though, as I don't think he has the real death-certificate data or even knows if it exists.

Seixon and Josh,

You both seem to be claiming that the Iraqi central government had the ability, before and after the invasion, to accurately keep track of death rates. This is not the case has Harald has clearly pointed out. Josh, perhaps you just missed it as it's a big thread now, so here is what Harald referred to (straight from the companion paper of the Lancet study):

"Even with the death certificate system, only about one-third of deaths were captured by the government's surveillance system in the years before the current war, according to informed sources in Iraq. At a death rate of 5/1,000/year, in a population of 24 million, the government should have reported 120,000 deaths annually. In 2002, the government documented less than 40,000 from all sources. The ministry's numbers are not likely to be more complete or accurate today."

Note, they've determined this death rate of 5.5/1000 by using the same method as in the main study. This number is verified by the CIA's own estimates. It should be clear that the Iraqi government did not have the ability to determine accurate figures before the war, so why would this ability be improved after the war? So, to repeat to Seixon, will you please stop using this (discredited) argument?

"Well, that would make the estimates inefficient but why would it systematically bias the estimate upwards?"

I haven't suggested it would (reserving the right to object). The effect on the SEEs could be bothersome enough.

"In any event, the authors appear to have been aware of it--see the discussion on the left hand column of page 3 of the paper."

I read that too, Robert. That discussion just notes the garden-variety adjustments one makes to any clustered/stratified/weighted sample. Sandwich estimators, etc. We would always expect some degree of adjustment to clustered samples, because each household member shares characteristics with others in the same household. The discussion also suggests it's within-cluster temporal autocorrelation they're talking about (they cite Diggle's work on longitudinal data in the accompanying note). So I'm not sure the passage you cited addresses my question.

It seems certain people have forgotten who was in control of the Iraqi government before the war... Was it because they couldn't count all deaths accurately, or was it because they didn't want to? Why would Saddam want to publicize numbers of all the people he had killed? Doesn't sound like a good thing to do as a dictator.

The media and the Iraqi government have access to all the data on death certificates. This would be easy to track down and compile. Why, you'd almost think that it would be a much better idea than interviewing 1800 families!

But no, that wouldn't have provided the result they wanted. The Iraqi government says the study is absurd, you know, the folks who actually preside over the bureaucracy that has the records to prove how many death certificates have been given out.

That won't stop the usual suspects from claiming they know better, because of a friggin cluster survey. Sheesh. Get back to screaming about the 2004 exit polls, at least you weren't being political opportunists with the lives of Iraqis.

ekmi,

How many morgues and coroners do you think exist in Iraq? Less than 1800? Why didn't the JHU team start there instead?

I know the answer, do you?

Paul, I know what Harold referred to and I addressed it above. The passage you quote is both false and misleading.

Seixon has a valid "challenge" open, and my previous comments regarding the evasion of it keep getting reinforced.

"The media and the Iraqi government have access to all the data on death certificates."

What planet are you on? One with reliable computer systems, sternly conscientous bureucrats, frictionless administration and supreme efficiency? I can't determine the ancestry of all Norwegians, even though the old church books, recording births, marriages and deaths, are accessible to all! If the church books were all we had, even deciding what the population was in 1870 would have been a tremendous task. Now add half a century of war and endemic corruption...

Seixon,

"The media and the Iraqi government have access to all the data on death certificates."

That's as compiled by the Iraqi government. The Iraqi government which can't count birth certificates before the chaos of invasion. As noted, their account is a gross underestimation, not because of a Saddam lead coverup, as you insinuate, but because their counting system was ineffectual. Why else was the study able to uncover a pre-war death rate of 5.5/1000 deaths per annum versus the Iraqi government's figure of roughly 1.5/1000. 1.5/1000 is extremely low compared to a bunch of western countries. The only other coutries with such dubiously low death rates are, you guessed it, other countries with poor central administration and are therefore incapable of counting death certificates. The post war death rate is estimated at 13.2/1000 per annum.

It's funny actually, because you, and others, claim the death rate is high in Iraq without seemingly doing any kind of comparison with any other countries. Did you know the UK has a death rate of 10.3, and France is estimated at 9.1. Put in that perspective I'd say 13.2 doesn't seem too high, and 5.5 seems really low. That makes me doubt how effective the Iraqi's are at issuing death certificates....

Is there anyone here who can comment on the validity of those UK and French death rate figures?

Josh,

"The passage you quote is both false and misleading."

OK, if that passage is false please can you provide a link or some sort of evidence that the number of deaths in 2002 was 80,000 (excluding kurdistan), as you suggest and not 40,000 as the study suggests. I'm only going by what was in the study, so if there is some other information out there it'd be good to see it.

To follow up on myself: there's a reason that goverments and large organisations often prefer statistical sampling to comprehensive data gathering. For one thing, it's cheaper, for another, it will often give more correct results!

(There are also some, especially in government, who prefer statistics because it's easier to mislead people with them. As far as I know, US unemployment numbers are sampled, whereas in most other countries they are comprehensive. I have heard allegations that these samples are made using phone calls, thus eliminating people who don't own a phone, or couldn't afford to pay the phone bill last month. I mention this as an apropos, but I don't doubt that the Seixonians will rather "answer" this paragraph than the former!)

Dr. Steve wrote:

That discussion just notes the garden-variety adjustments one makes to any clustered/stratified/weighted sample. [...] We would always expect some degree of adjustment to clustered samples, because each household member shares characteristics with others in the same household.[...] So I'm not sure the passage you cited addresses my question

I'm not sure it doesn't. First, I agree those were garden-variety adjustments, but spatial correlation in clusters is a pretty garden-variety concern to adjust for. Second, the sampling unit is the household, so the intra-household variability isn't the issue -- I thought your original comment about spatial correlation was about the correlation between households. Nonetheless, if your concern is that spatial correlation may affect the SEE's and not the estimates, then it's helpful to clarify what that means for the readers who've never taken econometrics: you're not questioning (at least, now) the central estimate of 655K for total excess deaths but rather you're concerned that the stated CI around it is too narrow.

Paul, first things first. What is the source of the 40,000 assertion in the study? I noticed no citation for it in the study, but certainly at least a few folks here have a direct line to Les Roberts or others, and could easily pass along their source.

"Seixon has a valid "challenge" open, and my previous comments regarding the evasion of it keep getting reinforced."

That would be this challenge(?):

"If you feel the Iraqi government and I are in error about Iraqi casualties, please have a go at gathering in the 500,000+ death certificates that this study would have to claim exist in Iraq."

Or maybe the challenge is to tell poor Seixon where the elephant is hiding in the living-room.

I'll tell you what, Josh, you go to Iraq, gather up just 10,000 death certificates and a letter from Ayatollah Sistani confirming that there aren't any more.

Do you have a serious query or are you just being a pain in the arse?

The death certificate issue is one I mentioned myself in an earlier thread and there are a few possible resolutions in no particular order--

1. The survey was a fraud
2. The survey was accurate, but there is some motivation Iraqis have to fake deaths and have death certificates handy
3. The Iraqi government is incompetent and doesn't correctly add up the number of certificates it issues
4. They could do a better job adding, but they lie about it for political reasons.

We know that there are cases where they have suppressed data and there is always some incompetence. We will each have our own reasons for believing one or another of these factors.

The solution is obvious--pressure from everyone on the US and Iraqi governments to support an independent survey to determine the number of deaths that have occurred. It's outrageous that Bush and Casey quote incomplete Iraqi government reports (which are guaranteed to be too low by some unknown amount) or worse, compilations of media reports as a substitute for their clear responsibilities. I hope Josh's response in this thread is not indicative of the main thrust of Iraq Body Count's attitude when they are finished digesting the Lancet paper. Everyone should get behind what the Lancet authors have repeatedly asked for. Not that I actually expect the government to do this, but if they don't, it will be because they know there is something to hide, even if the numbers aren't really as high as 600,000.

Josh, the source of this figure seems to be Iraqi government documented figures from all sources. Presumably a data sheet exists somewhere with all the numbers compiled on it.

Have you got the source of the 80,000 number yet?

Seixon, the challenge just seems to be for you produce some credible criticism of cluster sampling under this set of conditions. I haven't seen you do this yet.

Paul asked:

the UK has a death rate of 10.3, and France is estimated at 9.1. Put in that perspective I'd say 13.2 doesn't seem too high, and 5.5 seems really low. That makes me doubt how effective the Iraqi's are at issuing death certificates....Is there anyone here who can comment on the validity of those UK and French death rate figures?

All of those death rates (for the UK, France, and Iraq) are what are known as "crude mortality rates" which means they aren't age-adjusted. Iraq has a much younger age structure than the UK or France (or the US); 5.5 isn't out of line for pre-war Iraq.

BTW, death registration, even in a modern country like the US, takes a long time to collate. I think we've just recently published the 2003 data. Not only would it not surprise me that the Iraqi government would have some difficulty knowing exactly how many death certificates had been handed out in the last couple of years, it would surprise me if they did.

Paul says: "the source of this figure seems to be Iraqi government documented figures from all sources."

Doesn't seem that way to me Paul. Back to my previous posting.

Josh,

OK then, if it doesn't seem that way to you, what do you make of this specific statement from the companion paper (extract and link shown above):

"In 2002, the government documented less than 40,000 from all sources."

Is it just your opinion that this statement is false or to be disregarded, or do you have some evidence to support that opinion/assertion? Just show the evidence and I'll take a look. If there isn't any evidence, then I'm sorry but I'm going to trust the study and not your opinion.

I thought the mortality rate in Iraqi children before the invasion was high. Which would tend to put the mortality rate up.

Do you think anyone noticed the extra grave digging activity not to mention the huge amount of land required to bury an extra 600,000+ bodies? Where did the timber come from to make all of those coffins? Do you think anyone noticed the extra bureaucrats hired to prosess the paperwork? What about the contracts for the printing of death certificates and the filing cabinets to store them in? How many extra pens would it take to fill out all of those forms? No one noticed any of this?

Paul, I've already said what I make of the statement. If you'd like to know the source of my statement I'd be happy to provide it. But as I said, first things first.

If any exists, you can acquire proof of the 40,000 assertion easily enough in such a forum as this, or you can choose to walk around believing an unsubstantiated assertion/opinion. You seem content to choose the latter. I won't lose sleep over your choice, but the former would be more consistent with someone who purports to want substantiation on the issue.

You're joking, aren't you, KJB? It's sometimes hard to tell serious criticisms from the parodies. 600,000 certificates would be 600,000 sheets of paper, or about the volume of 600 large dictionaries. Spread over various offices in Iraq. Yeah, can't imagine how anyone could hide 600 dictionaries in Iraq.

However, on a serious note, I did hear Les Roberts on Democracy Now earlier today say it would be easy for reporters to verify or refute the study. He said they should go to the gravediggers at four or five villages and ask them to compare the number of burials in 2002 with the post-invasion rates. The Lancet paper claims that death rates have quadrupled, so it shouldn't take a careful statistical analysis to observe something like this. So not all of your comment was silly.

Those of us who are American or British citizens should probably spend more of our energies writing to newspapers and politicians demanding a definitive study to determine how many people we are killing in Iraq and less time squabbling about it online. If the two governments refuse(as they almost certainly will) and if the press doesn't pressure them over it ( we're only talking about a humanitarian catastrophe possibly worse than Darfur and we're responsible), then that will tell us exactly how much we can trust either the press or the government to tell the truth on this issue.

"He said they should go to the gravediggers at four or five villages and ask them to compare the number of burials in 2002 with the post-invasion rates. "

This would certainly increase the death rate of journalists working in Iraq.

Josh,

I already asked you (three times) to provide the source for the 80,000 figure you quote, please provide it.

Does anyone posting here have access to Iraqi government death certificate records from 2002? Can anyone contact the authors to confirm this point?

Why would you doubt the professional credibility of the authors by suggesting that they failed to accurately quantify the number of deaths in 2002 ie. the numbers produced from the Iraqi government's own records? If you have a credible source that would be valid reason. Otherwise you should believe them when they say they used the Iraqi governments own records to derive this figure. Ahem, these are the records which seixon would rather trust above the author's study. You really want to continue undermining his position?

What you're suggesting is kind of like saying they (the authors) are liars isn't it? You better have some proof then.

Donald Johnson asks me if I'm joking and then suggests grave digging activity as corroboration for the Lancet figures.

The deaths would not be spread around Iraq, much of which is sparsely populated, but would have been concentrated in and around Baghdad. Now I don't know if you've ever worked for a government but it's highly likely applications for death certificates and the certificates themselves are multipage forms. If these multi-copy forms are stored in filing cabinets or lever arch files or some other conventional means of paperwork storage they're going to take up a lot more room than 600 1,000 page dictionaries. I just thouight maybe some Iraqi bureaucrat might have noticed this and, perhaps taking a cue from bean-spilling western bureaucrats, might have gotten the word out. You know, to embarass the US.

If this study is accurate, then you have to believe that approximately the same number of people have died in about the last year due to airstrikes as died during the initial invasion of Iraq. Furthermore, the number dying in airstrikes has barely fluctuated at all over the last few years.

How likely is that?

Richard wrote--

"That would certainly increase the death rate of journalists in Iraq."

Journalists do occasionally sneak around in Iraq and interview people, so it might be possible for an exceptionally brave journalist to do this. I'd be more worried about the gravedigger if his identity was revealed in the story.

Of course, if the US or Iraqi government really wanted people to know the true death rate, it wouldn't be that difficult for them to allow outside investigators to come in and replicate the Lancet study on a larger scale and protect the interviewers. But they prefer to invoke their own authority--the death rate can't be greater than 50,000 because that's what our records show and why would we lie about such a thing?

I take that last part back. I mean the percentage of deaths from airstrikes has remained roughly constant from the time of the invasion onward. You'd have to believe far more people are dying in airstrikes these days than in the initial invasion of iraq.

How likely is that?

KJB, I gave you credit for one part of your remark and criticized the part that was silly. That's what honest critics do. And you then elaborate on the silly part--the death toll can't be 600,000 because we might be talking about the volumetric equivalent of thousands of dictionaries of memos, rather than merely 600. Boy, that's convincing.

By the way, one bureaucrat in one office can't embarrass the US--it would have to be a bureaucrat who had knowledge of the total number of death certificates in all or most of the offices in Iraq and assuming such a person exists, he would then have to be willing to risk his life to expose it.

I'm agnostic on this report--the number is way above what my own intuitive guess would be. But again, it seems to me that all you critics should be demanding an outside independent investigation. In fact, all of us should be doing that.

DJ, I'm not going to discredit the Lancet study by commenting here but that's not my intent anyway. It just seems odd to me that there would be this huge increase in deaths in and around Baghdad but no-one seemed to notice. The way I figure it (allowing 2 square meters per grave) an extra 131 hectares (323 acres) of cemetary would be required. That's a lot of land.

If we are going to have more silly comments about graves and death certificates, can we please get the number right? I reckon it's about 1.1 million. Every corpse has to buried (and a certificate obtained if that is practicable). The gravedigger can't shirk the task by asking: "Is this an excess death we are dealing with here?" Even if it's a death due to old age, the digging has to be done.

You know, somehow or other, places with a crude mortality rate of 13.2 per 1,000 do manage to dispose of the bodies. I know of some rural areas where most of the population is quite elderly. Yes, they have a lot of funerals, but they get by.

Where do they get the prewar mortality figure from and how certain can we be that it is accurate?

Where do they get the prewar mortality figure from and how certain can we be that it is accurate?

Mike, please read the study.

KJB,

Out of 1,849 households surveyed, containing 12,801 people, there were 1,474 births and 629 deaths. So what's all this fuss about death certificates? Where are all the birth certificates? If you think the gravediggers are overworked, how the hell do you think the midwives feel?

Paul, I hope someone here provides you with the source info for their 40,000 claim. It should be easy for any number of people here, including the host, to acquire it directly from the authors who made the claim. But my guess is that they will chose not to provide it to you.

Donald Johnson: "if the press doesn't pressure them over it ( we're only talking about a humanitarian catastrophe possibly worse than Darfur and we're responsible), then that will tell us exactly how much we can trust either the press or the government to tell the truth on this issue."

I think there has been more than ample time (several years) for the American press to exert pressure on the US government on the subject of civilian deaths in Iraq.

The newspapers of America have demonstrated beyond any reasonable doubt over the past few years that they are not worth the paper that they are printed on.

How can anyone trust American newspapers (and media in general) when they got things so wrong on pre-war WMD?

And as far as trusting the US government on this kind of stuff, they don't have a partiicularly good track record (Vietnam), which is the very reason why they hav adopted the official policy for this war that they "don't do bodycounts".

Kevin Donoghue,

I don't have access to the study as I've indicated earlier and I won't for some time.

Josh,

"But my guess is that they will chose not to provide it to you."

In the same that you've completely dodged my request for you to provide any evidence to support your claim that 80,000 people died in Iraq (excluding Kurdistan) during 2002? I've asked you four times and you've not provided it. I'll assume the evidence doen't exist then.

Paul,

I think Josh has in mind a comment of Les Roberts' in a response to IBC during the flap they had:

"According to my colleague Riyadh Lafta (cc:ed above), only about 1/3rd of deaths were captured by the Government's surveillance system in the years before the Coalition invasion (that was based on a conversation with Riyadh and the assumption that there were 24,000,000 people in Iraq dying at a rate of 5/1000/year, or that 120,000 people were dying per year. In 2002, the Gov. of Iraq documented less than 40,000 from all sources)."

If Josh has a point he will make it. If he goes on being coy, he hasn't.

I may be misreading this study, so if someone could help me understand this correctly. It appears that if the figures for this study are accurate, over 60,000 people have been killed in airstrikes during this entire period. Half of those people have been killed in just the last yearly period. So over 30,000 people died in airstrikes during this last period?

I agree those were garden-variety adjustments, but spatial correlation in clusters is a pretty garden-variety concern to adjust for.

Sure. Now we just need to see if they actually did it. Citing Diggle (which is on temporal rather than spatial correlation) doesn't assure me on that point.

Second, the sampling unit is the household, so the intra-household variability isn't the issue -- I thought your original comment about spatial correlation was about the correlation between households.

Yes, I was talking about correlation between contiguous households. I mentioned within-household correlation because that's another design element that would commonly require adjustment depending on the design, though it's not necessary for this design. Sorry for the confusion.

I'm going to ask Dr. Burnham for his data, protocols, instruments, dispo records, Stata .do files and .log files by the end of the week. Right now I'm on a deadline of my own.

KJB: The deaths would not be spread around Iraq, much of which is sparsely populated, but would have been concentrated in and around Baghdad.

If you look at the amp showing the ditribution of deaths, you'll see that death rates across the Sunni-majority areas north and west of Baghdad are as high as, or higher than, those in Baghdad.

This post gives the estimated provincial populations used in the study:

http://scienceblogs.com/authority/2006/10/the_iraq_study_-_how_good_is_…

The Sunni-majority provinces of Ninwa, SalahDin, Diyala and Anbar have a combined population comparable to that of Baghdad.

I don't have access to the study as I've indicated earlier and I won't for some time.

It's online. You can guess the name of the site. It's eight pages in length.

Donald Johnson,

"However, on a serious note, I did hear Les Roberts on Democracy Now earlier today say it would be easy for reporters to verify or refute the study. He said they should go to the gravediggers at four or five villages and ask them to compare the number of burials in 2002 with the post-invasion rates. The Lancet paper claims that death rates have quadrupled, so it shouldn't take a careful statistical analysis to observe something like this. So not all of your comment was silly."

So why didn't Roberts and the gang just do this themselves?? Why didn't they go to morgues and such to ask how many death certificates they had records of?

Their answer in the study is, summarized, is "Bush is hiding it, therefore screw it". That's "science" for you alright.

ahem,

Thank you! Excellent!

Ok. Apparently the source for the assertion in the Lancet report of the Iraqi gov only recording 1/3 of deaths in 2002 is "a conversation with Riyadh" combined with an "assumption".

So at some point one author of this report said it to another. This still leaves us none the wiser as to the origin of these figures I'm afraid.

Point taken, IG. I should have been more specific -- much of southern Iraq, the Kurdish north and sparsely populated areas would have experienced lower levels of violence. I'll now leave you guys to duke it out.

I don't think he had the usual morgue bureaucrats in mind--he probably meant local guys in villages. But I'm not sure. Anyway, Roberts and the Lancet people have been trying for two years to get other people interested in trying to replicate or refute their results. Without much success. (The ICLS survey in 2004 is the closest, and their violent death toll where comparable was only about 30 percent lower than the Lancet estimate.)

Again, why is this entirely about the credibility of the Lancet people? Suppose they're all liars. Shouldn't we all be demanding that the governments responsible support an independent investigation?

If I'm reading the data correctly (and that is always a big "if," I'm no statistician, so I humbly ask the more statistically educated to correct me if I'm drawing incorrect conclusions), according to the Lancet study, in the post-invasion period, men were approximately ten times more likely to be killed in air strikes than women.

I can only presume such an exceedingly high ratio comes from hitting insurgents, as opposed to civilians, very accurately. Even when Zarqawi was killed, his teenage wife was among the victims. In fact, in that particular air strike, three out of six of the deaths were females. If they are hitting insurgents with such accuracy, I suppose they've killed tens of thousands of insurgents with airstrikes alone in just the last yearly period. Furthermore, they must have killed even more than 30,000 total individuals in that period when you weigh in the fact that any foreign fighters would be presumably unaccounted for in this study. Remarkable!

Most American journalists actually seem to revel in the Iraq war and only take an interest in the dead when their colleagues are the ones dying.

It just seems odd to me that there would be this huge increase in deaths in and around Baghdad but no-one seemed to notice.

Do you suppose it is just possible that Iraqis have noticed, and that this is a major part of the reason they overwhelmingly want the US out of their country really fast?

Anyone lying to cover up genocide is guilty of collaboration.

Anyone lying to cover up genocide is guilty of collaboration.

ThinkTank, you wouldn't happen to be an Iraqi insurgent, would you?

Josh,

I've got a pair of citations taken directly from the study. In the study it refers to 18 and 19 here:

"Our estimate of the pre-invasion crude or all-cause mortality rate is in close agreement with other sources.18,19 "

This is 18. CIA 2003 Factbook entry for Iraq
http://permanent.access.gpo.gov/lps35389/2003/iz.html
(accessed Oct 2, 2006).

19 doesn't have a weblink but here is the reference:

"19. US Agency for International Health and US Census Bureau. Global population profile: 2002. Washington, DC: US Census Bureau, 2004:."

Both of these studies corroborate the estimates of the Lancet study, and show that the pre-war death rate was roughly 5.5/1000, thus about 120,000 died in 2002.

The companion paper (cited in the first comment above) clearly explains on pages 16 and 17 that the Iraq Ministry produced data showing less than 40,000 people died in 2002.

With no evidence to the contrary how can you still claim that the Iraq Ministry recorded 80,000 deaths instead of less than 40,000?

This affirms the fact that the Iraqi Ministry is no reliable current judge of the death rate within its own country.

"The companion paper (cited in the first comment above) clearly explains..."

"clearly explains" = asserts without citation or substantiation, and so it remains.

And Paul says the unsubstantiated assertion "affirms" another unsubstantiated assertion.

The origin of the assertion is still a mystery.

And I'll leave for others to decide how compelling an unexamined estimate of mortality in Iraq offered by the CIA is. Maybe accurate, maybe not, but that bit is irrelevant unless the unsubstantiated 40,000 assertion is true. So, let's get to answering the mystery of its origin.

Michael Moore has 655,000 chins. So there.

when person dies at, or body is brought in to any sort of health facility, a death certificate is issued. One copy goes to the family; which they keep for many reasons (benefits, burial, social) and another copy is supposed to work its way up the chain to MoH. The first happens to a nearyly 100% & system is not broken; it is local. The latter not so much... dependent on intact infrastucutre. that is why families have certificate and central MoH does not.

Other estimates based on reported bodies from newspaper accounts have always been ridculously low... what % of death nationwide make a newspaper 5%? 10%? Other than knowing it is ridiculously low, how low is unmeasurable. Such counts are profoundly biased to be low and incomplete and how incomplete is unmeasurable.

As Juan Cole points out, the 500 per day nationwide in big country is quite reasonable, and amounts to 4-5 per day per town/city.

There are some legit questions about selection bias in choosing clusters and households missed; missing people who have left, etc. But...

Bottom line: the lancet report is the best statistically valid estimate we have.

I found a series of calculation errors in the table at the bottom of page 4. Lousy proofreading or poor writing. They do cast a small shadow on the rest of the numbers.

The statement that there were virtually no violent deaths in Iraq in the fiftenn months prior to the Liberation is laughable. Iraq had a criminal murder rate approaching zero?

Those still sympathetic to Seixon's "argument from incredulity" might best read and learn from an informed source.

If 470 were dying every day, what would that look like?

West Baghdad is roughly 10% of the Iraqi population. It is certainly generating 47 dead a day. Same for Sadr City, same proportions. So to argue against the study you have to assume that Baquba, Hilla, Kirkuk, Kut, Amara, Samarra, etc., are not producing deaths at the same rate as the two halves of Baghad. But it is perfectly plausible that rough places like Kut and Amara, with their displaced Marsh Arab populations, are keeping up their end. Four dead a day in Kut or Amara at the hands of militiamen or politicized tribesmen? Is that really hard to believe? Have you been reading this column the last three years?
...

Steve Sailer has some interesting analysis:

http://isteve.blogspot.com/2006/10/updated-depressing-news-of-day.html

Saddam must be really jealous now...

And the winner for losing reputation the quickest goes to:

"Steve Sailer has some interesting analysis"

Rob,

And the winner for losing reputation the quickest goes to

Wow, Rob! What an insightful and informative ad hominem attack. I guess you're the winner for "least clever rebuttal" or perhaps just the "least relevant statement overall."

Okay, I'll bite.

Do tell us more about the calculation errors you've found in Table 4, Chuck. I'm dying to hear them.

It still doesn't pass the smell test, even after sifting through the methodology in detail. Granted, I'm not a statistician, but it doesn't take a detailed analysis to see the very blatant holes in the study. For one, the UN couldn't reproduce the 2004 study and found a casualty rate that was significantly less than the 100,000 figure commonly cited.

For instance, when you have a statement like Baghdad has between "2-10 deaths per thousand per year" you're not stating anything. There isn't a sufficient level of precision to draw any reasonable inferences from that data set. Furthermore, if 80% of respondents could produce certificates of death, why is there such a massive discrepancy between the study and other records?

And how in the world can one rationally argue that Iraq has suffered a higher level of per-capita casualties than Germany did in the Second World War, when the Germans were being saturation bombed, occupied by forces that had little compunction against summary executions, and then suffered the complete collapse of nearly all their infrastructure? Again, I wouldn't even think of making that argument in a peer-reviewed publication.

Not only that, but if one wanted to seriously present a substantive and scientific finding, presenting it during the October before an American election and filling it with highly partial political statements strikes me as one hell of a breach of basic scientific ethics. Why would one suffer the inevitable appearance of partiality rather than wait a few weeks?

No matter how many attempts at rationalization are made, it doesn't change the facts that this was a politically motivated study whose conclusions are not borne out by the evidence presented and whose findings cannot be scientifically reproduced by more rigorous and impartial groups.

A total of 15 households of 1849 refused to participate.

That's a refusal rate of 0.8%. In 25 years of market and social research, I've never seen a survey with such a low refusal rate. And if I did, I'd be conducting an audit of the interviewers' logs.

Caveat - I haven't read the report and am relying on this quote in the above thread.

James, have you done any surveying in Iraq? Do you think that in Arab culture it might be rude to refuse to participate?

Tim Lambert,

I think you've got the completely wrong idea about Arab culture if you think Arabs are typically open to discussing personal matters like family deaths with relative strangers. Quite the opposite, Arabs are typically extremely polite with strangers so long as the subject matter is of a superficial nature, but don't readily get into discussing personal issues like deaths and other family matters with outsiders. Even to inquire into such matters, unless brought up by the family of the deceased beforehand, would often be considered quite rude. Maybe the researchers were able to coax people into giving them that information anyway, but that is an extraordinarily low rate.

Hi Tim,

No I haven't done a survey in Iraq, or any Arab country for that matter, although I know people who have. It is fair to point out that cultural differences might explain differences in refusal rates, although I wouldn't have thought conducting a survey in areas prone violent death would be a factor that enouraged participation.

Leaving the above aside, people refuse to participate in surveys for a whole host of quite understandable reasons which would not appear to be culturally dependant, for example, they want to go to the toilet, they're playing cards, they are preparing for sexual activity, they are engaged in criminal activity, they are about to go out, they suffer from a mental illness, they're physically ill or maybe they are otherwise just busy.

To be honest, I consider a refusal rate of 0.8% (if that's really what it was) to be incredible.

Another result that raises my eyebrow is that 90% plus were able to produce a death certicficate on request. That strikes me as amazingly high. On this point I'm more willing to concede cultural practice (e.g. if they keep them under their pillows or some such).

A poster over at Crooked Timber claimed that the two fieldwork teams each averaged about 25 interviews per day (again I can't confirm these numbers). That also strikes me as hardly believable, even with the negligible refusal rate.

BTW, there is nothing wrong with cluster sampling per se. Almost all "door-to-door" fieldwork uses this method, and it's probably the only way to conduct a survey in a place like Iraq.

I don't know what to make about the 2006 Lancet report. Could someone post a link to the refusal rate and detailed method?

Right after mentioning the refusal rate on page 4, the Lancet article says "One team could typically complete a cluster of
40 households in 1 day."

James asked:

Could someone post a link to the refusal rate and detailed method?

Page 4 of the report. Also, it says that one team could do all 40 households in a cluster in one day. On page 2, it says that one team consisted of 4 interviewers.

Sorry I can't find anything about refusal rates or field "hit rates" on page 2 or page 4 of the report (or anywhere in the report or the appendix)?

The more I think about the mechanics of carrying out the survey on the street without getting killed, the more I suspect that the Iraqi interviewers didn't actually implement the purely random survey design that the American professors from MIT and Johns Hopkins dreamed up for them. It would be nuts to to let luck determine which streets you'd choose, as the report claims they did. You'd want to only go where you knew you'd be safe. Then you'd tell the Americans you did exactly what they told you to do. What are they going to do? Leave the Green Zone and check up on you? Yeah, right ...

It could be that the interviewers got in contact ahead of time with neighborhood leaders to see if their presence would be welcome to reduce their chances of being killed. (That's not good random surveying hygiene, but are you going to blame them?) Then, in a neighborhood where the local big shot wanted their presence, he might have passed the word around to aggrieved families to get ready to tell their stories to the interviewers when they showed up.

This could cause a bias upward in the number of deaths reported. Or, it's easy to imagine scenarios where departures from randomness would bias the results down.

The overall point, however, is that nobody else appears to be doing this kind of study because it is so hideously dangerous, which ought to tell us something.

More analysis is necessary, but, after a few hours of kicking the tires, these numbers don't strike me as obviously implausible. I wouldn't put tremendous confidence in them either, though, due to the savage conditions under which this heroic effort was carried out.

James wrote:

Sorry I can't find anything about refusal rates or field "hit rates" on page 2 or page 4 of the report (or anywhere in the report or the appendix)?

"Appendix?" Oh, you're looking at the report from the MIT website. The page numbers I was referring to were from the Lancet article, which doesn't have an appendix.

Keep yabbering Tim, you'll eventually convince yourself. Smells like bullshit, looks like bullshit and gawd, look at my shoe, it is bullshit.

well, having consulted your shoe, what's the real number, slatts?

Paul wrote: "The companion paper (cited in the first comment above) clearly explains on pages 16 and 17 that the Iraq Ministry produced data showing less than 40,000 people died in 2002."

For those who had been demanding a source for my statements above which called the 40,000 figure false (while happily accepting the 40,000 figure despite its source being unknown), you may look here:

"the figure of 40,000 claimed as the number of deaths recorded by the MoH in 2002 is false. No specific citation is offered by the Lancet authors for this figure other than a vague attribution to "informed sources in Iraq". But official Iraqi figures for 2002, forwarded to IBC courtesy of the Los Angeles Times, show that the Ministry registered 84,025 deaths from all causes in that year. This excluded deaths in the Kurdish-administered regions, which contain 12% or more of the population."
http://www.iraqbodycount.net/press/pr14/4.php

I'd previously been asking for the origin of the 40,000 figure. Assuming it wasn't just made up, I'd still be interested to know.