The BBC used a Freedom of Information Request to find out what the scientific advice to the British government about the Lancet study was:
The British government was advised against publicly criticising a report estimating that 655,000 Iraqis had died due to the war, the BBC has learnt.
Iraqi Health Ministry figures put the toll at less than 10% of the total in the survey, published in the Lancet.
But the Ministry of Defence's chief scientific adviser said the survey's methods were "close to best practice" and the study design was "robust".
Another statistician agreed the method was "tried and tested".
The Iraq government asks the country's hospitals to report the number of victims of terrorism or military action.
Critics say the system was not started until well after the invasion and requires over-pressed hospital staff not only to report daily, but also to distinguish between victims of terrorism and of crime. ...
Shortly after the publication of the survey in October last year Tony Blair's official spokesperson said the Lancet's figure was not anywhere near accurate.
He said the survey had used an extrapolation technique, from a relatively small sample from an area of Iraq that was not representative of the country as a whole.
President Bush said: "I don't consider it a credible report."
One of the documents just released by the Foreign Office is an e-mail in which an official asks about the Lancet report: "Are we really sure the report is likely to be right? That is certainly what the brief implies."
The reply from another official is: "We do not accept the figures quoted in the Lancet survey as accurate. " In the same e-mail the official later writes: "However, the survey methodology used here cannot be rubbished, it is a tried and tested way of measuring mortality in conflict zones."
Asked how the government can accept the Lancet's methodology but reject its findings, the government has issued a written statement in which it said: "The methodology has been used in other conflict situations, notably the Democratic republic of Congo.
"However, the Lancet figures are much higher than statistics from other sources, which only goes to show how estimates can vary enormously according to the method of collection. There is considerable debate amongst the scientific community over the accuracy of the figures."
In fact some of the British government criticism of the Lancet report post-dated the chief scientific adviser's report.
Speaking six days after the CSA had approved the study's methods, British foreign office minister Lord Triesman said: "the way in which data are extrapolated from samples to a general outcome is a matter of deep concern...."
It would seem that the thing that deeply concerned Triesman was that his scientific advice was that the methodology "cannot be rubbished".
Hat tip: Mark Harrison.
Craig Murray has an interesting angle on this. It seems that an earlier version of the story (now buried) included a DFID opinion that 655,000 is likely, if anything, to be an underestimate. It would be interesting to know how the DFID arrived at that conclusion. On the face of it, that's the way you would expect most biases to work: interviewers are more likely to shy away from high-risk areas if they can sense where they are.
http://www.craigmurray.co.uk/archives/2007/03/lying_abouth_th.html
That aside, the interesting thing is how officials on both sides of the pond reacted in much the same way - recall that USAid's experts were also asked to rubbish the study and reacted much like their UK counterparts. All in all it's reassuring.
Lambert wrote: "It would seem that the thing that deeply concerned Triesman was that his scientific advice was that the methodology 'cannot be rubbished'."
Wrong. Most purveyors of nonsense will rubbish a study regardless of the facts. For further references, see also Climate Audit, Stephen McIntyre, Roger Pielke Jr., and Pat Michaels.
>>Critics say the system was not started until well after the invasion and requires over-pressed hospital staff not only to report daily, but also to distinguish between victims of terrorism and of crime. ...
This is not terribly convincing but it is the only thing approaching a substantive critique of the Iraqi official figures. Asking hospital staff to make a report daily is not that onerous, and I have no clue why your average household member would be more likely to accurately estimate whether a shooting was due to terrorism or other crime.
>>"However, the survey methodology used here cannot be rubbished, it is a tried and tested way of measuring mortality in conflict zones."
Les Roberts on the proven accuracy of cluster sampling:
"LR: That is a good question. There is a little evidence of which I am aware."
"Asking hospital staff to make a report daily is not that onerous"--
Hmm, not being an Iraqi hospital staffer I'm reluctant to say what might or might not be onerous. I've read the circumstances there aren't exactly like those that prevail in the Mayo Clinic, what with soldiers or terrorists breaking in to arrest or kill people from time to time, where family members are sometimes afraid to come for fear of being killed (or maybe that was morgues). But no doubt writing up the daily reports is no big deal--obviously something you would do faithfully and accurately every single day.
Given that the Minister for Health made it perfectly clear last November that even he didn't believe the official figures, anyone going to the bother of writing a critique of said figures is either very poorly informed or very badly stuck for something to do.
"Asking hospital staff to make a report daily is not that onerous, and I have no clue why your average household member would be more likely to accurately estimate whether a shooting was due to terrorism or other crime."
Yeah, if one of my relatives was an insurgent and got wounded, I'd be sure to tell the hospital staff that.
Re: "I have no clue why your average household member would be more likely to accurately estimate whether a shooting was due to terrorism or other crime."
Correct me if I'm wrong, but as I understand it the study reported in the Lancet *didn't* ask household members to estimate whether deaths were due to terrorism or other crime. They just estimated total deaths and found that there are a hell of a lot more people dying in Iraq these days than there used to be. Speculating as to why this might be was left as an exercise for the reader.
Did any US agencies try to rubbish the report? Please send links, because it's obvious that Bush would do the same.
Thom, USAid's experts were asked to pick holes in the study but came up with nothing:
http://scienceblogs.com/deltoid/2007/02/gilbert_burnham_seminar_on_lan…
Hi, lurker delurking.
This story got a slot on the BBC's PM programme ( http://www.bbc.co.uk/radio4/news/pm/ ), which also mentioned the DFID official's opinion. The impression given was that the reason DFID think it's an underestimate is because the researchers insisted on seeing a death certificate, and they thought not every family would be able to comply.
"On the face of it, that's the way you would expect most biases to work: interviewers are more likely to shy away from high-risk areas if they can sense where they are."
Posted by: Kevin Donoghue
None of the critiques of the study that I've seen has cared to dwell on the fact that empty houses were skipped over. IMHO, the death rate among former members of a household which left home would be quite high.
Barry, this would apply even more strongly where entire households have been killed.
Oh dear, The Lancet's defender has form!
"Two years ago it was Roy Anderson who created the computer model used by the government to claim that the number of foot-and-mouth cases would fall to zero by 7 June 2001. As the Eye noted at the time, a certain amount of statistical jiggery-pokery was required to achieve this desirable if implausible result, but it allowed the Dear Leader to call an election for that very date and boast that he had the crisis licked.
Back in 1987 Anderson's mathematical talents again proved useful to a politician's election prospects. He was invited by Norway's prime minister Gro Harlem Brundtland to help produce an "independent assessment" of how many minke whales Norwegians could sustainably kill each year. The International Whaling Commission had introduced a moratorium on commercial minke whaling, which Brundtland feared would lose her support in the northern whaling constituencies. She hoped that a report by a four-man committee of experts, including Anderson, would persuade the IWC to ease the ban.
Lo and behold, the committee came up with exactly the same figure - 200 whales - which the whalers thought they needed to make a profit. But then a mathematical biologist on the IWC's scientific committee went through the algebra and found "fundamental flaws in the methodology": Anderson and his chums had achieved the result Brundtland wanted only by creating unreal (indeed "impossible") parameters."---Private Eye, May 3, 2003
Private Eye, May 3, 2003
Just found this.
http://www.businessweek.com/bwdaily/dnflash/feb2003/nf2003026_0167_db05…
During the first Gulf War, a sociologist, Beth Osborne Daponte, in the Census Bureau did a report on Iraqi casualities. Her numbers contradicted the number put out by then Defense Secretary Dick Cheney.
The government rubbished her report and then tried to fire her. She fought back and is now at Carnegie Mellon.
Donald: Seriously, writing down "X died from gunshot wound" Y times per day is the deal breaker for you? Far too onerous for Iraqi hospital staffers?
Ian: The Lancet study explicitly avoided asking such questions for that reason, so there is no reason at all for the criticism levied against the official figures to be valid.
On cluster sampling:
"The problem of this sampling is that inferential statistics are not appropriate for analyzing data resulting from a study using cluster sampling. Inferential statistics generally require random sampling."
Isn't the judgement of excess deaths in Iraq an inference?
Further:
http://www.opinionjournal.com/editorial/feature.html?id=110009108
The Lancet study didn't record any demographic information necessary to validate the general applicability of their results. Also the op-ed's complaint of a low number of clusters being responsible for an inaccurate response is borne out here:
http://www.ipsos-mori.com/publications/aiz/cluster-sampling-a-false-eco…
and here:
http://www.who.int/vaccines-documents/DocsPDF01/www592.pdf
The last is notably from the WHO and notes that cluster sampling generally tries for heterogenous clusters, such as are not likely to be found in canvassing neighborhoods. According to the WHO report, polling homogenous clusters is again more likely to produce inaccurate results.
Maybe I'm missing something, but what's this talk of hospital workers. In a war torn country with a collapsing infrastructure, is it reasonable to assume that deaths recorded in hospitals are reflective of the total number of deaths? In order words, do most people who get shot make it to a hospital? And if not, is this compensated for in the official number?
"Donald: Seriously, writing down "X died from gunshot wound" Y times per day is the deal breaker for you? Far too onerous for Iraqi hospital staffers?"
Yep. If the hospital is in shambles, gunmen sometimes enter and shoot patients or staff and so on, it wouldn't surprise me if the bureaucratic niceties get lost in the shuffle. Not to mention the reported fact that there is sometimes political pressure to push casualty figures up or down.
Plus I've read a little about statistics in other wars. You can't always trust them.
Kevin, [they did collect demographic data](http://scienceblogs.com/deltoid/2006/10/les_roberts_responds_to_steven…).
And yes, cluster sampling leads to a larger confidence interval than you would get with simple sampling. But that's already included in the confidence interval they reported.
In one regard -- and really the critical one -- the Lancet results are irrelevant.
The Lancet report was clearly intended for an American audience, but what we Americans think is basically irrelevant at this point.
It is what the Iraqis think that makes a difference and that will determine the ultimate outcome in their country.
"according to an authoritative poll sponsored jointly by ABC, BBC and USA Today: Only 38 percent of Iraqis believe that the country is better off today than under Hussein, while nearly four out of five oppose the presence of coalition forces in Iraq." -- From Saddam has the last laugh, by Robert Sheer
http://alternet.org/columnists/story/49589/
People can debate the Lancet results from now until they turn blue in the face and it would amount to nothing because the Iraqis have already made up their mind that things are very bad in their country (worse than under Saddam) and getting worse by the day.
It makes no difference what the rest of us think. We are fooling ourselves if we believe it does.
Well for one thing the folk in the morgue have only been reporting the unclaimed bodies, which means anyone taken away to be buried does not appear in the count. Pay attention
So then truth is merely a function of politics. Jean Baudrillard was right it seems. For some of us, even death is merely simulacra. How about this?: the Lancet results are revelant because 650,000 actual people are actually dead because of the war.
Kevin wrote:
First, statistical inference refers to distributional aspects of an estimate, not the estimate itself. Cluster sampling produces statistically consistent (i.e., asymptotically unbiased) estimators. Statisticians are quite familiar with the properties of cluster sampling estimators and their strengths and weaknesses. This is not cutting-edge stuff. Pick up any sampling book and look through the table of contents for cluster sampling and you'll see that everyone discusses the proper way to do inferential statistics for cluster samples.
Second, the Burnham and Roberts studies did, in fact, collect demographic information, and Moore admitted (right here on Deltoid) that he is not qualified as a statistician. His claim in the WSJ article was that no one uses 47 cluster samples, but that is contradicted by the WHO report you yourself cite: the standard WHO technique for immunization purposes used 30 clusters.
Third, speaking of that WHO document, you've misquoted it. It doesn't say that homogeneous clusters gives "inaccurate" results; it says homogeneous clusters give less precise results. This is related to both of the points above: you've confused inaccuracy with imprecision.
it is a rather large confidence interval and i'd assume on a study as important as this it would be well over 95% CI then at the very least by a standard hypothesist test we can reject the null hypothesis that the official figures are correct.
supporters of the war seem very touchy about this report and it's as if they can't beleive that america has caused all this carnage. no where in the report does it say that x amount of people were killed in the war or by americans. it simply states that based on the death rate of years previous to the war 300 000 - 900 000 more people have died then would be expected from the previous rate.
with all the violence that continues to this day as well as lack of medical services and just simple things like sewerage and nutrition at levels much further below what was the case of course the mortality rate due to the decision to go to war is sky high.
and jb i don't think that the results are irrelevant if it leads people in democracies to not allow their governments to instigate this sort of carnage and tradgedy again.
Mark,
Don't get me wrong. It's not that I don't think 650,000 dead (or even 1000 dead) means nothing. I certainly think it means a great deal. That's why I phrased my statement as I did
"In one regard -- and really the critical one -- the Lancet results are irrelevant."
Perhaps if I had included two more words "...the critical one for Iraqis"
Note that I did not say that "the Lancet results are irrelevant in all regards." I certainly don't belive that. It's just that I would consider what is going on right now to be the critical thing and the Iraqis have already spoken in that regard.They want us the hell out.
The Iraqis already know that things are atrocious in their country. They don't need anyone to tell them that. Lancet was intended to convince the rest of us (many of whom will never be convinced, no matter how many facts and figures you shower them with).
and notallright:
If the results of Lancet did "lead people in democracies to not allow their governments to instigate this sort of carnage and tradgedy again", I'd agree it was worth debating, but based on what has happend in the past (Vietnam) I'm not at all hopeful in the regard.
That requires not only that people accept the results but that they learn from them. If Iraq is not repeated, I am quite certain that the primary reason will not be that tens or hundreds of thousands of Iraqis died, it will be because of how costly the whole thing has been (in lives and money) to America. That's very sad, I agree, but it's also true.
JB and Mark,
Very thoughtful posts. I agree with both of you - when super poweful and super violent states are intent on military and economic expansion, little will get in their way. Public opinion is a nuisance, hence why the Iraq invasion and occupation have required a propaganda campaign 'So mendacious that it is unparalleled in peacetime democracies' according to analyst Anatol Lieven.
How does the political and corporate establisment get away with mass murder? The essay by Edward Herman spells it out clearly (below). There are 'worthy' and 'unworthy' genocides, and the United States has granted itself (along with some of its client states and proxies) 'aggression rights'. When wars of aggression go badly wrong, then the victim is usually blamed, as the second essay by Paul Street (also below) makes clear.
The real problem for many of the US-UK critics of the Lancet studies has little to do with the methods of the studies but more to do with blind faith in the 'inherent goodness' of the west, of our 'basic benevolence', which is of course shattered if one examines the empirical evidence. This evidence, therefore, must be marginalized, or dismissed entirely as fiction, and the mainstream media outlets and 'respected jouralists' do this wth gusto. The myth of our moral high ground is a foundation of the way in which we are programmed to see ourselves from the cradle to the grave, and any notion that it 'ain't necessarily so' just does not register in the minds of many of our citizens. Many just cannot fathom that our governments wage wars and commit mass murder in support of policies that are mostly aimed at benefiting the priviledged few.
http://www.zmag.org/content/showarticle.cfm?SectionID=80&ItemID=12404
http://www.zmag.org/content/showarticle.cfm?SectionID=72&ItemID=12432
Notallright: "with all the violence that continues to this day as well as lack of medical services and just simple things like sewerage and nutrition at levels much further below what was the case of course the mortality rate due to the decision to go to war is sky high."
The one aspect of the Lancet study which gives me pause is the claim that the great bulk of the excess mortality is due to violence rather than to factors such as public health and sanitation.
It may be that the end of sanctions and the improvement in living standards in the Kurdish regions have largely offset the impact of these factors in the rest of the country.
Or it may be that either the interviewers or the interviewers are causing some nonviolent deaths to be reported as violent.
The checking of death certificates makes it seem improbable that the total death rate is significantly misstated.
Tim: Whoops, I see I did uncritically accept that claim about the lack of demographic data.
Eli: If the possession of a death certificate is validation of a claimed death in a household, why is gathering death certificate information from all the local authorities issuing them not going to be far more accurate and precise than cluster polling?
Robert: Your claim about WHO's polling practice is founded in the argument from tradition, where Moore's was founded in the argument from popularity and from his personal authority. However, the paper I cited just above the WHO poll cast doubt on the entire practice of cluster sampling.
Further, Roberts claimed to know of little objective verification of the accuracy of cluster sampling. I should have written about imprecision rather than inaccuracy in reference to the WHO paper. But that the statistical community is well aware of the imprecision of cluster sampling and that Roberts is aware of little proof for its accuracy is hardly a defense of it.
Out of curiousity, has anyone actually analyzed the demographic data from the Lancet study to verify the poll was representative?
Also, didn't the interviewer still contradict Roberts and Burnham about contacting locals to find clusters away from town center?
Didn't the Lancet group's account of how many people were doing the interviews at a time still change several times? Clarified from quartets to pairs, in addressing Hicks. Changed from pairs to singles in addressing Nature. Upon contradiction by an interviewer, changed from singles to "you misunderstood, I was talking about 2004 and not 2006," addressing Nature again and claiming they did not go singly. And from "you misunderstood..." to "sometimes pairs, sometimes singles," from Burnham.
Didn't Roberts still make a seemingly inconsistent claim about there having been 2 hours in the field per household?
Why isn't the lack of familiarity with, changing account of the study by, and rhetorical bent of its authors a reason to question the objectivity of the poll or at least its authors?
Isn't cluster sampling, a multi-stage sampling, the first stage being planning to ensure a relevant sample? If the methodology was important, and it wasn't followed, as implied by the interviewer Nature spoke to, why doesn't this have negative implications for the accuracy of the poll?
If the reason to include male and female interviewers on teams was to prevent sexual biasing in the response, why isn't interviewing singly an issue for the accuracy of the poll?
My acquaintance with statistics is remote. Why is finding a result that 1) far exceeds other estimates 2) in a study not well suited for the type of work to be done 3) performed in a manner inconsistent with its stated methods 4) from an author who readily admits his political bias and preference for haste in publishing and 5) is generally unaware of the accuracy of his method and seemingly of many particulars of the study, not likely to indicate either the sample of this study was not normally distributed or the polling was not well suited to produce accurate results?
Kevin wrote:
Sigh. It doesn't at all "cast doubt on the entire practice of cluster sampling." It's a discussion of the trade-off between the number of clusters sampled and the number of observations per cluster, i.e., the design effect. In fact, the paper is about the robustness of cluster sampling.
And Hicks is a hack.
"Why is finding a result that 1) far exceeds other estimates"
The other estimates are based on official sources, hospital records and so forth. I agree it's a little hard to believe these are off by a factor of ten, but maybe they are.
And the recent ABC/USA Today/BBC poll found 1 household out of 6 had suffered some sort of casualty, which suggests a toll a lot higher than the official numbers.
Donald was "casualty" defined as "fatal injury"? The term can mean simply injury.
I seem to recall a recent news piece using a figure of 6 for the average Iraqi household. Given a population in the 24-27 million range, that would imply 3-3.5 million households.
So 1 in 6 households suffering a casualty would imply around 5-600,000 casualties. If those are fatalities, it accords frightingly well with the mid-range of the Lancet figure.
According to the recent USA Today poll (http://www.usatoday.com/news/world/iraq/2007-03-18-poll-cover_N.htm), 17 percent of Iraqis in the sample say they know someone in their household that has been physicaly harmed by war-related violence.
If we assume that each person polled represents a different household, this means 17 percent of Iraqi households have someone who has been injured or killed by violence in the war.
The average Iraqi household had 7.7 members in 2002. That means that the Iraqi population of 26 million has 3.38 million households. If 17 percent of these households have had at least one person wounded or killed by war related violence, that implies at least 575,000 injuries or deaths.
Within this number, there may be some very minor injuries included and some double counting, either because more than one respondent in the same household was surveyed or because the person answering the survey interpreted "household" to mean his/her extended family. So, we could adjust the numbers downward by, say, 20 percent, and get a minimum estimate of 459,000 implied injuries or deaths. On the other hand, there are likely to be instances in which more than one member of the same household was hurt, and the poll doesn't pick this up, so the number could be higher than 575,000. If we add 20 percent to get a maximum, that creates an implied range of 459,000 to 690,000 injuries or deaths.
The Iraq Body Count (IBC) data set typically lists about 3 injuries for every death. Even if one thinks IBC is a systematic undercount, there is no reason to assume that its ratios of injuries to deaths is off. So, if this ratio holds, one fourth of the implied injuries or deaths would be deaths. This suggests that the USA Today poll implies 115,000 to 173,000 war-related deaths from violence for the first 4 years of the war compared to 601,000 violent deaths for the first three-and-a-half years of war estimated by the Lancet study (note: the Lancet study did not distinguish between combatant and non-combatant casualties).
There are no credible studies of combatant casualties, but around 9,000 Iraqi troops were killed during the invasion, I've seen estimates for insurgent fatalities placed at around 20,000-50,000, and many thousands of Iraqi Security Forces (army and police) have been killed. If we subtract, say, 70,000 combatant casualties from the totals, we get 45,000-103,000 civilian deaths. The current IBC total for Iraqi civilian deaths from war-related violence is currently around 65,000.
Now, there are a ton of debatable assumptions in this exercise, but it seems that the USA Today poll question on households suggests that the civilian fatalities in Iraq are much closer to the IBC numbers than the Lancet numbers.
Now it could be argued that the better question to look at from the poll is the one that says that 53 of Iraqis surveyed said they knew someone who has been physically harmed. (Actually, I would think that the number on households from the USA Today poll is more reliable because there is less risk of double counting, but for the sake of argument, lets analyze the 53 percent number). As a point of comparison, consider the fact that 17 percent of Americans in a recent poll say they know a fellow American who has been killed or injured in Iraq (and 27 percent of those under 35 know someone) (http://www.gmanews.tv/story/31950/Americans-underestimate-Iraqi-death-t…). And this is in the context of 3,100 American deaths and 23,000 injuries (about 5 percent of the lowest Iraqi total), out of a total U.S. population of 300 million (more than eleven times the Iraqi population). This means that physical injuries to 0.0087 percent of the U.S. population fighting a war 8,000 miles away has been sufficient to translate to 17 percent of Americans knowing someone has been hurt.
Now, if we took the numbers I extrapolated above from the household question in the USA Today poll (which some would consider conservative), we get somewhere in the neighborhood of 115,000-173,000 Iraqi deaths (including combatants) and 344,00-517,00 injuries, and considerably more events that "harmed" people in other ways (like kidnappings), totaling about 2 percent of the total Iraqi population of 26 million. As a percentage of the population, this is 230 times (!) the number of American injuries/deaths. It is therefore actually surprising that the number of Iraqis who know somebody who has been harmed is not much higher than 53 percent (only a little over 3 times the percentage of Americans who know someone hurt) given the very large social networks Iraqis are embedded in compared to Americans.
Now consider what things would look like if the Lancet numbers were correct. According to the Lancet, 601,000 Iraqis were killed by violence in the first three and a half years of war, and, at a ratio of 3 injuries to 1 death, this implies 1.8 million additional people have been wounded. That total of deaths and injuries would represent 9 percent of the entire Iraqi population. Does anyone really think that, in a society where the average household has almost 8 members and extended kin networks are huge, where modern road and communications networks keep family connected, and where, according to the Lancet, deaths and injuries are spread out fairly evenly across the country that there would be a single Iraqi who wouldn't know somebody who was killed or injured if the Lancet numbers were true? In this context, there is NO WAY the 53 percent figure from the USA Today poll can support the Lancet numbers.
The war in Iraq has been horrific. It is a tragedy and a calamity and perhaps even criminal. But nothing else we know about the war supports the astronomical Lancet numbers. I know many, many people who have served in Iraq, worked in Iraq providing aid, contracts, or working on Iraq issues inside and outside the government over the entire course of the war. Not one of them thinks the Lancet numbers can be correct. As Jon Pedersen, the lead researcher for the UNDP household cluster survey conducted in 2004 (the only other, and much larger, study to use the Lancet method) recently stated, the Lancet numbers are "high, and probably way too high. I would accept something in the vicinity of 100,000 but 600,000 is too much" (http://blog.washingtonpost.com/worldopinionroundup/2006/10/is_iraqs_civ…).
"Death Toll Hits 'Million'
Updated: 16:38, Sunday March 18, 2007
The number of deaths in Iraq since the start of the conflict could be as high as one million, according to a new estimate.
An Australian scientist published the figure using four sets of independent data, including information from the United Nations Children's Fund and the UN's population division, as well as medical literature.
Dr Gideon Polya's figure is far higher than the previous biggest estimate of 655,000.
"Using the most comprehensive and authoritative literature, and UN demographic data yields an estimate of one million post-invasion excess deaths in Iraq," he said."
http://news.sky.com/skynews/article/0,,30000-1256336,00.html
Add to this about 1.8m refugees in the neighboring countries, PLUS about another 1.8m people displaced within Iraq - and this makes us safer HOW?
.
The Australian scientist Gideon Polya's estimate is not a new estimate based on new data. It is an extrapolation from the Lancet estimate, which covered the first three-and-a-half years, forward to the current period. See: http://www.aljazeerah.info/Opinion%20editorials/2007%20Opinion%20Editor…
I did a rather shorter back of the envelope calculation with the USA today poll in a previous thread and have done it a couple of other times. Basically, depending on how many "harmed" people there are per household, you can get a death toll in the low hundred thousand range or up in mid-hundreds of thousands. Some households might have been totally wiped out and it's possible that those households which have been hit (by insurgents, death squads, checkpoint shootings, mortar attacks, rockets, etc...) would likely suffer multiple casualties. So there's no particular reason to assume the average number is only slightly greater than 1.
The fraction which are dead would depend--death squads might produce more dead than injured. Explosions tend to produce large numbers of injuries, but bullets to the head mostly produce deaths. There's no particular reason to think that the media reports of dead/injured are accurate. Deaths might be more newsworthy. Plus there's the usual skepticism one ought to have about the accuracy of statistics that come from interested parties during wartime.
I avoided the question of how many people "know" someone killed because that is really vague.
It's interesting how little we know about the insurgent death toll. There are scattered reports one can find, but they all seem to be guesses, or information passed on from some vague Pentagon source. There might have been tens of thousands killed, or maybe fewer--the press doesn't seem to have any way of determining the number or much interest in pushing to find out. Yet they have the answers regarding the number of civilian dead. Curious.
Forgot to make my other point regarding insurgents. Insurgents which fight in urban areas are difficult to kill without killing civilians. So if the US has managed to kill tens of thousands of insurgents, how many civilians were killed in the process? Iraq Body Count counted about 7000 dead killed by the US in 2003, almost entirely in the invasion months, and roughly 2000 or so in Fallujah in two assaults in 2004. In other months it's typically averages about one civilian death a day attributable to US forces. In the third year of the occupation, for instance, IBC counted 370 deaths attributable to coalition forces.
If during the same post-invasion period they've managed to kill tens of thousands of insurgents, it's a remarkable performance. Really, really, really remarkable.
Donald's point about the uncertainty of how many Iraqis were "harmed" per household in the USA Today poll is significant. But if we assume the IBC ratio of wounded to dead (which is based on all the different types of attacks you mention: gunshots/executions, car bombs, crossfire, airstrikes, etc.), then there would have to be, on average, at least 4 members of EVERY household included in the 17 percent figure would have to be harmed for the Lancet numbers to be correct, with the remaining 83 percent of the households having zero casualties. That seems highly implausible on multiple levels.
Two other things. First, I should have made clear in my previous post that IBC includes media reports of morgue and hospital data, which definitely pick up executions, so this is not a reason, in and of itself, to question their ratio of wounded to dead. Moreover, this ratio is strikingly similar to other civil wars.
Second, much has been made about the refugee statistics, but the statistics on refugees actually produce additional skepticism on the Lancet findings. As of October 2006 (the month the Lancet was published), the UNHCR estimated that there were 1.5 million internally displaced persons (IDPs) in Iraq, of which approximately 800,000 were displaced PRIOR to the 2003 invasion and 754,000 afterwards. At the time, another 1.6 million Iraqi refugees were living in neighboring countries, although the UNHCR did not estimated how many of these have fled Iraq since the invasion. If it follows the IDP ratio, it would represent another 800,000, suggesting a grand total of around 1.55 million internal and external refugees created since the invasion. If we included all of the external refugees, the total number would be roughly 2.35 million for the period covered by the Lancet study.
If we take these totals and divide by the number of excess violent deaths from the Lancet study (601,000), we get the following refugee to death ratios: 2.6:1 (for the low end of refugees) and 3.9:1 (for the higher estimate).
According to research on refugees produced by violent conflict, there are very few historical cases, especially those involving a considerable amount of communal conflict and the direct targeting of civilians, with ratios this low. It is very rare to see ratios as low as 10 refugees per death, and it is much more common to see ratios 100 to 1 or higher. Indeed, in a country like Iraq, with viable roads, air travel, and many neighboring countries, we would expect the ratio of refugees per death to be on the higher side. Even if we take the low end of "typical" refugee ratios here (10:1) we should have seen 6 million Iraqi refugees (about 22 percent of the entire Iraqi population) by autumn 2006 if the Lancet estimate is correct, and we did not.
Colin, the 53 percent figure was for people who knew a friend or relative harmed, not for people who knew someone harmed. It's a vague category, but not this open-ended very large group that you postulated.
I gave an argument myself that went against my own case. If the wounded are under-reported relative to deaths, then the dead/casualty rate is lower than IBC statistics would indicate. I need to go home and look at the IBC two year analysis--my vague recollection was 24,000 deaths and 40-something thousand wounded.
As for refugee flows, I doubt your claims. I'll use your 10/1 ration. Are there 40 million refugees from the Congo? Were there 17 million refugees from Pol Pot's Cambodia? Uh, no, there weren't that many Cambodians. Were there 2 million refugees from East Timor? No again, and for the same reason. In South Vietnam, there were 5-7 million internal refugees and the death toll (both civilian and guerilla) was probably anywhere from the high hundreds of thousands to over 2 million, depending on who you believe.
I think this poll supports a number considerably higher than IBC would acknowledge--your own guestimate has to assume that the majority of the deaths go unreported, even if you assume that these were all combatants. That brings back my question about the number of insurgent deaths and the mysterious absence of large numbers of civilian casualties if insurgents are being killed by the US in large numbers.
Donald, as you stated previously, the household numbers are more telling and you did not respond to my point that we would have to assume that at least 4 members (out of 7-8 total) in EACH of the households represented in the poll were harmed to get close to the Lancet numbers, with the remaining households having ZERO casualties. Do you honestly think that is likely?
As far as the 53 percent number from the USA Today poll goes, it depends on the interpretation a respondent has to "relative" or "friend." Remember, an average Iraqi household contain between 7-8 people (all likely to be considered immediate family) and the number of additional people considered to be close kin can easily number in the dozens (given the tribal structure of Iraqi society). Throw friends in the mix, and we would expect 100 percent of Iraqis to know some relative or friend "harmed" (especially since that term undoubtedly includes not only people actually wounded, but people physically harrassed, kidnapped, etc.). Even if the total number of relatives plus friends is, say, ONLY 10 (and we can be confident it is much higher than that), 600,000 Lancet deaths (plus 1.8 million implied injuries), suggests that 100 percent of respondents should know a relative or friend who is harmed (remember the high U.S. poll totals for far fewer harmed and a MUCH, MUCH larger populations).
On refugees, you are right that there are some cases where the ratio is below 10:1 -- which is why I referred to "typical" ratios. Excluding cases of genocide, there are perhaps half a dozen or so over the past several decades of civil conflict (it is difficult to say exactly because the data on both casualties and refugees/IDPs is so problematic for many cases). But based on available data, ON AVERAGE, ratios as low as the ones implied by the Lancet are very, very rare. And when they occur, as in the Congo or East Timor or Guatemala, they typically occur in countries with very small populations, or countries where bad transportation infrastructure and/or countries geographc factors makes movement or exit difficult. None of this describes Iraq. Moreover, refugee ratios appear to be especially high in cases where there is extensive communal violence and sectarian/ethnic cleansing (where the whole purpose of the violence is to drive out civilian populations from territory), a description that fits Iraq very well.
Eli is charmed by Kevin's faith in Iraqi bureaucracy. Not impressed, but charmed.
Donald, as to your point about the number of insurgent deaths compared to civilian deaths caused by U.S. forces, there are uncertainies all around. Based on extrapolations from IBC, the Brookings Index, and the U.S. military's own reporting of "escalation of force incidents" at checkpoints and convoys (many of which do not get coded as "U.S. caused" in the IBC dataset), we get a range of about 8,478-14,750 civilian fatalities caused by U.S. forces or crossfire during engagments with anti-coalition forces. Whether this seems implausible given the number of likely insurgent deaths hinges on the assumptions one makes about the precautions U.S. troops take to avoid killing civilians (see, for example, www.foreignaffairs.org/20061101faessay85608/colin-h-kahl/how-we-fight.h…)
But, it is important to note that, in this regard, the Lancet is also highly problematic.
According to the Lancet findings, attacks by coalition forces, including air strikes, accounted for 31 percent of excess violent deaths in the first 3 1/2 years of the war. This translates into 186,000 deaths attributable to U.S. forces -- nearly 4,700 a month or an average of 155 (or 6.5 Hadithas) every day for 1,200 straight days.
Furthermore, while the study notes that the proportion of deaths attributed to U.S. forces has gone down over time, the absolute number has gone up every year since the beginning of the war: from 32,000 between March 03-April 04 to 70,0000 from May 04-May 05 to 86,000 from June 05-June 06, with 2006 being the highest total yet.
Indeed, if these estimates are correct, almost three times as many were killed by U.S. forces in the last year of the study period than the first year of the war (when the invasion occurred and combat was high-intensity).
Moreover, in addition to overall deaths from U.S. forces, the Lancet data also suggests that the sub-set of these deaths attributed to coalition airstrikes has also gone up every year (from 12,000 in the first period to 26,000 in the second to 40,000 in the most recent period of the study).
All told, if these estimates are right, it implies that at least 172,000 Iraqi fatalities caused directly by U.S. forces have gone unreported by anyone since March 2003 even though insurgents, militias, and others that oppose the coalition have huge incentives to report these incidents for political purposes and have ready access to media.
Based on extensive research on U.S. conduct during this period, these numbers and trends border on the impossible, suggesting that something is seriously wrong with the methods or sample used in the Lancet study. The scale and trends suggested by the Lancet study cannot be reconciled with a number of facts:
First, the scale of U.S. operations in later periods is way down compared to the major combat phase in 2003 and major offensives in Fallujah, Karbala, Najaf, and Sadr City in 2004.
Second, there has been a drastic reduction in airstrikes over time. In the first thirty days of the war, there were 20,733 airstrikes (18,695 of which were from American planes). In contrast, official figures from Central Command Air Forces (CENTAF) suggest that there were 285 airstrikes in 2004, 404 in 2005, and 229 in 2006. These numbers underestimate the number of airstrikes because they do not include Marine Corps aviation units (but the same type of underestimate applies to the CENTAF numbers cited for the invasion). Based on this data, on average there appear to have been fewer airstrikes per year in 2004-2006 than the average number of airstrikes per day (623) during major combat. Now, how is it that, according to the Lancet, airstrike deaths go up every year?
Third, there has been a significant decrease in the use of artillery by U.S. forces over time.
Fourth, the size of the munitions used has gone down over time and the accuracy has gone up (this is true of both air delivered munitions and artillery rockets).
Fifth, training, tactics, procedures, and the emphasis on avoiding civilian deaths within the Rules of Engagment have improved over time.
Finally, until the recent Baghdad "surge" (which happend AFTER the Lancet study was conducted), there were significantly fewer patrols by U.S. forces and shrinking presence in most Iraq villages/towns/cities over time, which helped create the security vacuum producing the civil war but also meant fewer possible engagements in which Iraqis could be killed by U.S. forces even if we assume U.S. forces behave badly.
This all suggests two possible implications. First, it provides more evidence that the general findings of the Lancet study are highly questionable. It also suggests that, at the very least, even if the Lancet study's overall numbers are right, the survey respondents might be blaming coalition forces for deaths caused by other sources.
The argument about the air strikes is old news. I read IBC's response to their Lancet-inspired critics when it came out last year (before L2). I don't take the L2 number for air strikes very seriously, but nor do I take much of anything I read about air strike statistics too seriously either. Governments lie, you know. But yeah, I agree that 80,000 civilians killed by US air strikes is hard to believe and I'd want more evidence before I believe it.
I'm not totally in the L2 corner either. It's one reason why whenever this issue comes up I'm constantly wishing someone else would do a survey that would replicate or refute it, though it's something I'd say that even if I were completely convinced by L2, since it's something that could be done. This ABC poll was large enough that they could have determined the death toll (I say as an amateur) if they wanted, but as always seems to happen, nobody in the press is interested in commissioning a poll designed to do just that, even though they all know perfectly well how controversial the issue is. The government could do it too, of course, but I'm glad they haven't bothered--no one in their right mind would trust them.
If I were going purely on this ABC poll, I'd plug in a broad range of numbers and see what comes out. So assume an average casualty rate per harmed household of 1 to 3 and assume a death to casualty ratio of 1/4 to 1/3. You get 500K to 1.5M casualties and 125K to 500K deaths, the latter falling within the L2 CI. If I just wanted to guestimate from this poll, I'd go for the middle--maybe around 1 million casualties and 250K deaths. That wouldn't satisfy either side. But I wouldn't bet a dime on my number either.
Anyway, my sense from reading the poll is that Iraq is a lot more violent than the official numbers suggest, whether or not L2's midrange number is correct.
I looked at the two year IBC analysis downloaded on my home computer. When they had data on both, they found a ratio of 3 wounded to 1 dead, as Colin said. When small arms alone were involved, the ratio was 1.2 wounded to 1 dead. Death squads presumably don't leave as many wounded as bombs.
I agree with Donald completely that we need better data all around. I wish the media, military, etc. would provide credible numbers. Indeed, I've called for just such actions in print. But I think that anybody who spends a great deal of time looking at the entire range of existing available data and studies can find a lot of questionable or suspicious findings and implausibilies related to the Lancet estimates. In that context, people should be careful about uncritically accepting it just because they hate the war. A reasonable person can conclude that the "real" number of deaths is likely to be closer to 100,000 than 600,000 without implying that the war was worth it. Iraq is a horrible tragedy regardless. We should therefore let logical inference and our overall knowledge about the nature of the conflict (as incomplete as it is), not politics, drive our interpretations of the Lancet.
Colin is magnanimous. He says , "It [the Iraq invasion and occupation] is a tragedy and a calamity and perhaps even criminal".
Perhaps? Maybe? There's a chance?
Colin, the war and subsequent occupation were and are criminal. No ifs and buts. And what about the 2 million Iraq refugees who have fled the country, the largest such exodus since the Palestinians were displaced in 1948? And the 'sanctions of mass destruction' that resembled a medievel siege and which left many hundreds of thousands (or more) dead? Or the Gulf War when the coalition deliberately targeted the entire civilian infrastructure of Iraq for desctruction? How many Iraqi corpses - based on control of an area of great economic and strategic importance which gives the U.S. 'veto power' over the world economy (in the words of George Kennan) - constitute a crime? Trying to placate the reader by claiming the real civilian death toll in Iraq (since 2003) is probably only '125,000-150,000' is like saying that "I didn't blow up your house with dynamite! I just burned it to the ground!".
The fact is this: U.S. criminality in Iraq goes back a long time, befre that to be replaced by British criminality.
" In that context, people should be careful about uncritically accepting it just because they hate the war."
It appears to me that the reverse has happened much more often. The Lancet estimates were first ignored by the mainstream media that supported the invasion of Iraq, then later attacked and ridiculed most vigorously by those who supported the war. None of the latter, it appears, have made any real effort to conduct their own scientific studies. Extrapolations from a poll hardly seems like a good substitute.
Eli:
Glad to be charmingly naive. Now please explain why the inefficient Iraqi bureacracy apparently managed to distribute death certificates to a vast majority of households interviewed by Lancet, when they remembered to ask for them, and how tracking whatever local authorities are issuing them wouldn't be more accurate. Call me dense, as well.
Kevin asked:
It's because they're different bureacracies, with different inefficiencies. I'll add that unless you had complete tracking of every issuing authority, you'd have to sample them -- and sampling is something you seem to be against as being "inaccurate."
Well, this has been pointed out before.
"It's because they're different bureacracies, with different inefficiencies."
How do you know this? Must have taken considerable study of the structure and inefficiencies of all the various Iraqi bureacracies to know this explains this kind of discrepency. Either that, or this excuse is a pure invention to parry an inconvenient criticism of the Lancet study.
And why hasn't anyone in these bureacracies that supposedly issued 550,000 death certificates ever used this documentary evidence to show that they've recorded vastly more violent deaths at every stage of the conflict than the figures that are circulated publicly from that other bureacracy?
"how tracking whatever local authorities are issuing them wouldn't be more accurate."
Some, including Richard Garfield of L1 fame, seem to agree with you on this:
"Over-reporting of deaths was regarded as limited because 92% of reported deaths were supported by death certificates, but Burnham and colleagues do not report who issued these certificates. ... If the death certificates are valid and the availability above 90%, it seems better to monitor mortality by compiling data from the local agencies that issue these certificates than by doing further dangerous household surveys."
http://www.thelancet.com/journals/lancet/article/PIIS0140673607600580/f…
I'd agree too, but for some reason, I suspect that method wouldn't produce anything like the Lancet numbers.
[joshd](http://www.apa.org/journals/features/psp7761121.pdf) asked:
Because I've worked in bureaucracies that used and analyzed the data on death certificates and so had to know how they're filled out, issued, and aggregated. Basically, the issuance of all vital certificates (birth, death, and marriage certificates) is quite localized and so are subject to all the problems of local bureaucracies while the collection and aggregation is handled (or mishandled) by centralized bureaucracies. Even in places where the vital registration system works quite well (like the US or Western Europe), there are different kinds of reporting errors and delays between rural and urban areas, between hospitals and morgues, between deaths attended by a physician and those not, and even between certificates filled out by a coroner and a pathologist. What this means is that a breakdown in the aggregation system, even one sub-part of the aggregation system, can be quite independent of the issuance system. To what extent this has happened in Iraq is the critical question; however, simply claiming that because a death certificate was issued at the local level it must also have been properly counted at the national level is quite a leap except, of course, in bizarro-[joshd](http://www.apa.org/journals/features/psp7761121.pdf) world.
So Robert has worked somewhere in some capacity involving death certificates in some way, therefore he's intimately familiar with the workings of Iraqi bureacracies and knows that missing 500,000 of them and getting about 50,000 of them (with none of the "local bureaucracies" coming forward with their documentary evidence for this baldly, egregiously, obvious discrepency between their records and the publicly stated figures) is perfectly reasonable.
Since you're so attuned with the particular workings of Iraqi bureaucracies Robert, why don't you tell me who issued the 550,000 death certificates for violent deaths, and please be specific.
"To what extent this has happened in Iraq is the critical question"
In short, you don't know and have no evidence that this happened, yet it's true because the Lancet study needs it to be. Therefore, the explanation emerges from whole-cloth.
Then, as usual, Robert goes immediately to straw man:
"simply claiming that because a death certificate was issued at the local level it must also have been properly counted at the national level is quite a leap"
Yes, I'm making a claim about "a" death certificate.
[joshd](http://www.apa.org/journals/features/psp7761121.pdf) babbled:
Dude, wipe the spittle off your chin and calm down. Since you don't believe me, call up your local registrar of deaths and ask him or her this question: if there were a failure in the reporting system up the chain from them, would they continue to hand out death certificates or would they have to stop?
Robert:
I think there are many reasons why polls are skewed and sampling is subject to biases unforeseen by the pollsters; cluster sampling seems more prone to those errors than respresentative sampling. Why not do a representative, rather than a cluster sampling of local, death cert. issuing authorities? Would this allay your concerns, since your distrust of central collection in Baghdad is so much higher than your distrust of the Lancet study despite the authors and interviewers not being able to agree on what the interviewers did?
Kevin wrote:
First, I agree that there are many reasons why polls are skewed; but I think you've misunderstood the word "representative" in your sentences above. All scientific sample designs are intended to be representative of the population that's being sampled. Perhaps you meant "simple random"? The main sampling designs (simple random, systematic and cluster) and their variants each have strengths and weaknesses; I don't know where you got the idea that cluster sampling is somehow "more prone" to bias (in fact, systematic sampling--as you may be able to tell from its name--is often most susceptible to systematic bias). However, cluster sampling *does* tend to be less precise; I think we went over the difference between inaccuracy and imprecision above.
Second, just because there are fewer places where death certificates are issued doesn't mean that it's a snap to design a good sample of them. In fact, mortality clustering makes the problem of getting a representative sample of issuing authorities even worse than for clusters of households: that's because, unlike household clusters, you'd be trying to select the sample based on the very thing you're trying to count. Here's an example that may make this a bit clearer: suppose you were trying to get a count of the total number of parking tickets issued over the course of a year in the US. How would you get a representative sample of ticket issuers? There are city police departments, county sheriffs, state police, transit authorities, housing authorities, national park police, and Indian reservation police; and some are huge and others are tiny. How would you construct a sample of issuing authorities that would let you make a nation-wide estimate?
Third, the real advantage of sampling households as opposed to issuing authorities is that the household sample provides the population at risk so you can calculate death rates, i.e., household sampling gives you both the numerator and the denominator that you need. Sampling issuing authorities only gives you the numerator.
This is a fairly ludicrous example of what is known as overanalyzing data. Remember that whereas most reported deaths were documented, the interviewers simply tabulated the opinion of the residents of the house as to whether the death was caused by US forces--residents who in many, perhaps most, cases did not observe the actual event. So this number can be expected to tell us very little about the actual cause of death, and a lot more about who Iraqis were inclined to blame at a particular point in time.
Regardless of the validity of the sampling technique, it would be surprising to find a good correlation of these reports with levels of types of US military activity. I can't imagine why anybody would make such a patently unreasonable assumption unless they had a political axe to grind.
Robert:
I don't understand how the relative imprecision of cluster sampling is somehow a defense of the Lancet study.
>Here's an example that may make this a bit clearer: suppose you were trying to get a count of the total number of parking tickets issued over the course of a year in the US. How would you get a representative sample of ticket issuers? There are city police departments, county sheriffs, state police, transit authorities, housing authorities, national park police, and Indian reservation police; and some are huge and others are tiny.
You seem to be making this harder than I think it needs to be. How many authorities are issuing death certificates in Iraq? I somehow doubt the local meter maid is churning them out from the community photocopier. Why not attempt to gather data from every issuing authority in Iraq for some set timeframe?
>Third, the real advantage of sampling households as opposed to issuing authorities is that the household sample provides the population at risk so you can calculate death rates, i.e., household sampling gives you both the numerator and the denominator that you need. Sampling issuing authorities only gives you the numerator.
I thought excess death rates were calculated by comparing assumed rates of death to prior rates of death. How would sampling issuers of death certificates fail to parallel that?
How would a cluster sampling in which houses which didn't answer were skipped and never returned to solve the problem of determining the population more accurately than the last census or whatever?
"All told, if these estimates are right, it implies that at least 172,000 Iraqi fatalities caused directly by U.S. forces have gone unreported by anyone since March 2003 even though insurgents, militias, and others that oppose the coalition have huge incentives to report these incidents for political purposes and have ready access to media.
Based on extensive research on U.S. conduct during this period, these numbers and trends border on the impossible, suggesting that something is seriously wrong with the methods or sample used in the Lancet study."
That was Colin. Anyway, as I recall, in late 2003 some Iraqi group claimed to have actually counted 37,000 deaths by the fall of 2003, far higher than IBC numbers at the time, and in 2005 the Washington Times carried a story about another Iraqi group claiming to have actually counted 130,000 or so deaths.
In general the Iraqi insurgents seem rather hostile to Westerners and journalists, more inclined to kill them than take them around and show them the conflict from their viewpoint. Or such is my impression. But there are those two exceptions where groups obviously opposed to the occupation did present numbers much higher than the official ones of the time. (And lower than Lancet 2, at least the second one, but an actual count would miss deaths.) Whether these numbers are real or fictitious is another question, but it's not as if there are no figures opposed to the official ones other than the Lancet.
Universally agreed upon numbers for the Vietnam and Algerian/French conflict were and are also notoriously hard to find. The Vietnamese government claimed after the war to have hugely understated its civilian losses to keep from undermining morale.
Kevin wrote:
Oh dear. I was simply pointing out that you have (once again) confused imprecision with bias - and that cluster sampling isn't biased. What you've been doing is sort of like criticizing a guy for being overweight but using his height as your evidence. In any event, the "relative imprecision" of cluster sampling simply means that one needs a larger sample size relative to simple random sampling in order to get equivalent precision. Both the
Roberts and Burnham studies did this.
As I said, the problem isn't the number of authorities (though in a country with a population of around 25 million it wouldn't be surprising if there were a few thousand). The problem is that some of them are going to be quite large while others are quite small. I happen to think this is quite a difficult problem but since you think the problem is simpler than I've been saying, please describe how you would construct a representative sample of all issuing authorities and then use it to make an unbiased estimate of the nationwide total.
Your assumption is incorrect. The Roberts and Burnham studies are cohort studies where one counts not only the deaths in each household and when they occurred but also the "population at risk." Counting death certificates doesn't provide you with the population at risk.
Haven't you read the Burnham paper? There were a total of 31 households out of 1880 that were either skipped or never returned to, i.e., this particular problem is not a very big problem.
trrll does not dispute any of the trends in U.S. military activities noted in my previous comment. Nor does trrll contest that it makes the L2 estimate of 186,000 excess violent deaths attributable the U.S. forces, and the trends in those deaths, highly improbable. Instead, trrll argues that it is ridiculous to extrapolate from the data in L2 regarding those directly responsible for violent deaths, arguing, "the interviewers simply tabulated the opinion of the residents of the house as to whether the death was caused by US forces--residents who in many, perhaps most, cases did not observe the actual event. So this number can be expected to tell us very little about the actual cause of death, and a lot more about who Iraqis were inclined to blame at a particular point in time."
Fair enough. However, I encourage you to apply the same argument to the L2 authors. On p. 6, for example, L2 contends: "Deaths were not classified as being due to coalition forces if households had any uncertaintly regarding the responsible party; consequently, the number of deaths and the proportion of violent deaths attributable to coalition forces could be conservative estimates." (Note: they never say that it "could" also be the case, and indeed is much more likely to be the case, that the estimates are way too high.) And, on p. 7, L2 follows this up with the claim that "Coalition forces have been reported as targeting all men of military age."
Thus, the L2 authors seem to have no problem using their estimates of fatalities directly attributable to U.S. forces as evidence that those forces are systematically and indiscriminantly targeting ALL Iraqi military aged males regardless of their status as combatants or noncombatants. The author's only evidence for this, independent from their estimate, is based on accusations by defense attorneys representing soldiers charged with murdering three Iraqi men during a raid near Samarra in May 2006 regarding the ROE issued by a single brigade commander (see L2 endnotes 27 and 28).
Given the evidence presented in L2, trrll, do you really think the most reasonable interpretation of the estimate of 186,000 violent deaths attibrutable to coalition forces is that it is "conservative"?
Nowhere in L2 do you find the note of caution found in trrll's post regarding our ability to meaningfully extrapolate the cause of death from the answers given by survey respondents (regarding coalition forces in general, airstrikes, car bombs, etc.). trrll, given the nature of the presentation in L2, are you willing to admit the possibility of a political bias (on your part or the L2 author's part) in the opposite direction to the one you (incorrectly) accuse me of?
Have you been off on vacation anywhere nice Colin?