Comments

1. #1 Carl Christensen

   October 12, 2006

   well I’m awaiting a “LancetAudit.org” site to be created.

2. #2 Oskar Shapley

   October 12, 2006

   Seixon asked for a death certificates “recount”. I believe it should be possible to verify at some morgues how many certificates were issued.

   The data at the central level might be manipulated or incomplete, but one should be able to make a survey of morgues and investigate what evidence there is. Are there lists and registers of issued certificates? How much are certificate stocks being “used up”?

   This data could then be corrected by the ratio of total/certificated deaths from the Lancet study.

3. #3 DrSteve

   October 12, 2006

   Tim, I’d honestly be very interested to know what you think about how the methods employed by Dr. Burnham and his colleagues would handle spatial correlation.

4. #4 clazy

   October 12, 2006

   Following on Oskar’s comment, one of the most interesting aspects of this new study is the assertion that 90% of the deaths were documented by death certificates.

   Shocked as I was by the large number of deaths extrapolated, my first impulse was to wonder if the figure of 90% was genuine, since the death certificates lend the study their authority. Assuming it is genuine raises another question, though: If death certificates are so closely correlated with actual deaths, why risk life and limb wandering the countryside to take a small sample when you can visit the hospitals that issue death certificates and make a precise accounting? (The investigators needn’t rely on the central government.) The number of visits would be no greater, and the numbers would be more accurate. Consider the value such a comprehensive study would have. We’d be able to see the rise and fall of deaths across time and by region. And the investigators would be able to look for manipulation based on whether the trends accorded with the news coming out of the area.

   How much more than this study’s $55k would that cost? I would be interested in whether the group behind this study ever considered such an approach.

5. #5 Kevin P.

   October 12, 2006

   Tim Lambert:

   Since only a small fraction of the deaths were caused by bombing, this is hardly relevant.

   Incorrect. Source

6. #6 JB

   October 12, 2006

   I think the real question — which those questioning the Lancet results are conveniently avoiding — is “How many Iraqi civilian deaths are ‘acceptable’.”

   30,000? (Bush’s number)

   60,000? (the number provided by the Iraqi government and based largely on Baghdad morgue deaths).

   100,000? (The first Lancet study number for a shorter time period and based on surveys)

   650,000 (The second Lancet study number for deaths to date)

   If you ask me, even 30,000 deaths is a very large price to pay if you happen to be in one of the affected families.

   But I’m not the one to ask.

   Ask those who are now questioning the Lancet number (or the Iraqi ministry number, or even the Bush number) what they think.

   I suspect that the later group (reprsneted by some of the commenst on this blog) do not give a damn how many Iraqis have died since the war began, just as they don’t give a damn that no WMD were found, or that Saddam was not in cahoots with Osam bin Laden, or that many brutal dictators have been deposed quite bloodlessly over the past few decades.

7. #7 BruceR

   October 12, 2006

   Can we stop saying the 650K number refers to *civilian* deaths? The Lancet does not make that claim.

8. #8 Robert

   October 12, 2006

   Clazy wondered:

   If death certificates are so closely correlated with actual deaths, why risk life and limb wandering the countryside to take a small sample when you can visit the hospitals that issue death certificates and make a precise accounting?

   Because even if those records are complete, it doesn’t give you the population at risk (i.e., the denominator that you need to calculate the death rate). At very best, that only gives you the numerator.

9. #9 Gar Lipow

   October 12, 2006

   >Because even if those records are complete, it doesn’t give you the population at risk (i.e., the denominator that you need to calculate the death rate). At very best, that only gives you the numerator.

   This took me a minute to get.
   What you are saying is that you could not simply sample morgues and hospitals – because so long as you failed to hit every morgue or hospital you still be doing sampling and would need general population to compare it against – something you can’t directly get from hospitals or morgues. To make sure each death represents a death per certain number of live Iraq’s you have to hit households.

   An alternative is that someone could try to muster the resources to visit every morgue, hospital, etc. in Iraq.Orders have been already been given that these are now barred from giving out figures to reporter, cause nosey journalists kept putting together figures showing the official figures were nonsene. I’ll bet if a full independent mortaltity census was begun in Iraq, it would quickly be extended to researchers.

10. #10 JB

    October 12, 2006

    I think Richard Horton, editor of the Lancet, has given the best explanation for what the study means:

    http://www.guardian.co.uk/frontpage/story/0,,1920167,00.html

    “Why is this Lancet estimate so much higher than the figures put out by President Bush or the Iraq Body Count website? They put the number of casualties in the tens of thousands, not the hundreds of thousands. To be fair, Iraq Body Count does not claim to publish accurate absolute numbers of deaths. Instead, their figures are valuable for measuring trends. But the reason for the discrepancy between these lower estimates and the new figure of 650,000 deaths lies in the way the number is sought. Passive surveillance, the most common method used to estimate numbers of civilian deaths, will always underestimate the total number of casualties. We know this from past wars and conflict zones, where the estimates have been too low by a factor of 10 or even 20.

    “Only when you go out and knock on the doors of families, actively looking for deaths, do you begin to get close to the right number. This method is now tried and tested. It has been the basis for mortality estimates in war zones such as Darfur and the Congo. Interestingly, when we report figures from these countries politicians do not challenge them. They frown, nod their heads and agree that the situation is grave and intolerable. The international community must act, they say. When it comes to Iraq the story is different. Expect the current government to mobilise all its efforts to undermine the work done by this American and Iraqi team. Expect the government to criticise the Lancet for being too political. Expect the government to do all it can to dismiss this story and wash its hands of its responsibility to take these latest findings seriously.”

11. #11 Tim Lambert

    October 12, 2006

    DrSteve: spatial correlation means that in a cluster survey you have to survey more people than simple random sampling to get the same standard error for your estimate. They took this into account when they decided how many people to survey and when they calculated the confidence intervals.

12. #12 djotefsoup

    October 12, 2006

    it is time to call for volunteers from the 101st fightin’ keyboarders to do the hard slog of verifying the figures at iraq’s hospitals.

    http://www.cbsnews.com/stories/2006/10/04/eveningnews/main2064668.shtml

13. #13 Ragout

    October 12, 2006

    In their methods section of their paper, the Lancet authors discuss at length a “log-linear regression model.” Tim Lambert says (of the methods section) “And if you don’t understand that paragraph you probably shouldn’t be criticising the study’s methodology.”

    Perhaps Tim can point me to where in the paper the results of the log-linear model appear? These results should take the form of relative risks, I believe. But unlike their previous study, I don’t see any relative risks reported.

    In other words, the authors devote most of their methods section to discussing a particular statistical model, and then never report the results of that model! I don’t mean to suggest that anything nefarious is going on (I’m a little suspicious, but only a little). But I do think that this tells you that the peer reviewers and editors didn’t examine the paper’s statistical methods all that carefully.

14. #14 Seixon

    October 13, 2006

    Tim,

    You quote the study saying:

    “Even with the death certificate system, only about one-third of deaths were captured by the government’s surveillance system in the years before the current war, according to informed sources in Iraq. At a death rate of 5/1000/year, in a population of 24 million, the government should have reported 120,000 deaths annually. In 2002, the government documented less than 40,000 from all sources. The ministry’s numbers are not likely to be more complete or accurate today.”

    Riddle me this:

    1. Why do the practices of Saddam Hussein’s government have any relevance to the current state of affairs in Iraq?

    2. Who are these “informed sources” and why should we believe them, and back to #1, why does it matter what they know about how Saddam Hussein did his thing?

    3. Is that last sentence supposed to be coming from a scientist? Sounds like yet another convenient assumption on the part of the JHU folks.

    In fact, that last sentence is an affront to the current Iraqi government and the Iraqi people as a whole. Also notice that their only reason for not checking morgues and such is that “there is a suspicion” that things are being hidden because of political reasons. Again I ask, upon which scientifical basis is this being said? None. It’s the political bias of the authors being inserted into the study to make excuses for why they didn’t do something that would have given more trustworthy and reliable results.

    If they had checked with the government records and the morgues, they could at least have established a minimum number of excess deaths. It’s quite apparent that that’s not what they were interested in, however.

    I also find it hilarious that they think you get a more accurate number by interviewing 1800 families and extrapolating from there instead of by actually counting reported deaths.

    IBC has a basis of at least 45,000 recorded deaths. The JHU study has around 600? I think there comes a point where scientists believe a little bit too much in their methods and it becomes dogma instead of science.

15. #15 dsquared

    October 13, 2006

    Seixon, the last time round you were very keen on making detailed comments on statistical methodology. This time, you are writing things like:

    [I also find it hilarious that they think you get a more accurate number by interviewing 1800 families and extrapolating from there instead of by actually counting reported deaths.]

    which obviously attempt to undermine the whole basis of sampling. Do not think this hasn’t been noticed; it goes very badly for my assessment of your general credibility.

16. #16 Kevin P.

    October 13, 2006

    Posted by: Seixon

    IBC has a basis of at least 45,000 recorded deaths. The JHU study has around 600? I think there comes a point where scientists believe a little bit too much in their methods and it becomes dogma instead of science.

    This reminds me of a long time ago when Tim Lambert and I clashed over some stupid professor’s study about how, in the US, criminals know which houses have guns and can target them, thus proving that legally owned guns in houses have a greater risk of being stolen than being used for self defense.

    I pointed out that this did not meet the reality test, because:
    Registration of ordinary guns is forbidden at the federal level and is rare at the local and state level.
    When there is registration, the records are almost always private and open to only criminal investigations and court orders.
    Decades of demonization of guns have made gun owners unlikely to volunteer gun ownership to strangers.
    There is no visible way to distinguish between a house with a gun and without a gun.
    There is simply No Frigging Way that criminals have some means of knowing which house has a gun.

    I don’t think Tim every fully accepted this. As far as he is concerned, since the study followed an accepted statistical method, the result must be true.

    I am a chemical engineer by profession and have performed my share of statistically designed experiments and correlation studies in a semiconductor chip factory. Unlike the JHU authors, my job was on the line if my statistics came up with an unrealistic result.

17. #17 Kevin P.

    October 13, 2006

    Clayton Cramer has another reality checkpoint.

18. #18 Kevin P.

    October 13, 2006

    dsquared wrote:

    …which obviously attempt to undermine the whole basis of sampling. Do not think this hasn’t been noticed; it goes very badly for my assessment of your general credibility.

    Sampling is a tool. Like any other tool, it can be used properly, or used inappropriately, or misused. There is a reason why censuses tend to be actual enumerations of headcount rather than selective samples.

    Use the tool, don’t become a tool. Your worship of sampling is near religious in nature.

19. #19 seriously

    October 13, 2006

    “In fact, that last sentence is an affront to the current Iraqi government and the Iraqi people as a whole.”

    This is getting silly.

20. #20 Kevin P.

    October 13, 2006

    Tim Lambert:

    [Omar]‘s obviously boiling mad because of the all bloodshed, but shouldn’t his anger be directed at those responsible rather than at those who have merely reported what is going on?

    Tim, who do you think is responsible for the bloodshed?

21. #21 Jeff Harvey

    October 13, 2006

    I can’t help but love the denialists (those who swallow the ‘American creed hook, line and sinker) coming out of their various caskets to lambast the JHU study. Armchair experts, all. The Lancet paper must have been rigorously peer-reviewed, probably by 5 senior researchers in the field. Moreover, Colin Powell cited results from the JHU research team when it estimated the death toll in the Congo using exactly the same methodology as used in Iraq. Where the hell were Seixon, Kevin P., Tim Blair, Jack Loacton and their empire apologists then? I am sure that they said nary a word about the validity of the Congo research. Why? Because it didn’t challenge the ridiculous notion of America’s benevolence, it didn’t question the outrageous belief that the US government supports human rights, freedom and democracy in its fregin policy. And it covered the deaths of what those with power and priviledge consider to be as an ‘unpeople’, to quote an appropriate term from British historian Mark Curtis. The US historian Stuart Creighton Miller once wrote that America is ‘that mst peace-loving of nations’, presumably with his tongue firmly planted in his cheek.

    The Iraq debacle and the death toll there would be far more questionable if there weren’t many historical precedents. The 1902-02 Phillipines conquest which left over half a million dead. The ‘heathens war’ against Korea in 1870. Many examples in Latin America, which the US has considered a ‘plantation’ since the advent of the Monroe Doctrine. The second Korean War. Viet Nam. The first Iraq war, in whihc the civolian infrastructure was deliberately targeted, followed by 12 years of sanctions that resembled a medievel siege. Heck, combine these three in Iraq and there are probably 1.5 million dead.

    Decorated US General Smedley Butler said in 1934: “I spent thirty- three years and four months in active military service as a member of this country’s most agile military force, the Marine Corps. I served in all commissioned ranks from Second Lieutenant to Major-General. And during that period, I spent most of my time being a high class muscle- man for Big Business, for Wall Street and for the Bankers. In short, I was a racketeer, a gangster for capitalism. I suspected I was just part of a racket at the time. Now I am sure of it. Like all the members of the military profession, I never had a thought of my own until I left the service. My mental faculties remained in suspended animation while I obeyed the orders of higher-ups. This is typical with everyone in the military service.

    I helped make Mexico, especially Tampico, safe for American oil interests in 1914. I helped make Haiti and Cuba a decent place for the National City Bank boys to collect revenues in. I helped in the raping of half a dozen Central American republics for the benefits of Wall Street. The record of racketeering is long. I helped purify Nicaragua for the international banking house of Brown Brothers in 1909-1912 (where have I heard that name before?). I brought light to the Dominican Republic for American sugar interests in 1916. In China I helped to see to it that Standard Oil went its way unmolested.

    During those years, I had, as the boys in the back room would say, a swell racket. Looking back on it, I feel that I could have given Al Capone a few hints. The best he could do was to operate his racket in three districts. I operated on three continents”. Not surprisingly, Butler was ‘marginalized’ from society once he started making his views clear.

    Before the denialists accuse me of fervent anti-Americanism, I want to ask them this: what specifically is being ‘anti-American’? Does it mean I hate jazz or blues music? Yosemite National Park? The Chicago Bulls? Giant Sequioa trees? I lived in the US for some years, and I love the country and I made quite few good friends there. I have traveled coast to coast and think its one of the most beuatiful nationsd on Earth. But like me, my America friends hate what is being done in their name. Mickey Z, a New York based writer made this point forcefully recently. Under the title of “Why I hate America”, he went on to say how much he loved his country, but hated what its governments do around the world in the name of empire and corporate expansionism. Its all about being accountable. We all know what monstrous regimes existed under communism. I am certanly aware of this, having visited Czechoslovakia when it was under communist rule. It was a horrible place at the time. But the US is supposed to expound and to defend ideal and compassionate values. The evidence is clear in foregin policy that it has become a monster, a rogue state which rotuinely supports (or carries out) mass murder, torture and commits barbarous acts. Anyone with a shred of humanity cannot defend this.

22. #22 Kevin P.

    October 13, 2006

    Yawn, Jeff, you anti-Americans are all so predictable.

    A question: are you British? I have noticed that your screed is what seems to pass for debate among the UK left; breath-taking generalizations and slanders, notably unsupported by evidence or reason, like:

    denialists
    those who swallow the ‘American creed hook, line and sinker
    empire apologists
    the ridiculous notion of America’s benevolence
    the outrageous belief that the US government supports human rights, freedom and democracy in its fregin policy.
    followed by 12 years of sanctions that resembled a medievel siege
    it has become a monster, a rogue state which rotuinely supports (or carries out) mass murder, torture and commits barbarous acts.

    Take a chill pill, dude. It’s a good thing that we barbarian Americans didn’t stay at home, instead of doing things like this, else you’d likely speak German now.

23. #23 Kevin P.

    October 13, 2006

    Paul, I checked out your buddy Mickey Z’s blog:

    I believe it would not only be accurate, but also extremely relevant to start referring to the white supremacist, capitalist patriarchy currently known as the United States of America as “the occupied territories.”

24. #24 DrSteve

    October 13, 2006

    Tim Lambert: Just to confirm, it’s your view that spatial correlation in deaths is just the kind of normal within-cluster association that the analysts already “pay the price for” with larger standard errors? That strikes me as completely reasonable, of course, but I’m still asking Dr. Burnham for all his materials.

25. #25 Jeff Harvey

    October 13, 2006

    Kevin P,

    There’s a lot of Americans who dont’ agree with you. Most of the fervent critics of US policy I read from are Americans. Many of them. Certainly as American as you are. Smearing critics of US foreign policy as being ‘anti-American is bullshit, and you know it. It is the last refuge of a scoundrel, this kind of tactic.

    Also, dude, learn some history. Question: which country during WWII destroyed most of the German panzer divisions? A: it wasnt’ the USA. Not even close. Moreover, read some of the historical files of planning documents written by the Counmcil on Foreign Relations (ever heard of them, ‘dude’?). You’ll see that US entry into the war wasn’t based so much on restoring freedom to Europe as you might think. Ever hear of the “Grand Area Strategy”? I didn’t think so. Methinks its time you learned a little bit about your own history, instead of depending upon we Eurpeans to do it for you.

26. #26 Tim Lambert

    October 13, 2006

    Yes. The higher the spatial correlation, the larger the design effect.

27. #27 DrSteve

    October 13, 2006

    Thanks for the prompt response. Note also that one of the things I’ll be looking at in the data is whether there even is spatial correlation, to the extent that can be determined (I suspect that for the same reason as for identifiers, much of the locational data wasn’t recorded). But I can see what percentages of households in each cluster reported a death of a certain type…

28. #28 Robert

    October 13, 2006

    Ragout rightly observed:

    In other words, the authors devote most of their methods section to discussing a particular statistical model, and then never report the results of that model!

    Yeah, that struck me, too. The models they were talking about in that section are a proportional hazards model and what sounds like a proportional hazards ME model, which are used to look at the effects of covariates on the rates. The main summary finding of the total number of excess deaths isn’t dependent on either of those: you can calculate that from the summary rates. So it sounds like the more detailed covariate results have been done (as they were in the 2004 paper) but they decided that no one paid much attention to them in the first paper so they’ll eventually show up later in some other paper.

29. #29 Barry

    October 13, 2006

    Steve, you’ve raised the issue numerous times; Tim has pointed out how the authors addressed it. You were obviously hoping that the term ‘spatial correlation’ would frighten people away. You’re wrong.

30. #30 DrSteve

    October 13, 2006

    Barry, with all due respect, you don’t have the first clue about my motivations. I never made this personal, I have no quarrel with anyone here, and I don’t see what you have to gain by attributing some particular intent to me.

    I’m happy to learn anything any day, even if the lesson is that my intuition was wrong.

    Your piling on doesn’t accelerate that process. I’m sorry, I’m just not that into you.

31. #31 JB

    October 13, 2006

    Kevin P.: “It’s a good thing that we barbarian Americans didn’t stay at home, instead of doing things like this, else you’d likely speak German now.”

    We Americans?

    I see. If you are taking persoanl credit for America’s role in winning WWII, I assume you must be 80 or older, right?

    And in case you may have not noticed, now that the facts have come out — 2759 dead US troops, 20,687 wounded US troops (many with brain damage and gross disfigurement), tens (if not hundreds) of thousands of dead Iraqi civilans, no WMD in Iraq, no collaboration (indeed outright animosity) between Saddam and al Qaida, gross mismanagement of post-invasion Iraq, corruption and war profiteering on a mammoth scale — the vast majority of Americans now believe invading Iraq was a mistake (The later group aparently includes our CIA analysts, whose recently released NIE conlcudes that the war in Iraq has become a “cause celebre for jihadists”).

    So, Kevin P., don’t try to include the rest of us Americans in your little neoconservative cluster fuck when it comes to cheerleading for Dubya’s disastrous (mis)Adventure in Blunderland, OK?

32. #32 nik

    October 13, 2006

    DrSteve;

    As I understand it the method – in theory – works like this. You collect data on the change in death rates of a number of clusters. This can be viewed as a sample from the population of all possible clusters. So you can use the data from the sample to make an inference about the change in death rates from the population of all possible clusters – which has a physical analogue in terms of the change in death rate across Iraq.

    The units in the regression used to make the inference are the clusters. And the data from clusters are iid by definition, so spatial effects are designed out of the study. It’s the ‘clever’ step of using clusters as sampling units which does this. If the inference was made from households you wouldn’t have a random sample (of households), and so you would have to worry about spatial autocorrelation, as you would if cluster locations weren’t random. But because the inference is made from clusters which are iid instead – you have a random sample – and spatial correlation isn’t important.

    At least that’s how I think it works. I may be talking rubbish, in which case correct me.

    P.S. In practice they used stratified sampling to assign clusters to governorates (I’m not sure why, or how they justify this, anyone?). And the random part of their sampling obviously isn’t very good. But that doesn’t change the theory.

33. #33 DrSteve

    October 13, 2006

    nik, first, thanks for a friendly comment.

    I understand how cluster sampling works; I work with clustered data all the time. My problem was I got tied up in the phenomenology rather than just thinking about how the data look. The fact is that having a strong association among reported deaths in a cluster representing a small geographic area (which, as I noted, I’m not even sure is the case!) is really no different from having a strong association in educational attainment among children in a cluster representing one household. Shared traits are shared traits, and there’s no need for different types of corrections based on the nature of what generates them.

    And to revisit Barry’s little poke at my character, I have a lot of respect for the people who post and comment at scienceblogs, and I hardly think I could scare anyone here away by asking a question about (wait for it!) spatial correlation.

    Does anyone know a good source for data on within-county migration in similar conflicts? I’m wondering whether it’s possible to do an order-of-magnitude assessment on Dr. Burnham’s point about movement from high-mortality to low-mortality areas.

34. #34 Kevin P.

    October 13, 2006

    Jeff Harvey:

    Smearing critics of US foreign policy as being ‘anti-American is bullshit, and you know it. It is the last refuge of a scoundrel, this kind of tactic.

    Jeff, I call them as I see them. Let me remind you of your own words:

    denialists
    those who swallow the ‘American creed hook, line and sinker
    empire apologists
    the ridiculous notion of America’s benevolence
    the outrageous belief that the US government supports human rights, freedom and democracy in its fregin policy.
    followed by 12 years of sanctions that resembled a medievel siege
    it has become a monster, a rogue state which rotuinely supports (or carries out) mass murder, torture and commits barbarous acts.

    I call the above as anti-American rhetoric and I guarantee you that the vast majority of Americans, including most liberals (but not most leftists) would call it the same way.

    Out of morbid curiosity, can you provide some example of rhetoric about American foreign policy that you would consider anti-American?

35. #35 gwangung

    October 13, 2006

    I call the above as anti-American rhetoric and I guarantee you that the vast majority of Americans, including most liberals (but not most leftists) would call it the same way.

    I wouldn’t.

    I’d say it was the exaggeration of a bitterly disappointed idealist, but not necessarily anti-American. Nor ncessarily correct, either.

36. #36 Scott Lemieux

    October 13, 2006

    Shorter sexion: the 1936 Reader’s Digest presidential survey was far more accurate than modern sampling, because it asked so many more people!

37. #37 Ian Gould

    October 14, 2006

    Scott – and the 1948 phon surveys which showed Dewey winning were obviously much more reliable the actual count which showed Truman ahead.

    That result was so absurd on the face of it that the survey methodology (I think the Commie Hippie Fag traitors of the liberal media called “an election”) was obviously flawed.