It seems that war supporters with actual knowledge of statistics aren't willing to criticise the new Lancet study, leaving the field to folks who don't know what they are talking about.
Well, I don't believe that John Hopkins research, I don't. It's not plausible, it's not based on anything other than a house-to-house survey. I think that's absolutely precarious.
It is a ... an unbelievably large number and it's out of whack with most of the other assessments that have been made.
Surveys are the best way to measure these things. The other assessments that are lower such as IBC are based on media reports. There aren't reporters everywhere or even in most places in Iraq.
Human flesh is abundant and all they have to do is call this hospital or that office to get the count of casualties, even more they can knock on doors and ask us one by one and we would answer because we've got nothing to be ashamed of.
They did knock on doors and ask one by one.
They shamelessly made an auction of our blood, and it didn't make a difference if the blood was shed by a bomb or a bullet or a heart attack because the bigger the count the more useful it becomes to attack this or that policy in a political race and the more useful it becomes in cheerleading for murderous tyrannical regimes.
They did distinguish between the different ways people died as well reporting the total. I think it is better to know the truth even if it does hurt Omar politically.
When the statistics announced by hospitals and military here, or even by the UN, did not satisfy their lust for more deaths, they resorted to mathematics to get a fake number that satisfies their sadistic urges.
Given Omar's previous statement about the political effect of high death numbers, there is good reason to expect folks in hospitals and military bases to report much lower numbers than what really happened. So if you want an accurate figure you have to go out and knock on doors as Omar suggested earlier in his post. Omar does not provide any evidence that the number is "fake" other than "they resorted to mathematics".
When I read the report I can only feel apathy and inhumanity from those who did the count towards the victims and towards our suffering as a whole. I can tell they were so pleased when the equations their twisted minds designed led to those numbers and nothing can convince me that they did their so called research out of compassion or care.
Omar continues in this vein, making vicious and unfounded attacks on the people who conducted the study. He's obviously boiling mad because of the all bloodshed, but shouldn't his anger be directed at those responsible rather than at those who have merely reported what is going on?
Next we have Seixon. Seixon claimed that the first Lancet study was incorrect because he claimed that the sampling was biased. Well the new study does the sampling the way Seixon said that the first one should have, so I would have thought that he would accept the new study but I am surprised to find that he has not:
it seems that the Johns Hopkins has largely rectified all the problems they had with their previous survey which I summarized last year. Yet even so, the result of this study is simply absurd, which gets us back to what Bush said. Does a cluster sample provide reliable results when used to estimate mortality in a war zone?
If it's done correctly, yes.
Seixon asks:
Using the findings of the study just released, can you extrapolate how many death certificates were published in the post-war period?
About 900,000.
How do you think that will match up with Iraqi government records?
Government figures will be much lower. As noted in the appendix:
Even with the death certificate system, only about one-third of deaths were captured by the government's surveillance system in the years before the current war, according to informed sources in Iraq. At a death rate of 5/1000/year, in a population of 24 million, the government should have reported 120,000 deaths annually. In 2002, the government documented less than 40,000 from all sources. The ministry's numbers are not likely to be more complete or accurate today.
And if you look at Omar's comment above, there seems to be strong incentives for the government to understate the numbers.
Next, Tim Blair:
It is a larger number than were killed in Germany during five years (and 955,044 tons) of WWII bombing
Since only a small fraction of the deaths were caused by bombing, this is hardly relevant. If you want a WWII comparison: Yugoslavia suffered 1.3 million civilian deaths from a population much smaller than Iraq's is today.
Blair also offers this:
Remember: Lancet came up with this via a survey that identified precisely 547 deaths (as reported by the New York Times).
Yeah, that's how random sampling works. Apparently he's willing to reject the entire field of statistics if it comes up with a result he doesn't like. Glenn Reynold endorsed Blair's argument.
Kieran Healy has more comments on Lancet denial, while lenin, Tom Bozzo and
Nexus 6 also play whack-a-mole with Lancet critics.
well I'm awaiting a "LancetAudit.org" site to be created.
Seixon asked for a death certificates "recount". I believe it should be possible to verify at some morgues how many certificates were issued.
The data at the central level might be manipulated or incomplete, but one should be able to make a survey of morgues and investigate what evidence there is. Are there lists and registers of issued certificates? How much are certificate stocks being "used up"?
This data could then be corrected by the ratio of total/certificated deaths from the Lancet study.
Tim, I'd honestly be very interested to know what you think about how the methods employed by Dr. Burnham and his colleagues would handle spatial correlation.
Following on Oskar's comment, one of the most interesting aspects of this new study is the assertion that 90% of the deaths were documented by death certificates.
Shocked as I was by the large number of deaths extrapolated, my first impulse was to wonder if the figure of 90% was genuine, since the death certificates lend the study their authority. Assuming it is genuine raises another question, though: If death certificates are so closely correlated with actual deaths, why risk life and limb wandering the countryside to take a small sample when you can visit the hospitals that issue death certificates and make a precise accounting? (The investigators needn't rely on the central government.) The number of visits would be no greater, and the numbers would be more accurate. Consider the value such a comprehensive study would have. We'd be able to see the rise and fall of deaths across time and by region. And the investigators would be able to look for manipulation based on whether the trends accorded with the news coming out of the area.
How much more than this study's $55k would that cost? I would be interested in whether the group behind this study ever considered such an approach.
Tim Lambert:
Incorrect. Source
I think the real question -- which those questioning the Lancet results are conveniently avoiding -- is "How many Iraqi civilian deaths are 'acceptable'."
30,000? (Bush's number)
60,000? (the number provided by the Iraqi government and based largely on Baghdad morgue deaths).
100,000? (The first Lancet study number for a shorter time period and based on surveys)
650,000 (The second Lancet study number for deaths to date)
If you ask me, even 30,000 deaths is a very large price to pay if you happen to be in one of the affected families.
But I'm not the one to ask.
Ask those who are now questioning the Lancet number (or the Iraqi ministry number, or even the Bush number) what they think.
I suspect that the later group (reprsneted by some of the commenst on this blog) do not give a damn how many Iraqis have died since the war began, just as they don't give a damn that no WMD were found, or that Saddam was not in cahoots with Osam bin Laden, or that many brutal dictators have been deposed quite bloodlessly over the past few decades.
Can we stop saying the 650K number refers to *civilian* deaths? The Lancet does not make that claim.
Clazy wondered:
Because even if those records are complete, it doesn't give you the population at risk (i.e., the denominator that you need to calculate the death rate). At very best, that only gives you the numerator.
>Because even if those records are complete, it doesn't give you the population at risk (i.e., the denominator that you need to calculate the death rate). At very best, that only gives you the numerator.
This took me a minute to get.
What you are saying is that you could not simply sample morgues and hospitals - because so long as you failed to hit every morgue or hospital you still be doing sampling and would need general population to compare it against - something you can't directly get from hospitals or morgues. To make sure each death represents a death per certain number of live Iraq's you have to hit households.
An alternative is that someone could try to muster the resources to visit every morgue, hospital, etc. in Iraq.Orders have been already been given that these are now barred from giving out figures to reporter, cause nosey journalists kept putting together figures showing the official figures were nonsene. I'll bet if a full independent mortaltity census was begun in Iraq, it would quickly be extended to researchers.
I think Richard Horton, editor of the Lancet, has given the best explanation for what the study means:
http://www.guardian.co.uk/frontpage/story/0,,1920167,00.html
"Why is this Lancet estimate so much higher than the figures put out by President Bush or the Iraq Body Count website? They put the number of casualties in the tens of thousands, not the hundreds of thousands. To be fair, Iraq Body Count does not claim to publish accurate absolute numbers of deaths. Instead, their figures are valuable for measuring trends. But the reason for the discrepancy between these lower estimates and the new figure of 650,000 deaths lies in the way the number is sought. Passive surveillance, the most common method used to estimate numbers of civilian deaths, will always underestimate the total number of casualties. We know this from past wars and conflict zones, where the estimates have been too low by a factor of 10 or even 20.
"Only when you go out and knock on the doors of families, actively looking for deaths, do you begin to get close to the right number. This method is now tried and tested. It has been the basis for mortality estimates in war zones such as Darfur and the Congo. Interestingly, when we report figures from these countries politicians do not challenge them. They frown, nod their heads and agree that the situation is grave and intolerable. The international community must act, they say. When it comes to Iraq the story is different. Expect the current government to mobilise all its efforts to undermine the work done by this American and Iraqi team. Expect the government to criticise the Lancet for being too political. Expect the government to do all it can to dismiss this story and wash its hands of its responsibility to take these latest findings seriously."
DrSteve: spatial correlation means that in a cluster survey you have to survey more people than simple random sampling to get the same standard error for your estimate. They took this into account when they decided how many people to survey and when they calculated the confidence intervals.
it is time to call for volunteers from the 101st fightin' keyboarders to do the hard slog of verifying the figures at iraq's hospitals.
http://www.cbsnews.com/stories/2006/10/04/eveningnews/main2064668.shtml
In their methods section of their paper, the Lancet authors discuss at length a "log-linear regression model." Tim Lambert says (of the methods section) "And if you don't understand that paragraph you probably shouldn't be criticising the study's methodology."
Perhaps Tim can point me to where in the paper the results of the log-linear model appear? These results should take the form of relative risks, I believe. But unlike their previous study, I don't see any relative risks reported.
In other words, the authors devote most of their methods section to discussing a particular statistical model, and then never report the results of that model! I don't mean to suggest that anything nefarious is going on (I'm a little suspicious, but only a little). But I do think that this tells you that the peer reviewers and editors didn't examine the paper's statistical methods all that carefully.
Tim,
You quote the study saying:
"Even with the death certificate system, only about one-third of deaths were captured by the government's surveillance system in the years before the current war, according to informed sources in Iraq. At a death rate of 5/1000/year, in a population of 24 million, the government should have reported 120,000 deaths annually. In 2002, the government documented less than 40,000 from all sources. The ministry's numbers are not likely to be more complete or accurate today."
Riddle me this:
1. Why do the practices of Saddam Hussein's government have any relevance to the current state of affairs in Iraq?
2. Who are these "informed sources" and why should we believe them, and back to #1, why does it matter what they know about how Saddam Hussein did his thing?
3. Is that last sentence supposed to be coming from a scientist? Sounds like yet another convenient assumption on the part of the JHU folks.
In fact, that last sentence is an affront to the current Iraqi government and the Iraqi people as a whole. Also notice that their only reason for not checking morgues and such is that "there is a suspicion" that things are being hidden because of political reasons. Again I ask, upon which scientifical basis is this being said? None. It's the political bias of the authors being inserted into the study to make excuses for why they didn't do something that would have given more trustworthy and reliable results.
If they had checked with the government records and the morgues, they could at least have established a minimum number of excess deaths. It's quite apparent that that's not what they were interested in, however.
I also find it hilarious that they think you get a more accurate number by interviewing 1800 families and extrapolating from there instead of by actually counting reported deaths.
IBC has a basis of at least 45,000 recorded deaths. The JHU study has around 600? I think there comes a point where scientists believe a little bit too much in their methods and it becomes dogma instead of science.
Seixon, the last time round you were very keen on making detailed comments on statistical methodology. This time, you are writing things like:
[I also find it hilarious that they think you get a more accurate number by interviewing 1800 families and extrapolating from there instead of by actually counting reported deaths.]
which obviously attempt to undermine the whole basis of sampling. Do not think this hasn't been noticed; it goes very badly for my assessment of your general credibility.
Posted by: Seixon
This reminds me of a long time ago when Tim Lambert and I clashed over some stupid professor's study about how, in the US, criminals know which houses have guns and can target them, thus proving that legally owned guns in houses have a greater risk of being stolen than being used for self defense.
I pointed out that this did not meet the reality test, because:
Registration of ordinary guns is forbidden at the federal level and is rare at the local and state level.
When there is registration, the records are almost always private and open to only criminal investigations and court orders.
Decades of demonization of guns have made gun owners unlikely to volunteer gun ownership to strangers.
There is no visible way to distinguish between a house with a gun and without a gun.
There is simply No Frigging Way that criminals have some means of knowing which house has a gun.
I don't think Tim every fully accepted this. As far as he is concerned, since the study followed an accepted statistical method, the result must be true.
I am a chemical engineer by profession and have performed my share of statistically designed experiments and correlation studies in a semiconductor chip factory. Unlike the JHU authors, my job was on the line if my statistics came up with an unrealistic result.
Clayton Cramer has another reality checkpoint.
dsquared wrote:
Sampling is a tool. Like any other tool, it can be used properly, or used inappropriately, or misused. There is a reason why censuses tend to be actual enumerations of headcount rather than selective samples.
Use the tool, don't become a tool. Your worship of sampling is near religious in nature.
"In fact, that last sentence is an affront to the current Iraqi government and the Iraqi people as a whole."
This is getting silly.
Tim Lambert:
Tim, who do you think is responsible for the bloodshed?
I can't help but love the denialists (those who swallow the 'American creed hook, line and sinker) coming out of their various caskets to lambast the JHU study. Armchair experts, all. The Lancet paper must have been rigorously peer-reviewed, probably by 5 senior researchers in the field. Moreover, Colin Powell cited results from the JHU research team when it estimated the death toll in the Congo using exactly the same methodology as used in Iraq. Where the hell were Seixon, Kevin P., Tim Blair, Jack Loacton and their empire apologists then? I am sure that they said nary a word about the validity of the Congo research. Why? Because it didn't challenge the ridiculous notion of America's benevolence, it didn't question the outrageous belief that the US government supports human rights, freedom and democracy in its fregin policy. And it covered the deaths of what those with power and priviledge consider to be as an 'unpeople', to quote an appropriate term from British historian Mark Curtis. The US historian Stuart Creighton Miller once wrote that America is 'that mst peace-loving of nations', presumably with his tongue firmly planted in his cheek.
The Iraq debacle and the death toll there would be far more questionable if there weren't many historical precedents. The 1902-02 Phillipines conquest which left over half a million dead. The 'heathens war' against Korea in 1870. Many examples in Latin America, which the US has considered a 'plantation' since the advent of the Monroe Doctrine. The second Korean War. Viet Nam. The first Iraq war, in whihc the civolian infrastructure was deliberately targeted, followed by 12 years of sanctions that resembled a medievel siege. Heck, combine these three in Iraq and there are probably 1.5 million dead.
Decorated US General Smedley Butler said in 1934: "I spent thirty- three years and four months in active military service as a member of this country's most agile military force, the Marine Corps. I served in all commissioned ranks from Second Lieutenant to Major-General. And during that period, I spent most of my time being a high class muscle- man for Big Business, for Wall Street and for the Bankers. In short, I was a racketeer, a gangster for capitalism. I suspected I was just part of a racket at the time. Now I am sure of it. Like all the members of the military profession, I never had a thought of my own until I left the service. My mental faculties remained in suspended animation while I obeyed the orders of higher-ups. This is typical with everyone in the military service.
I helped make Mexico, especially Tampico, safe for American oil interests in 1914. I helped make Haiti and Cuba a decent place for the National City Bank boys to collect revenues in. I helped in the raping of half a dozen Central American republics for the benefits of Wall Street. The record of racketeering is long. I helped purify Nicaragua for the international banking house of Brown Brothers in 1909-1912 (where have I heard that name before?). I brought light to the Dominican Republic for American sugar interests in 1916. In China I helped to see to it that Standard Oil went its way unmolested.
During those years, I had, as the boys in the back room would say, a swell racket. Looking back on it, I feel that I could have given Al Capone a few hints. The best he could do was to operate his racket in three districts. I operated on three continents". Not surprisingly, Butler was 'marginalized' from society once he started making his views clear.
Before the denialists accuse me of fervent anti-Americanism, I want to ask them this: what specifically is being 'anti-American'? Does it mean I hate jazz or blues music? Yosemite National Park? The Chicago Bulls? Giant Sequioa trees? I lived in the US for some years, and I love the country and I made quite few good friends there. I have traveled coast to coast and think its one of the most beuatiful nationsd on Earth. But like me, my America friends hate what is being done in their name. Mickey Z, a New York based writer made this point forcefully recently. Under the title of "Why I hate America", he went on to say how much he loved his country, but hated what its governments do around the world in the name of empire and corporate expansionism. Its all about being accountable. We all know what monstrous regimes existed under communism. I am certanly aware of this, having visited Czechoslovakia when it was under communist rule. It was a horrible place at the time. But the US is supposed to expound and to defend ideal and compassionate values. The evidence is clear in foregin policy that it has become a monster, a rogue state which rotuinely supports (or carries out) mass murder, torture and commits barbarous acts. Anyone with a shred of humanity cannot defend this.
Yawn, Jeff, you anti-Americans are all so predictable.
A question: are you British? I have noticed that your screed is what seems to pass for debate among the UK left; breath-taking generalizations and slanders, notably unsupported by evidence or reason, like:
Take a chill pill, dude. It's a good thing that we barbarian Americans didn't stay at home, instead of doing things like this, else you'd likely speak German now.
Paul, I checked out your buddy Mickey Z's blog:
Tim Lambert: Just to confirm, it's your view that spatial correlation in deaths is just the kind of normal within-cluster association that the analysts already "pay the price for" with larger standard errors? That strikes me as completely reasonable, of course, but I'm still asking Dr. Burnham for all his materials.
Kevin P,
There's a lot of Americans who dont' agree with you. Most of the fervent critics of US policy I read from are Americans. Many of them. Certainly as American as you are. Smearing critics of US foreign policy as being 'anti-American is bullshit, and you know it. It is the last refuge of a scoundrel, this kind of tactic.
Also, dude, learn some history. Question: which country during WWII destroyed most of the German panzer divisions? A: it wasnt' the USA. Not even close. Moreover, read some of the historical files of planning documents written by the Counmcil on Foreign Relations (ever heard of them, 'dude'?). You'll see that US entry into the war wasn't based so much on restoring freedom to Europe as you might think. Ever hear of the "Grand Area Strategy"? I didn't think so. Methinks its time you learned a little bit about your own history, instead of depending upon we Eurpeans to do it for you.
Yes. The higher the spatial correlation, the larger the design effect.
Thanks for the prompt response. Note also that one of the things I'll be looking at in the data is whether there even is spatial correlation, to the extent that can be determined (I suspect that for the same reason as for identifiers, much of the locational data wasn't recorded). But I can see what percentages of households in each cluster reported a death of a certain type...
Ragout rightly observed:
Yeah, that struck me, too. The models they were talking about in that section are a proportional hazards model and what sounds like a proportional hazards ME model, which are used to look at the effects of covariates on the rates. The main summary finding of the total number of excess deaths isn't dependent on either of those: you can calculate that from the summary rates. So it sounds like the more detailed covariate results have been done (as they were in the 2004 paper) but they decided that no one paid much attention to them in the first paper so they'll eventually show up later in some other paper.
Steve, you've raised the issue numerous times; Tim has pointed out how the authors addressed it. You were obviously hoping that the term 'spatial correlation' would frighten people away. You're wrong.
Barry, with all due respect, you don't have the first clue about my motivations. I never made this personal, I have no quarrel with anyone here, and I don't see what you have to gain by attributing some particular intent to me.
I'm happy to learn anything any day, even if the lesson is that my intuition was wrong.
Your piling on doesn't accelerate that process. I'm sorry, I'm just not that into you.
Kevin P.: "It's a good thing that we barbarian Americans didn't stay at home, instead of doing things like this, else you'd likely speak German now."
We Americans?
I see. If you are taking persoanl credit for America's role in winning WWII, I assume you must be 80 or older, right?
And in case you may have not noticed, now that the facts have come out -- 2759 dead US troops, 20,687 wounded US troops (many with brain damage and gross disfigurement), tens (if not hundreds) of thousands of dead Iraqi civilans, no WMD in Iraq, no collaboration (indeed outright animosity) between Saddam and al Qaida, gross mismanagement of post-invasion Iraq, corruption and war profiteering on a mammoth scale -- the vast majority of Americans now believe invading Iraq was a mistake (The later group aparently includes our CIA analysts, whose recently released NIE conlcudes that the war in Iraq has become a "cause celebre for jihadists").
So, Kevin P., don't try to include the rest of us Americans in your little neoconservative cluster fuck when it comes to cheerleading for Dubya's disastrous (mis)Adventure in Blunderland, OK?
DrSteve;
As I understand it the method - in theory - works like this. You collect data on the change in death rates of a number of clusters. This can be viewed as a sample from the population of all possible clusters. So you can use the data from the sample to make an inference about the change in death rates from the population of all possible clusters - which has a physical analogue in terms of the change in death rate across Iraq.
The units in the regression used to make the inference are the clusters. And the data from clusters are iid by definition, so spatial effects are designed out of the study. It's the 'clever' step of using clusters as sampling units which does this. If the inference was made from households you wouldn't have a random sample (of households), and so you would have to worry about spatial autocorrelation, as you would if cluster locations weren't random. But because the inference is made from clusters which are iid instead - you have a random sample - and spatial correlation isn't important.
At least that's how I think it works. I may be talking rubbish, in which case correct me.
P.S. In practice they used stratified sampling to assign clusters to governorates (I'm not sure why, or how they justify this, anyone?). And the random part of their sampling obviously isn't very good. But that doesn't change the theory.
nik, first, thanks for a friendly comment.
I understand how cluster sampling works; I work with clustered data all the time. My problem was I got tied up in the phenomenology rather than just thinking about how the data look. The fact is that having a strong association among reported deaths in a cluster representing a small geographic area (which, as I noted, I'm not even sure is the case!) is really no different from having a strong association in educational attainment among children in a cluster representing one household. Shared traits are shared traits, and there's no need for different types of corrections based on the nature of what generates them.
And to revisit Barry's little poke at my character, I have a lot of respect for the people who post and comment at scienceblogs, and I hardly think I could scare anyone here away by asking a question about (wait for it!) spatial correlation.
Does anyone know a good source for data on within-county migration in similar conflicts? I'm wondering whether it's possible to do an order-of-magnitude assessment on Dr. Burnham's point about movement from high-mortality to low-mortality areas.
Jeff Harvey:
Jeff, I call them as I see them. Let me remind you of your own words:
I call the above as anti-American rhetoric and I guarantee you that the vast majority of Americans, including most liberals (but not most leftists) would call it the same way.
Out of morbid curiosity, can you provide some example of rhetoric about American foreign policy that you would consider anti-American?
I call the above as anti-American rhetoric and I guarantee you that the vast majority of Americans, including most liberals (but not most leftists) would call it the same way.
I wouldn't.
I'd say it was the exaggeration of a bitterly disappointed idealist, but not necessarily anti-American. Nor ncessarily correct, either.
Shorter sexion: the 1936 Reader's Digest presidential survey was far more accurate than modern sampling, because it asked so many more people!
Scott - and the 1948 phon surveys which showed Dewey winning were obviously much more reliable the actual count which showed Truman ahead.
That result was so absurd on the face of it that the survey methodology (I think the Commie Hippie Fag traitors of the liberal media called "an election") was obviously flawed.