Les Roberts replies to a shamelessly dishonest WSJ editorial:
Your editorial entitled, "The Lancet's political hit" regarding our study of Iraqi deaths was a unique blend of error and innuendo. For example, I was not opposed to removing Saddam; I was opposed to invading a country while the UN Secretary General was stating that it would violate the UN Charter. Your suggestion that our Iraqi colleague Riyadh Lafta was suspect because he recorded child mortality during his career is particularly ironic. He was one of few professors in the country that never joined the Baath Party. You further suggest that because some of the second round of survey funding came from the Soros Foundation (unknown to the authors until last month) the results are suspect. In my work in eight war zones over the past two decades, I have seen the Soros Foundation bring heat and water to the beleaguered people of Sarajevo, bring the internet to millions of people trapped in Eastern Europe, and help the victims of torture in Zimbabwe. Were those efforts devoid of merit? If the Wall Street Journal applied this logic to yourself and was unable to research or write on issues related to your advertising funding, your paper would become rather slim.
A certain number of Iraqis died because of the invasion. We reported the death rate went up 2.5 fold, the Iraqi government now claims that it only doubled. Either way, hundreds of thousands have died and downplaying that fact is a disservice to your readers.
If it weren't shamelessly dishonest, it wouldn't be a WSJ editorial ...
the editorial, as always, has the basic facts wrong.
but look at this:
Yet it was trumpeted by the political left because it fit a narrative that they wanted to believe. And it wasn't challenged by much of the press because it told them what they wanted to hear. The truth was irrelevant.
perhaps the best part of the article. you just need to correct a single word. so lets better make that "right". :)
Editorial looks pretty good to me. Roberts is a politicized scientist.
TCO, why don't you use your awesome trolling powers for good, and post over at Climate Audit?
Roberts is a politicized scientist.
Good for him.
from the editorial: "And it wasn't challenged by much of the press because it told them what they wanted to hear."
No, it wasn't reported by much of the press because it told them something they didn't want to hear. The WSJ, as usual, gets things totally backwards. If they're really in any doubt about this, why don't they just take a look through their own archives, and count how many times they've mentioned The Lancet studies other than to denigrate them. I suspect you could count the mentions on one hand.
Comment on the WSJ editorial: Neil Munro was pro-war and therefore politicized.
Partly as a result of the fuss surrounding the sale of the WSJ to a notorious and culturally destructive media mogul, the rep of the editorial page, which may have been known only to a narrow range of the population, is now known to all: truth and reason are not to be found on that page. So all that is left is to note the occasional event of WSJ coverage of an issue as a marker that the US establishment has just taken an hit and it stings. Well, well, it could not happen to a nicer bunch of guys.
Tim,
Since you are in touch with Les Roberts, could you ask him if he stands by his claim that data on the sex/age of household members was collected in L2? I bet that he won't. (If he does, I owe you/him/Deltoid an apology.) And, if he does, perhaps you could ask him to make that data available to, say, the researchers behind IFHS? If the sex/age distributions match up, that would be great confirmation for both surveys. If not, then at least one of the surveys is very wrong.
If we can't even figure out what data L2 collected (whether or not you or I can analyze it), we aren't going to make very much progress.
The one thing I'm not exactly sure about regarding the Burnham/Roberts study is how they arrived at the pre-war death rate estimate. Thus far it looks to me as though Iraqi Ministry of Health data were used to estimate that statistic, thus making "excess deaths" a meaningful concept. Is that right? I could well be wrong -- anyhow, if I am right and MOH data must be used, or if I'm wrong and another source or sources were used, I'd be interested to hear what you think of that methodology, Tim and others here. By the way, I'm not a hostile observer, but a former public health student in Roberts' and Burnham's classes a few years back.
David,
No. The question is perfectly clear ("record M/F and the age"), as is Roberts' statement. I'm not going to waste his time just so you can call him a liar yet again. The claim that they didn't record it is a fabrication. I suspect that you are the one who fabricated it, and Munro is just your dupe.
That's hardly helpful, Tim. You don't think that Roberts could have been mistaken? You want to rely on statement from months ago? You doubt that I have a pretty good idea of what is in the data?
But here is the key: If this were a "fabrication," then it would be the most flagrant mistake in Munro's piece. But Roberts does not even mention it in his letter to the National Journal. Why would that be? Why would Roberts fail to even mention the worst "fabrication" in the entire article?
My explanation is simple: There is no fabrication because this data was not collected. Roberts was mistaken, perhaps honestly. Garfield himself notes that Roberts did not supervise the fieldwork as much as he (Garfield) thinks he should have. If the data was collected, don't Roberts and Burnham have an affirmative obligation to correct the record?
But all I want you to do is ask Roberts. Or do you worry about what he will say?
David, since you're here, are you willing to admit that this NEJM paper makes your criticism of Lancet 1 pointless? After all, they are in close agreement on the violent death toll for the first 17 months--roughly 120 violent deaths per day. So we can now all agree that a figure of 60,000 violent deaths for the first 1 months is quite plausible, and you can just forget you said anything about that.
It depends on what you mean by "pointless," I guess.
I haven't looked closely, but is your comparison correct? L1 estimated about 180,000 (guessing off the top of my head) violent deaths for 2003-2004 (for all of Iraq, including Falluja). IFHS estimates the same fewer deaths for a time period twice as long. Only one of these estimates can be correct. My bet is on IFHS.
If L1 and IFHS were really in "close agreement on the violent death toll for the first 17 months," then that would count in L1's favor. But I am pretty sure they are not. Could you walk through your analysis more slowly? Don't forget that the Falluja cluster in L1 is responsible for about 200,000 excess deaths, most of the violent. Again, I might be wrong about the details. Show us your work and we can discuss.
I was excluding Fallujah. The outlier gives one a ridiculous mortality distribution, as Robert showed here. If you exclude Fallujah, the median estimate for the violent death rate is 60,000 over 17 months, which is roughly 120 deaths per day, 3 times the (corrected) IBC numbers for the same period. And IBC and the Lancet critics weren't willing to admit even that much, back then, but according to the new paper that's exactly the case.
I remember all this, having seen it discussed to death here and at a couple of other websites. I don't recall any prowar critic limiting himself to the claim that L1 was ridiculous if the Fallujah outlier were included, but gave a reasonable result if one left it out. Maybe you were saying this all along--then perhaps you could provide a link where you said "Yes, L1's midrange estimate excluding Fallujah is really quite plausible."
Of course your own pet theory is that the Fallujah outlier would, if added to the mix, make it more likely that the death rate actually decreased. But Robert settled that issue--it doesn't. It inflates the mean and gives a very weird looking probability distribution, but it doesn't do what you claim.
Donald Johnson, inspired by your post, I decided to see whether the results from L1 and NEJM would converge.
According to Table 4 from the Supplementary Materials (NEMJ):
Mortality Rate All causes Pre-Invasion = 3.17
Mortality Rate All causes for Mar03 to Dec04 = 5.92
The excess death for Mar03 to Sep04 period (=L1):
(5.92-3.17)/1000 * (17.8/12) * 24400000 = 99531
Did I make a huge mistake somewhere or do the results from this new NEMJ article and L1 match almost perfectly?
Disclaimer: I'm not an epidemiologist nor a statistician, I just used the [formula](http://scienceblogs.com/deltoid/2007/08/robert_chung_on_david_kane.php#…) that Robert used to show David Kane how it was possible to calculate the 98000 estimate excess deaths found by the L1 article.
well you ought to start David. We are all eagerly waiting to see you apply the same rigour to this paper which you applied to L1, in service of the task of proving L1 right using this new, improved paper which you seem to like so much.
No David, by your estimation the Fallujah cluster was responsible for several 10s of thousands of excess resurrections, remember?
For example, I was not opposed to removing Saddam; I was opposed to invading a country while the UN Secretary General was stating that it would violate the UN Charter.
Hahahaha. What a dishonest dissembler the guy is.
Your suggestion that our Iraqi colleague Riyadh Lafta was suspect because he recorded child mortality during his career is particularly ironic.
Les, you were invited to join the Ba'ath party, dude. Saddam and his murderous sycophants invited you to join the party. Your buddy was passing on dishonest crap that uranium tipped shells were killing kids. That comment was so dishonest it's laughable. Nice types you hang out with, Les.
He was one of few professors in the country that never joined the Baath Party.
Of course, Les, how silly of us, he was a Jeffersonian Democrat and started the libertarian party in Iraq. No one did "well" in Iraq if they weren't sycophants to the Ba' ath party. Stop making absurd, dishonest comments.
You further suggest that because some of the second round of survey funding came from the Soros Foundation (unknown to the authors until last month) the results are suspect. In my work in eight war zones over the past two decades, I have seen the Soros Foundation bring heat and water to the beleaguered people of Sarajevo, bring the internet to millions of people trapped in Eastern Europe, and help the victims of torture in Zimbabwe.
Oh yes, the old leftist routine of when in trouble wheel out the kid on a wheel chair with the broken leg. LOL.
If the Wall Street Journal applied this logic to yourself and was unable to research or write on issues related to your advertising funding, your paper would become rather slim.
Does anyone know what Les is saying here, or is he just filling up space?
A certain number of Iraqis died because of the invasion. We reported the death rate went up 2.5 fold, the Iraqi government now claims that it only doubled. Either way, hundreds of thousands have died and downplaying that fact is a disservice to your readers.
Nice to see that even les is beginning to see the absurdity of his dishonesty becoming apparent. Les is no longer talking about 1 million Iraqi's killed. It's now " hundreds of thousands".
The disservice Les ought to be focusing on is his sheer brazen dishonesty for which he ought to publicly apologize. His contribution to the war was nothing short of poisoning the well.
jc wrote: 'Les is no longer talking about 1 million Iraqi's killed. It's now " hundreds of thousands"'
The last Lancet Johns-Hopkins study estimated a mean total of deaths as 680,000, which is, ahem, close to
seven-hundred-thousand.
Is that math too difficult for you, jc? Or would you like a tutorial in counting past 10?
I knew that was the mean, Toby, you numbnut. His upper range was in the squizillions.... How appropriate just before the election. And let's not forget he had Dr. Mengels helping him " figure" out the numbers on the Iraqi side. You know the guy. It was that college professor who didn't join the ba'ath party. LOL.
If comparing Les Robert's Iraqi colleague to the infamous Nazi doctor who killed huge numbers of death camp inmates in a variety of extremely painful ways in the name of "science" doesn't get JC banned ...
Will anything? Ever?
JC is vile.
Dhgoza, sadly it seems Scienceblogs has a policy of never banning anyone ever under any circumstances.
If threats of physical violence and blatantly repostingt copyrighted articles in full don;t get him banned nothing will.
Just ignore Joe Cumbria and with luck he'll go back to whatever sad pathetic travesty of a real life he possesses.
Yes everyone please do not bait the idiot. This will become the Thread of Doom quickly enough as it is without becoming the Fucked-up Thread of Doom.
You're right, Hoggsie. Dr. Mengels is the wrong comparison. The Nazi propaganda minister the more appropriate-Dr. Goebbels. My mistake! Let's find out why shall we?
And who is this Iraqi Goebbels? Roberts calls him a college professor who wasn't a member of the Ba' ath party.
Yet we find out Riyadh "Goebbels" actually worked for the Iraqi health ministry ( yes i know it was a contradiction in terms). And what was he doing while there? Let's see shall we:
We also learn that the key person involved in collecting the Lancet data was Iraqi researcher Riyadh Lafta, who has failed to follow the customary scientific practice of making his data available for inspection by other researchers. Mr. Lafta had been an official in Saddam's ministry of health when the dictator was attempting to end international sanctions against Iraq. He wrote articles asserting that many Iraqis were dying from cancer and other diseases caused by spent U.S. uranium shells from the Gulf War. According to National Journal, the Lancet studies "of Iraqi war deaths rest on the data provided by Lafta, who operated with little American supervision and has rarely appeared in public or been interviewed about his role."
So he was just doing an honest day's work "pointing" out how many people died from uranium poisoning as a result of uranium tipped shells. Yep. That sure sounds like a MidWest college professor type that had spent most of his adult life on campus teaching kids. Hey, he sure wasn't a Ba' ath party member, was he fellas?
And this was the KEY guy Roberts used to conduct the survey. He was a "college professor", who worked for Saddam's health ministry in the first Iraqi war.
And Gouldiechops imagine if a "racist" Republican (aren't they all) had used the services of someone from a distasteful regime. Tell me how many times you would be raising your skirt and screaming racist?
And nice touches trying to get a jab in using my name, you wretched creep. Doesn't hurt Gouldie so try another act, you clown
Wait, WTF?
David Kane:
in #9 you say
WTF are you playing at? So it isn't L1 here at all. I specifically responded on this already, and I know you saw the response. This was collected. I have seen it.
How many times does this need to be repeated? Are you EVER going to let it sink in, or will you simply keep ignoring it forever, in favour of claiming that researchers are lying?
If threats of physical violence and blatantly repostingt copyrighted articles in full don;t get him banned nothing will
You mendacious little creep, Gouldiechops. Try adding the fact that you called me a racist in an attempt to assassinate my character when I am nothing of the sort. I simply asked you if the private parts to call me that to my face. I still don't think you do.
As for posting a copyright piece.... don't be an idiot. I simply posted the piece instead of an excerpt. Maybe you should inform all of blogdom they are breaking the law, lawyer boy wannbe.
JC says: "Dr. Mengels ... Dr Goebbels ... mendacious little creep".
That's it -- you're banned, JC.
If you read the various responses by Kane in the three relevant threads (including this one), it's obvious that Kane is unable to keep his accusations of dishonesty and fraud straight.
L1, L2, not L1, oh not L2, L1, L2 ...
I'm beginning to see that, dhogaza. I thought for a while that it was just *me* getting confused. Now I know that's because I was trying to track constantly shifting accusations. Didn't Kane also claim to have seen the L2 data himself on at least one occasion? What's that about, then? Did he see it or didn't he? If he actually saw it, then he should *know* that it includes age and gender information for householders (not just the dead); was he lying about seeing it as well as lying about what's included in it?
Isn't this sort of thing actually misconduct?
Oh, speaking of misconduct: jc -- if you're reading this -- posting an entire piece, not a brief exerpt, without any citation or attribution, is what is called "plagiarism" and "violation of copyright" -- it is not covered under fair use rules, and is actually illegal. And the only person who'd been doing that here is you. And you're pretty much the one being really vile, too. Banning -- absolutely appropriate. (Thank you, Tim.)
David Kane has said many things ... reading carefully it sounds as though he's claiming that ...
1. He's seen a version of the data
2. It didn't include demographic data
3. He e-mailed someone involved with the work, who told him that the version of the data she had doesn't have the demographic data.
Therefore Lee Roberts and you (Luna) are both "misspeaking" when you claim demographic data has been collected, and in your case, that you've actually seen it.
dhogaza provides a fair summary. I am sorry if Luna has not been following the debate as closely as, say, Donald Johnson and so becomes confused over the issues.
1) I have seen and examined the L2 data. In fact, I have provided code for working with it. See here for all the messy details. (New version coming soon!) The last half of this pdf provides the only (public) detailed overview of the data. Read it closely. There is no demographic household data (beyond sex). The L2 authors claim (and I believe them) that this is the only L2 data that they have released.
2) The debate here is whether or not demographic household data (specifically the ages of each member of the household) was collected. Author Shannon Doocy told me it was not. Tim quotes Les Roberts saying it was. Someone is mistaken!
3) The only data from L1 that was made available to anyone is this summary data. See here. (And, not to brag too much, but were it not my efforts, you wouldn't even have that to look at.) As you can see, there is nothing but cluster-level summary data there. Les Roberts claims that demographic data was collected for L1 as well. I have no idea if that is true. (Doocy and I only talk about L2, since she was not an author on L1.)
Why does this matter? Because demographic data is hard to fake (see the data-heaping discussion in the National Journal article) and because, if there were fraud, it might be possible to see it in the demographic data. That's why I (and every other scientist looking at this topic) would like to see this data, if it exists.
Consider the following scenario: The L2 interviewers, for whatever reason, decide that they want to report more deaths. They go to a cluster and, following the procedure outlined in Burnham's discussion, gather all the neighborhood kids around and tell them about the survey. So far, so good. But then they ask the kids, "Has anyone in the neighborhood died in the last couple years, especially violently?" The kids know this and tell them. Then the interviewers preferentially select those houses, either a picking and choosing around the neighborhood or just placing the 40 house cluster in the part of the neighborhood that, by chance, had the most deaths. In that scenario, all the data is "accurate" in the sense that no one is making anything up, but the mortality estimate will be much too high.
Access to the demographic data might allow us to catch that because the houses would have way more young men than a random sample should have, given what we know about the age/sex distribution in Iraq from sources like ILCS.
Hope that this overview is helpful.
"Consider the following scenario: The L2 interviewers, for whatever reason, decide that they want to report more deaths."
Tell me David, other than your not liking the end-result is there the least reason to consider any such scenario?
Consider the following scenario: nefarious forces within the Bush administration, knowing that the casualties in Iraq are eactually far worse than even the Lancet figures, hire one David Kane to cast down on the Lancet studies...
Let me say that in fairness to Luna, I had to re-read the threads carefully to understand what you were and were not saying, and concentrate hard on your factual statements.
Weeding out the accusations of fraud, dishonesty, and a bunch of hand-waving.
If you left out the endless innuendo of the "consider ... L2 interviewers, for whatever reason, decide that they want to report more deaths" variety people might actually understand what you're saying.
One minor nit-pick - earlier you said age/sex information was not gathered.
Here you say "There is no demographic household data (beyond sex)." and "The debate here is whether or not demographic household data (specifically the ages of each member of the household) was collected."
Well, no. You've said repeatedly that no demographic household data was taken, not "specifically the ages of each member". In particular you've not stated that sex information was included in the data you've been offered.
For instance...
and
No where do you state that you believe that sex, but not age, data was collected.
You repeatedly state that "demographic data wasn't collected" or "age/sex data wasn't collected" ...
A bit disingenuous, no? Typical of your approach to this issue. Don't quite lie, don't quite tell the full story.
I believe Lee Roberts and Luna, just based on your past track record, David, in which you have shown a rather casual attitude towards the truth and a tendency to somewhat hastily accuse Roberts et al of fraud.
David, have you stopped beating your wife, yet?
David Kane: "Author Shannon Doocy told me it was not."
Perhaps you could provide a quote from the email?
Consider this scenario: Ages have been removed from the data set given to David Kane to protect the privacy of the respondents. Kane tells Munro that ages were not collected. When called on this, he invents an email from Doocy.
I bet Munro was told that NO demographic data was collected, since Kane's been implying strongly that neither sex nor age information was collected until he finally clarified this in his last post.
My explanation is simple: There is no fabrication because this data was not collected. Roberts was mistaken, perhaps honestly. Garfield himself notes that Roberts did not supervise the fieldwork as much as he (Garfield) thinks he should have. If the data was collected, don't Roberts and Burnham have an affirmative obligation to correct the record?
i have read o lot of false stuff on the web, but this is perhaps one of the worst.
let me get this straight:
you keep posting and spreading false accusations, and the other side has the obligation to correct them?
that is turning the world up side down!
i seriously hope that some more of your academic colleagues finally to comment on this behaviour of yours.
the obvious truth is,
that it is YOUR obligation, to find out the truth about accusations, BEFORE you decide to spread them via the blogosphere or right wing analysis or editorials!
it is your obligation as well, to correct any misperceptions caused by your baseless allegations. this doesn t depend at all upon any comment by Burnham et al.
but as we all see in your comments here, instead of making up for the mess you caused in the past, you re already releasing the next set of accusations, without any basis, of course.
This is from the methods section of the paper...
It does not mention asking for the age of each household member, but only the sex. However the methods section refers to appendex A and appendex B, which weren't attached to the copy I found in Google, so perhaps the summary above is incomplete.
So David Kane may be right in a very obtuse sense, skirting the truth in his statements without actually lying.
"When I said age and sex data were not collected, I really only meant 'age', and hey, it's your fault if you assumed I meant both and since you never asked me specifically I was under no obligation to clarify the statement."
It doesn't excuse his steady stream of innuendo and implications that age wasn't taken in order to facilitate fraud.
(If he does, I owe you/him/Deltoid an apology.)
by my vast experience with him and from a detailed study of the character of one David Kane, i will proudly present to you,
the David Kane version of an apology:
alrigh, it looks like the confusing and incomplete informations provided by the lancet authors left me with the impression, that this data did not exist. now that the data has been presented to me, i have to accept the existence, though i will still have to check them for fraud, as it could be that they just invented it to answer my request. i don t want to boast, but without my tireless and constant effort, this data would still not be public.
you will only ever hear another comment from me about the data, when it confirms by conviction that it was invented. (and i will accept the smallest hint to come to this conclusion). i will not correct a single of the articles published on the basis of my baseless false claims. i will continue to call the lancet authors liars and frauds, without having any evidence.
Meh, I had hoped to be able to add more to this. The dataset is not in my possession, personally, but someone I have occasionally worked for. I asked this individual if he would comment here or to any of the people present here on what was and was not included. However, he does not want to get involved in the blog commenting or blogwars or internet debate, or any other possible media debate that he might get drawn into, and has chosen not to publicly comment on the content. I can't say I'm not disappointed, but I need to respect that decision, and I won't "out" him, either.
All I can do is repeat what I said before: the 2006 survey followed the same procedure as the 2004 survey, so far as I can tell, in terms of collecting gender and basic age data for householders. This was in the dataset I saw as the standard sort of age categories; although I believe they may have actually collected the ages of children under 4 in months, as stated, I didn't see a breakdown of that category and couldn't comment.
Perhaps the ages were redacted from the dataset Kane saw. Why that would be, I don't know. But seriously, this accusation thing is getting ridiculous. Kane is levelling a very serious accusation against the researchers, of deliberate fraud and lying at the most basic level, by saying that they did not collect the data they claimed to -- although he's surely got to know that this is wrong. That skirts libel, surely?
dhogaza says:
Silence need not indicate inaction.
After all, Harvard made no public comment about Lubos Motl's behavior before they and he reached an amicable parting.
I didn't say that, Sod said that back in post #37.
Not that I find the association particularly offensive :)
It's hard to keep all the praise of Kane straight here.
" I am sorry if Luna has not been following the debate as closely as, say, Donald Johnson "
I'm going to take this nice thing David has said about me as a tacit concession that I'm right about how Lancet1 (without Fallujah) is in very close agreement with NEJM. (And thanks to dalazal for taking the argument further.)
So, David, if you are cited in any future magazine articles, in the interest of fairness you can say that the IFHS article casts doubt on the violent death toll of Lancet2, but you should also point out it seems to confirm the 98,000 excess death toll and even the violent death toll implied by Lancet1 (if Fallujah is excluded), which is something that none of the prowar critics of Lancet1 would have conceded when that paper was the main object of attack. As Les Roberts points out, IBC updates their death tolls for a given time period as new evidence comes in. When Lancet1 was published IBC was claiming around 15,000 deaths for the first 17 months (they later increased this to around 18-19,000, IIRC). That's why Lancet1, even with Fallujah excluded, came as such a shock to people.
It seems important to me that people understand both how the new paper conflicts with L2, but confirms the widely derided estimates of L1. Does it seem important to you? Why or why not?
I see that dsquared is making this point with a bit more, um vigor, over at Crooked TImber--
http://crookedtimber.org/2008/01/11/mea-culpa-mea-maxima-ahhhstickitupy…
"Thus far it looks to me as though Iraqi Ministry of Health data were used to estimate that statistic, thus making "excess deaths" a meaningful concept."
Er, yeah, how do we know the death rate before the invasion? Were the Kurdish regions that received the "health gas" from Saddam included in the cluster sampling? What about the 100k Shiites that I remember reading about being killed after they tried to rise up against Saddam after GW1?
And can you really trust any numbers coming from Iraqi "government" institutions from before the invasion?
If Ben has a point, I fail to see it. But that's normal when reading Ben's posts.
Ben, neither of the events you mentioned occurred in 2002, so no, they don't show up in either the Lancet papers or this NJEM paper.
What methodology was used to derive the 100K Shiites that were killed in the post Gulf War uprising? I've seen tens of thousands and sometimes even hundreds of thousands cited for that, but nobody ever bothers to say where these numbers come from. If it's the fault of an enemy, the journalistic convention seems to be that there's no need to cite evidence.
"Ben, neither of the events you mentioned occurred in 2002, so no, they don't show up in either the Lancet papers or this NJEM paper."
I claim that they still matter when judging the good/bad of the invasion based on the body count. Was Saddam a mass murderer? Yes. Was there a reasonable likelihood that he'd commit more atrocities in the future? Yes. Was the alternative better or worse, and not just in terms of the body count?
Good grief but some of you are one-dimensional.
Daniel, at CT, makes a point that I would like to hear more about. Apparently the WHO group couldn't survey in Anbar province, so they approximated using some combination of the IBC numbers and extrapolations from their surveys of other provinces. Is this right? If it is right, I don't understand why I'm to think Lancet 2 was wrong - it seems to me that the trend upward, by all surveys and estimates since Lancet 1, seems to point towards a number somewhere between the WHO and Lancet 2, at the very least.
But Roberts seems to trust the WHO numbers in that reply. Does this mean he accepts the Anbar estimates?
ps: I am thinking of this section of Daniel's post:
"The NEJM does have a much bigger sample size than either Lancet study (9345 households in 1086 clusters versus 1849 households in 50 clusters). But the randomness of the selection was seriously compromised - 11% of the clusters were too dangerous to travel to, and they have their data filled in by extrapolation from the Iraq Body Count website.
This matters, of course, because if we're thinking as Bayesians, the weighting we should put on the new study relative to the existing ones is inversely proportional to the uncertainty of the estimate. And the uncertainty of the current estimate is dependent on an assumption about how much inaccuracy this extrapolation introduced into the recipe. The authors write, fairly enough, that "Uncertainty in the missing cluster-adjustment factors was difficult to quantify, since we assumed that the excess risk of mortality in missing clusters in Baghdad and Anbar was normally distributed, with standard deviations of 0.2 and 0.1, respectively", but if you plugged higher numbers into this guess, it would blow out the confidence intervals quite materially. Anbar is the real question mark here - it is a very big contributor to the death rate estimates in the Lancet studies, and it was one of the least well-covered areas in Iraq in terms of the media reports that IBC relies on."
Ben, we also don't know what the long term consequences of our invasion might be. All we know (sort of) is what has happened so far--a death toll in the hundreds of thousands.
Roger, as best I can tell (as a nonexpert speaking) there's no certainty about which study is right. I think the new study would lower one's private estimate of what the true death toll is, but with the problems you mention (citing others), there's no particular reason to think the violent death toll in June 2006 was really as low as this latest paper claims. But you've read dsquared, so I'm only writing this to say I'm an agnostic on whether the true number is in the low hundreds of thousands or high hundreds of thousands.
"I claim that they still matter when judging the good/bad of the invasion based on the body count."
The point of the discussion here is "did the invasion result in any increase in mortality?"
The relevant figure for that is not the death rate in 1993 when the Shia rebellion occurred but the death rate in 2002. Kinda like if the US invaded Germany today, the death toll from the Holocaust would be essentially irrelevant.
Furthermore Ben the figures you're claiming for the deaths in the Anfal campaign and the Shia uprising have never been proven to anything like the degree of accuracy of the post-invasion deaths.
Hell, Dick Armey, as a KKR employer, was arguing after the Haditha massacre that the attacks were actually carried out by the Iranians.
Ben is also ignoring that we had Saddam under our thumb post-GWI.
No-fly zones in the north and south, semi-autonomy for the Kurds enforced by our military power, etc.
Not an ideal situation, but not one in which Saddam was hobbled.
Ben, since you think Saddam was wrong to kill Kurds, what do you think of Turkey's US-backed effort to kill those very same Kurds who want to re-establish Kurdistan at the expense of Turkey and Iraq?
The Turks have the same bitch with the Kurds that Saddam did, and have done a little raiding and bombing and shooting along the border of Kurdish Iraq with our blessing...
How's your sympathy meter for the Kurds reading now?
"Was there a reasonable likelihood that he'd commit more atrocities in the future?"
Yeah because overnight the US and UN might have lifted all sanctions; the US might have withdrawn all its forces from Kuwait; his Gulf state neighbours might have reverted to funding him and the Kurds might spontaneously have decided to give up their autonomy.
Then it would have been a simple matter of totally rebuilding his military from scratch while simultaneously reconstructing the country's infrastructure.
"Was the alternative better or worse, and not just in terms of the body count?"
In other words, you don't CARE how many Iraqis died. That being the case, why do you waste time trying to minimise the casualty figures?
Oh and Ben your earlier comment about how you'd rather die than live undser Saddam: would you rather watch as your father was hung on a meatbook after electroshock torture and your wife and mother were pack-raped to death than live under Saddam?
It's a virtual certainty that areas too dangerous for surveyors are also too dangerous for journalists, raising the quetion of how you use IBC, a compilation of media reports, to adjust for the unsurveyed clusters.
So over at crooked timber david kane has put himself on record as believing that the chance that the Lancet 2 report is fraudulent (not just wrong, but a fraud, built on data that was never taken) is 50%.
And that this new work makes him think the odds are higher, and that this may be one of the more significant cases of scientific fraud ever.
Thus far, though, he's not arguing that the new study must be untrue because of the extremely high response rate, which IIRC was one of his most vehement lines of argument against Lancet 2...
Ian Gould posts:
[[would you rather watch as your father was hung on a meatbook after electroshock torture and your wife and mother were pack-raped to death than live under Saddam?
]]
Ian, that's the kind of thing that happened under Saddam.
Not only ...
Tim writes:
Funny! Unfortunately, I do not like to quote from someone's e-mail without their permission. Fortunately, in this case I don't have to. I e-mailed Doocy to confirm (after all, I could have been wrong about this) and cc'd Tim. She replied and cc'd Tim. No age information was collected for the households. This means:
1) Munro is not guilty of "fabrication." Tim owes him a correction and apology.
2) Roberts has been lying (on more than one occasion?) about this. Why?
3) It is very hard to confirm that the L2 sample is representative without age information. Was there fraud? Without age information, it is harder to tell than it should have been.
Oh, come on.
You owe EVERYONE an apology for claiming, over and over again, that no sex/age information was taken, rather than being explicit that you were talking about age information.
You were caught in a half-lie, which is still a lie.
Why would you, with your willingness to lie, obsfucate, hint, spew innuendo, and generally engage in dishonorable activities here and elsewhere, give a shit?
I know that I, for one, will never read the name "David Kane" without assuming, in the absence of truly overwhelming evidence, that you're lying through your teeth.
And I started out totally neutral on the study and your reputation before you exposed yourself for being the dishonest piece of shit that you are.
In other words, the survey should've been designed not only to gather accurate information ...
but to defend the researchers from accusations of fraud by the likes of David Kane.
And now David Kane argues that since they did not actively gather data to effectively counter unfounded claims of fraud ...
that somehow they're at fault.
and that we should pay attention to his accusations.
You can see the brilliance of Kane's tactics. A new study comes out which casts some doubt (how much is debateable) on the 600,000 violent death toll, but which supports the earlier claim by Lancet1 that the excess death toll in the first 17 months was around 100,000. It appears from this new study that as a rough rule of thumb, one should take the Iraq Body Count's corrected death toll for a given time period and multiply it by 3. And that's from a study which might well be undercounting.
But we're not talking about that--we're talking about Kane vs. Roberts. Because, of course, the fact that we've been lied to about deaths in Iraq by our leaders and the fact that Iraq is suffering a humanitarian catastrophe just isn't that important, compared to Kane's obsession with Roberts. What could be more important than that?
David Kane, in comment #32 at Crooked Timber:
"The wheels of science grind slowly, but they grind very fine indeed. If the data underlying L1/L2 is fake, then the Lancet papers will be the most important scientific fraud of the decade. Think that is impossible? Think again. What can the Crooked Timber community do? Act like scholars and scientists. (As Kieran does in this post.) Keep an open mind. Consider all the evidence. Look at the underlying data. Study the statistical models. Replicate the results. Make our findings public."
Ah, so it's all about the science. The most important scientific fraud of the decade, even.
Funny... that brought to mind something I read a couple years ago:
"The position taken is seemingly an innocent one that we do not know enough to know what the facts of history were, and rather than condemning anyone we should await the ultimate decision of research. This is a manipulative misuse of the valued principle in science that facts must be proven before they are accepted in order to obfuscate facts that are indeed known, and to confuse the minds of fair-minded people who do not want to fall prey to myths and propaganda. The very purpose of science, which is to know, is invoked in order to justify a form of know-nothingness."
-- From Israel W. Charny, "The Psychological Satisfaction of Denials of the Holocaust or Other Genocides by Non-Extremists or Bigots, and Even by Known Scholars."
I am not, of course, suggesting that what is happening in Iraq is a holocaust or a genocide. It's merely a generic human disaster... or, if you prefer, a Schroedinger's Catastrophe. Kane's methods and behavior, however, area aptly described by Charny: This is "Scientificism in the Service of Confusion."
Regards,
Bruce
"Furthermore Ben the figures you're claiming for the deaths in the Anfal campaign and the Shia uprising have never been proven to anything like the degree of accuracy of the post-invasion deaths."
I'm shocked! Wasn't Riyadh Lafta running some sort of ministry of perfect statistics back then?
"Oh and Ben your earlier comment about how you'd rather die than live under Saddam: would you rather watch as your father was hung on a meat hook after electroshock torture and your wife and mother were pack-raped to death than live under Saddam?"
Er, say what? Like, isn't that what life was like under Saddam? Or are the statistics on Iraqi torture not accurate?
"Ian, that's the kind of thing that happened under Saddam."
Precisely my point - Ben wrote earlier that Iraqis should have been wlling to "die for freedom" rather than live under Saddam (with the implication that if America had to kill a few hundred thousand of them doing what they were to cowardly to do for themselves tough noogies).
I'm simply pointing out that the consequences for opposing Saddam went way beyond the doubtless heroic death Ben imagines he would have chosen.
Oh and the incidence of torture in Iraq is probably as high now if not higher than under Saddam - it's being carried out variously by the Iraqi government (former interim President Iyad Allawi hhimself a former Ba'athist recruited thousands of former secret police into the new security services); Sunni militia; Al Qa'ida; Shia death squads and religious vigilantes.
Some of the more complicated former methods (like the use of acid) have been substituted for with simpler techniques (such as the use of electric drills applied to knees and skulls) that's pretty much the extent of the change.
"with the implication that if America had to kill a few hundred thousand of them doing what they were to cowardly to do for themselves tough noogies"
Ian, I made no claim that they lacked the courage. They simply lacked the means. It would have been an impossible undertaking, courage or no.
Chosen heroic death? There was no point, the task would be impossible. Far better to have a great power come in and stomp him for me.
What would you have preferred, Ian, had you been a Saddam era Iraqi?
Donal Johnson #63- Spot on! This should be broadcast from the rooftops, radio and everywhere else.
What would you have preferred, Ian, had you been a Saddam era Iraqi?
Well, I can't speak for Ian, but frankly if someone had told me, "Yeah, we'll stomp Saddam, but people you're close to - and honestly, maybe you as well - might die in the process," I'd thank him and continue trying to live my life as best I could.
Have I framed the crux of the matter well enough for you, Ben?
Let me get this straight. Your claim is that even though I provided a summary of L2 data last year which described the household gender information and posted that summary on the web, I am now trying to deceive Deltoid readers by pretending that sex information was not recorded? Give me a little more credit, please. (And, by the way, as Shannon Doocy can confirm, I actually discovered some problems with the household sex data (missing data in Taneen) and reported them to the authors.)
Now, of course, blog comments are never entirely clear, perhaps mine especially. But the purpose of the age/sex description above was not to deny that we have sex data (we do), but to point out that we need to joint distribution of age and sex, as I specified here. Instead of calling this a lie, could we go with misunderstanding?
But, anyway, who cares? You may think I am a liar. You may think that I am a truth-teller. My status hardly matters. Yet, for the purpose of analyzing the Lancet studies, Les Roberts truthfulness (or lack thereof) matters a great deal.
To repeat (and thanks to Tim for pointing out these quotes), Les Roberts claimed that:
We now know, thanks to Shannon Doocy, that the reason that Roberts did not have this "raw data" is because it did not exist. This might be a simple mistake on Roberts' part. Perhaps he will now correct the record. But the important part here is that Roberts made this claim in the context of a fairly serious dispute with Moore about the reliability of L2. (For the record, I think that much of Moore's critique, especially the sample size and cluster stuff, was wrong.) Reading that discussion a year ago, I thought that Roberts had won and Moore had lost because I believed, naively, that Roberts was telling the truth, that he had collected age/sex household data and that Moore's claims to the contrary were evidence of incompetence and bad-faith on Moore's part.
I clearly owe Moore an apology. And maybe Tim does too! Recall the discussion that they had in the comments:
Good thing that the WSJ never printed that correction!
The sad thing is that the behavior of many Lancet supporters has driven honest experts like Moore and Mike Spagat and Josh D away from the discussion boards of places Deltoid. That's a shame.
And so I soldier on.
And to Donald Johnson. I do not claim that the L2 results are important. I do not claim that the Iraq war was a good thing or a bad thing. My only claims are that the L2 results are wrong, that they are likely caused by bad data, that the data may have been gathered in a fraudulent fashion, and that the behavior of Les Roberts (especially his refusal to share data and code) has been shameful. If you are uninterested in these claims, no worries! Don't read my comments.
But David, you also make claims about L1. Repeatedly. And yet L1, without the Fallujah data, seems to be in close agreement with this new study. For anyone interested in these Lancet discussions, or more importantly, determining the death rates in Iraq, that ought to be a very important point. Two studies both in agreement on the death rate up until Sept 2004, and both in agreement that IBC's numbers are low by a factor of 3 or so.
If you were really interested in how many people have died in Iraq, rather than simply trying to discredit the Lancet papers and Les Roberts in particular by any means necessary, you'd acknowledge this new paper actually vindicates what people here were saying about the L1 report and the likely number of violent deaths. All of us knew that L1 had large error bars for excess deaths and that the paper didn't give them at all for the violent deaths (IIRC), but it was understood that the mid-range estimate (excluding Fallujah) was somewhere near 60,000 for the period up to Sept 2004 and most if not all of the Lancet critics strongly denied that IBC's numbers could be off by a factor of 3-4. If you were arguing in good faith I'd expect you to admit this--the defenders here of L1 have been given support by this new paper, even if it also casts considerable doubt on the violent death toll from L2 .
When I see a Lancet critic making these points, I'll take that person seriously. It's not like I've ever fully accepted the 600,000 figure--I've been in the agnostic camp on L2 from the day I first heard about it.
David Kane wrote above:
Let us assume that there was no ambiguity either in the question Kane posed in his e-mail or in the answer he received from Doocy.
One still must wonder how Kane can be absolutely certain that Shannon Doocy is the one who is correct about the demographic data and not Roberts -- ie, that Doocy is not simply herself mistaken with regard to the existence (or possibly, absence) of the demographic data.
What if a police investigator (or scientist, for that matter) were to take such an approach to an investigation?
1) Tim also got the e-mail from Doocy. If there is any "ambiguity," I am sure that he will let you know! There isn't. Since Doocy was actually in charge of the data, I believe her and not Roberts. Call me cazy!
2) Donald Johnson insists that "And yet L1, without the Fallujah data, seems to be in close agreement with this new study." Back of the envelope, that seems true. (Note that I have not closely read the IFHS paper and, first pass, I am not sure how you remove Falluja from its sample to compare it to an L1 sample without Falluja.) But, anyway, does that admission make you happy? I am certain that if you pick some other subpart of the sample, like say Kurdistan, the results for L1 and IFHS (and even L2) match up nicely. Personally, I, like the IFHS authors, am most interested in the estimate of violent deaths for all of Iraq. If you think that this is a stupid thing to care about, take it up with them, not me.
Tim, can you tell us how Doocy responded to the email you were cc'd on?
David Kane says:
"Donald Johnson insists that 'And yet L1, without the Fallujah data, seems to be in close agreement with this new study.' Back of the envelope, that seems true. (Note that I have not closely read the IFHS paper and, first pass, I am not sure how you remove Falluja from its sample to compare it to an L1 sample without Falluja.) But, anyway, does that admission make you happy?"
It would probably make us happier if you'd email the Andy Kaufman set and tell them that the supposed "refutation" of L1 that you presented back in July was a bunch of nonsense.
No, scratch that... it wouldn't make us happier. But it might make us respect you a little bit more.
David Kane said:
I don't need to call you anything. Your words and actions speak for themselves.
But based on the task descriptions at the end of the L2 paper, I'd have to say that Lafta is probably the one most familiar with the questions that were asked as part of the survey.
"S Doocy managed the study data" (according to the paper) but it is at least possible that she may have only dealt with (and seen) the data that was directly applicable to deaths, since that is what would have been used in the mortality calculations.
But I honestly don't know -- and I'd be bet that neither do you. Your entire argument rests on placing more faith in the word of one person over that of another. Not a very scientific approach, I'd have to say. It comes down to the old saw that "absence of evidence is not evidence of absence."
Based on the job descriptions of the various authors, there is certainly not enough information to be accusing Roberts (or anyone else) of lying.
I find it particularly ironic that you gave this bit of advice to dhogaza above:
Perhaps you should follow your own advice.
/////
From the end of the L2 paper
Contributors
G Burnham, as principal investigator, was involved in the study design and ethical approval, took part in the analysis and interpretation of results, and led the writing of the paper.
R Lafta managed the field survey in Iraq, participated in the study design and the analysis, interpretation, and preparation of the manuscript.
S Doocy managed the study data and was involved in the analysis, interpretation, and the writing of the manuscript.
L Roberts instigated the study and assisted with the analysis and interpretation of the data and the writing of the manuscript.
Doocy's email was:
>You are correct, age was not collected for all household residents.
Now, of course, blog comments are never entirely clear, perhaps mine especially. But the purpose of the age/sex description above was not to deny that we have sex data (we do), but to point out that we need to joint distribution of age and sex,
as I specified here. Instead of calling this a lie, could we go with misunderstanding?
simple answer: NO.
you are a master of deceive David Kane. you can expect as much good will from us, as you are willing to offer to Roberts.
so let us look at the cliam made in the Munro article (you being the willing source, i think):
Lack of supporting data. The survey teams failed to collect the fraud-preventing demographic data that pollsters routinely gather. For example, D3 Systems, a polling firm based in Vienna, Va., that has begun working in Iraq, tries to prevent chicanery among its 100-plus Iraqi surveyors by requiring them to ask respondents for such basic demographic data as ages and birthdates.
the text does not specify this to a problem in lancet 2. (it routinely mixes in parts about lancet 1 and speaks of "lancet surveyS" quite often.
so with the information that we have at this moment (lancet 1 with sex and age information, lancet 2 with sex information) it looks like the Munro claim "no demographic data was collected" is 3/4 false.
let me just finish with some additional remarks:
1. David Kane and Co have so far presented exactly ZERO evidence of fraud in the polling for the Lancet surveys.
(such fraud would be the major reason for a requirement for this data)
2. in not a single survey that i participated in, was i asked to provided detailed sex/age information for all persons living in my household. i most likely would not provide such information.
(age categories seem much more likely. i will keep an eye open, for public data of surveys that uses the "David Kane version of demographic data male/23 ..")
3. David Kane has not made a convincing point, about how this data would stop fraud. in the "imaginative scenario" given by him, the pollsters ask kids for the most violent part of town, to poll there. (quite unlikely imho, but..) i don t see how demographic data would prevent this. neither would it prevent occasional "addition" of a death case or changes from non-violent to violent death.
it could prevent completely made up samples. but even this case is weak, with the current status of iraqi demographic information. (basically we don t know how many iraqis are living where and the pre-war death rates are an obvious indicator of the status of the data)
but surely David will give us some more information for the basis of his wild allegations in his next enlightening piece on the Lancet papers.
Bruce Sharp provided the following quote from Crooked Timber:
and attributed it to David Kane.
My question for Bruce:
Are you SURE you didn't accidently cut and paste a quote from Steve McIntyre on Climate Audit, regarding James Hansen's body of work, simply substituting "L2" for "climate models" etc?
Because man, it's word-for-word the kind of crap he spews.
Kane also opines
No, deliberately misleading people is a form of lying.
You're too smart to be this misleading so repetitively, over such a long period of time, over so many aspects of this debate, by accident.
Another gem
This statement give us further insight into your personal ethics. Thank you.
dhogaza said:" Thus far, though, he's not arguing that the new study must be untrue because of the extremely high response rate, which IIRC was one of his most vehement lines of argument against Lancet 2..."
Do you play bingo, dhogaza? If not, you should.
The response rate in L2 was 98.3% out of 1849 households.
The response rate in NEJM was 96.2% out of 9345 households.
The response rate in NEJM is slightly lower, but since the number of households was 5X as great in NEMJ, the small difference between the response rates means little if anything.
The high response rate in L2 was highlighted in Kane's now infamous A Case for Fraud (which was removed from the Harvard IQSS blog, as IQSS member Amy Perfors said, because the "tone is unacceptable, the facts are shoddy, and the ideas are not endorsed by myself, the other authors on the sidebar, or the Harvard IQSS."
In the above piece (of shoddy workmanship), Kane posited
three possibilities.
1) The survey teams provided fraudulent data.
2) There is something different about this survey team or about Iraq at this time which makes this situation different from any other survey ever undertaken.
3) The high response rate is a once-in-a-life-time freak event. It would not be repeated even if the same survey team took another survey.
I do not think that 2 or 3 are very likely. Fraud in surveys, on the other hand, is all too common.
Given that the response rates in L2 and NEMJ are essentially the same, I eagerly await Kane's next piece: "A Second Case For Fraud".
Maybe he can bind them up into a book. Based on the rate of responses here on Deltoid, I'm sure it would be a best seller.
This is absolutely bizarre. If Doocy says that age wasn't collected for all householders, can we clarify who age was collected for? I want to know what the heck it was that I saw.
Sorry, I neglected the indent for the Kane quote about the response rates above
David, here's a relevant quote from the NEJM paper--
"The rates and time trends of violent deaths differed considerably among the three sources (Table 4). On the basis of population estimates shown in Table 2 of the Supplementary Appendix, the IFHS data indicate that every day 128 persons died from violence from March 2003 through April 2004, 115 from May 2004 through May 2005, and 126 from June 2005 through June 2006. The Iraq Body Count numbers were 43, 32, and 55 civilian deaths per day for the same periods. "
Now what would make me happy would be if you'd go tell the rightwingers that take you to be an authority that this IFHS data, if correct, shows they were wrong about how many people were dying in Iraq. Apparently the numbers were three times higher than IBC's. And guess who was defending the likelihood of a death rate of that magnitude back in 2004-2006? People who took L1 seriously. Some defended an even higher death rate because of the Fallujah outlier, but that was always controversial, because there was no way to tell how much of a fluke that cluster was. So the conservative way to analyze the data was to throw out the Fallujah cluster and say that even without that, Iraq was suffering far more deaths than IBC had managed to count. Your pals on the right didn't agree. IBC itself didn't agree.
You've suggested (in one of the earlier threads) that most of those uncounted deaths were of Iraqi combatants. Interesting, since the US government told USAToday that it had only killed 19,000 insurgents from June 2003 to August 2007. So it's Iraqi combatants killing Iraqi combatants, and US forces have played only a secondary role in this purely military part of the conflict. That's certainly a revolutionary new way to look at the war.
I just noticed Kane has written his initial response to the NEJM (IFHS) survey and there is not a single comment in it about the high response rate of the survey (96.2%).
As they say, silence can be deafening.
It would certainly appear (at least on the surface) that Kane has a different standard for assessing the likelihood of fraud for the two surveys. In the case of L2, basing his argument largely on his incredulity at the high response rate, but not in the case of NEMJ.
If that were not the case, Kane would surely have etitled his latest piece "He said, she said" (instead of "Said and Unsaid"), since if the virtually identical high response rates (98.3% for L2 vs 96.2% for NEJM) for the two surveys implied that both involved fraud, deciding which (if either) was correct would amount to a game of "he said, she said".
Just an observation.
Come to think of it, in the absence of corroborating information (or perhaps a third party confirmation) "He said, she said" might also be a good description for the discrepancy between what Roberts and Doocy say about the collection of age data.
Please note that (unlike David Kane) I am not implying that either of them lied, just that one of them may be mistaken and it is not at all clear which, given the information I have seen.
"If Doocy says that age wasn't collected for all householders, can we clarify who age was collected for? I want to know what the heck it was that I saw."
Well, at the risk of being tautologically simplistic, the collection of dates of birth for births since the invasion would generate a subset of ages...
z -- that's a possibility, although to be honest I don't recall seeing an age breakdown for under-4s. I was wondering if I had perhaps seen the age breakdown on deaths, and was conflating that with "all householders". Unfortunately I can't even inquire about this until tomorrow, and right now my curiosity is eating me alive.
Luna,
That is almost certainly the source of your confusion. Age was collected for the 629 deaths, but not for anyone else. Feel free to apologize to me for all your nastiness above. For an overview of what data is there and what is not, see my pdf. A new version should be out next week. And, if you are looking for conversation topics with your friend on the Lancet team, you might ask them why they refuse to supply similar data (to anyone) about L1. Surely, any open-minded scientist would like to compare the demographic data from IFHS with that from L1 . . .
David, are you unable to at least pretend for ONE post, that you are an adult interested in scientific discourse?
sod, check the link. I am the fellow who created an R package with a suite of functions for working with the Lancet data. What contributions have you made to "scientific discourse" on this topic? Post a link and show us. And, by the way, anyone not working with my R package is getting the wrong answer since the raw data as distributed by the L2 authors contains a (small) mistake.
sorry, i forgot, i had a second request:
could we get a SINGLE post from you, not containing any praise of your own achievements?
ps: my latest calculations on the subject can be found here:
http://tinyurl.com/34s8tr
The conclusion that the reported response rate for Lancet 2 is implausibly high is entirely justified.
For L2, each house was visited only one time on one day. If no one was home or the occupants refused to answer, the house was skipped. In addition, responses were taken only from the head of household or spouse, not from adult children or any other adults living in the house.
For IFHS, survey teams were required to try each house three times if responses were not obtained on the initial visit. The IFHS website also reports that aggressive efforts were made to obtain responses from reluctant families ("many families were convinced by team members and supervisors to answer the questionnaire after repeated trials with the help of members of the local community") and responses were collected from other adult household members, not just the head of household and spouse.
And yet we're supposed to believe that L2 obtained a higher response rate than IFHS, despite its clearly weaker efforts to obtain responses from selected houses.
Another nail in the coffin of Lancet 2. I don't know why you guys are even bothering trying to defend it any more. It's junk.
Sounds good to me, Joe.
Ben, Kane's complaint about L2 was that the refusal rate was unbelievably low by Western standards and not to be believed. He maintained this position even after people found multiple surveys with similar response rates (some from a single door knock, incidentally).
But even allowing for multiple call backs, by David Kane's standards (Western response rates) the response rate is still unbelievable. Australian National Household Surveys of Health also do multiple call-backs, but have a response rate of around 50-70%. Getting up to 98% or 96% is unbelievable by Kane's standards.
(And note that Peeps' reference to the website confuses multiple sources of non-response, such as illiteracy and lack of interpreters, with one particular issue Kane had - the very low rate of initial refusal. But this study's rate of initial refusal, 0.4%, is LOWER than the Lancet)
But this study's rate of initial refusal, 0.4%, is LOWER than the Lancet
False. 0.4% is the total rate of declined household questionnaires, not the "rate of initial refusal."
Tough titties. Kane's endless argument that the high response rate exposed Lancet 2 as being fraudulent was based on the response rate, not the number of knocks involved in getting that number.
"The IFHS website also reports that aggressive efforts were made to obtain responses from reluctant families ("many families were convinced by team members and supervisors to answer the questionnaire after repeated trials with the help of members of the local community") "
Boy, that really increases my confidence in the survey. Iraq is in a state of civil war when these interviews were conducted, the government is hated by many Iraqis, and it turns out that some of the respondents were browbeaten into answering the questionnaire which is being put out by representatives of that government.
Donald
where did you get that quote from?
I got it from post 93, from Joe Peeps, who is trying to argue against L2 and in favor of the NEJM number. I presume he got it from either the paper or appendices to the paper.
Aly/Donald,
It's from the FAQ on the IFHS website, the question being: "What were the families' responses to IFHS?" The answer, in full:
"Most families in the sample cohort welcomed the field teams, but some families were afraid to respond to the interviewers. Others refused to answer. However, many families were convinced by team members and supervisors to answer the questionnaire after repeated trials with the help of members of the local community."
I agree with Donald Johnson that it's not all that reassuring, given the fact that in current Iraqi conditions some of the most influential members of the local community are likely to be quite scary people. But having said that I'm with Stephen Soldz on the need to refrain from applying the methods of the Lancet-bashers to this study. It's one of only three useful studies of Iraqi mortality. Some would say there are four, but I think the IFHS estimates confirm my view that the ILCS really doesn't tell us very much - does anyone now contend that "war-related deaths" actually means violent deaths excluding "ordinary" murders?
Well, I haven't seen any of our regulars here, or our host, immediately shout "fraud! fraud! fraud!" as happened with the Lancet-bashers from day one.
I think the most you're going to see here is some probing and poking looking for possible problems with an essentially sound survey.
I can't imagine anyone's going to claim the authors are guilty of fraud ... noting even approaching that.
"but having said that I'm with Stephen Soldz on the need to refrain from applying the methods of the Lancet-bashers to this study. "
What dhogaza said, right after your post, Kevin. I just think there's the potential of a big problem having the Iraqi government directly associated with a mortality survey in the midst of a civil war where the government is linked to some of the massive human rights violations. In one of these threads I made an analogy to Saddam Hussein authorizing a study of Shiite deaths following the 1991 uprising. To flesh that out, even if the survey teams themselves had the best of intentions, I would wonder what effect the government association would have on the respondents.
Donald,
I certainly didn't mean to accuse you or Aly (or anyone here) of doing a David Kane on the IFHS. Apologies for giving that impression. I completely agree that the role of the MoH is a problem - a real problem, in no way comparable to the faux-problem of what Soros does with his money. Actually I think it's such a big problem that critics of the IFHS will be tempted to focus on it more than they should, so you can read my comment as an example of projection if you like.
The hard fact is that no large-scale study is likely to be undertaken by people who are trusted by all sides in Iraq (even if such people exist) so we just have to make as much sense as we can of whatever we get.
No apology needed, Kevin. Your point is a good one.
The irony is that if the Lancet2 paper hadn't been published I'd have guessed a violent death toll by June 2006 that was around 200,000--at the high end of this NEJM paper. That's of course not based on anything scientific, but just projecting L1 forward and assuming violent death rates had gone up somewhat. Plus other admittedly dubious pieces of evidence, like the claim that some Iraqi group counted 128,000 deaths by 2005.
Joe Peeps, by initial refusal I meant refusing to do the survey point blank, rather than, say, refusing halfway through. I wasn't particularly referring to the number of efforts required to get that initial refusal. Even with 3 all-backs a 0.4% refusal rate is very low and in other cases would be cause for certain individuals to scream FRAUD.