Johns Hopkins completes review of Lancet study

the School has suspended Dr. Burnham's privileges

Watch this get quoted as proving the study was false.

1) Kudos to Tim for bringing this to our attention. Although I disagree with Tim on many topics, there is no doubt that he is a scientifically honest straight-shooter. It is a pleasure to be part of the Deltoid community.

2) Holy heck! Note that this meets (sort of) my 3rd option for the end game of the Lancet debate.

Censure of Roberts/Burnham by Johns Hopkins or (in the case of Roberts) Columbia. If leading research universities don't uphold the norms of open and transparent scientific inquiry, who will?

Now, admittedly a "censure" is not the same thing as a suspension of research privileges, but I'll take it. When was the last time that Hopkins suspended a tenured member of the faculty?

3) Who wants to bet that Roberts et al (2004) featured a similar violation of protocol? On to Columbia!

4) As always, I am a defender of Burnham. I think that he is basically a good guy, caught in an unfortunate situation. I suspect/hope that he has not been purposely lying to us for years about whether or not identifying information was collected. I hope that he instructed the interviewers not to collect such information and then either never knew that they collected it or found out much later. But if Burnham knew, as the forms were turned in to him in Jordan in late summer 2006 that they contained names, and then misled us for more than two years, well . . .

5) My secret fantasy is that Tim will (at some point) decide that the Lancet data is fraudulent and will then be honor bound to join the ranks of we "deniers." Won't that be a good time!

6) Hey Daniel Davies! Better get those boots on . . .

I have to say I find it very annoying that they took that data and said they didn't. If I had to read the tea leaves, perhaps they thought that taking names would be CYA for inevitable criticism, however in hindsight determined that that information could seriously put respondents at risk. This is a big no-no to go against the IRB. I don't in any way think fraud is at issue here, but this is an ethical breach by Burnham.

David Kaned:

A rather large point of clarification: he was suspended from leading specifically human subject research, not from doing research in general.

That is, if someone were to present him with data, he'd be allowed to analyze it. And he can lead research into areas that do not involve submitting proposals to the IRB.

To claim he was suspended from doing research is very, very wrong.

I have to say I find it very annoying that they took that data and said they didn't.

Indeed. The technical term for this is "lying." When a research team lies to you about item X, you should immediately wonder whether they are telling the truth about Y and Z. A good rule of thumb is to never trust the research team again.

Question: Is it fair to assume that Burnham has been lying about this all along? My understanding has always been that Burnham and Doocy traveled to Jordan in the summer of 2006 and that Lafta hand-delivered the forms to them straight from Iraq. Then, the data was entered into the computer. If so, then Burnham must have known from the start that the forms contained identifying information. (Roberts, on the other hand, may never have seen the raw forms and could, therefore, be arguing in good faith when he claimed that no identifying information was collected.)

But, before I create a master list of all the people that Burnham lied to (starting with me and all the other folks at his MIT talk in 2007), I want to know if there is some theory by which Burnham never saw the forms and so never knew that they contained names. Any thoughts?

I don't in any way think fraud is at issue here

Good to know. So, you are sure that the only thing Burnham lied about was the identifying information? He was 100% guaranteed truthful when it comes to, say, the details of the sampling plan? How can you be so sure?

Again, my hypothesis of the crime is that there were many, many "miscommunications" between the surveryors and the US authors. Burnham/Roberts told Lafta to do X (not record names), Y (sample all 18 governorates) and Z (insert your topic here). Alas, Lafta didn't do X (he did write down names), Y (he only sampled 16 governorates) and Z. Burnham/Roberts tried to cover for him (and for themselves) as best they could. Alas, the jig is up.

David:

You seem to have little experience with IRBs.

If there's any change, however minor, on any form used or in any protocol, that change needs to be approved by the IRB. For instance, a computer science user study of, say, a new graphical user interface would need to re-submit paperwork if they want to change how much they pay participants in the study. Depending on how busy the IRB office is, this can take quite a while.

It's also easy to start believing that, if you're not doing medical/psychiatric testing, the IRB process isn't all that important.

IRBs, of course, exist for a reason. But they don't catch all violations of stated protocols. This leads some researchers to be less careful with how they handle IRB-related matters.

And yet their conclusions are still valid, even if they break IRB rules, as long as their conclusions reflect the data they gathered.

At this point, there really just isn't evidence to support the massive leap to the conclusions you're trying make. Especially since Hopkins found that the study itself does appear to be sound.

Jon: Please quote the sentence(s) that you are disagreeing with. What "massive leap" am I guilty of?

Here is Burnham speaking (pdf) at MIT two years ago.

There were limitations to record keeping; we had criticisms that we could not produce a record showing which houses were visited and what the names of the were, and so forth. We intentionally did not record that, because we felt that if the team was stopped at a checkpoint, of which there are lots of checkpoints, and the records were gone through, some of you may have had this experience, where you stop the checkpoint, people go through all your papers, read everything, and they find certain neighborhoods. That might have been the increased risk, which we didn't want to do.

If Burnham is lying about that, why do you believe him about anything else?

Jon claimed: "Hopkins found that the study itself does appear to be sound."

This is not correct. As stated in Tim's post at the top, they actually said: "The review did not evaluate aspects of the sampling methodology or statistical approach of the study."
They never said the sampling methodology was sound, or even hinted at it. They simply did not evaluate these aspects.

But it seems that Lafta's side of things was indeed done OK, as I had suspected. Good, so now we just need to know what he did.

It doesn't sound like they found any evidence of fraud in the data itself. I'm baffled about why they took names though. It seems like a crazy thing to do, no matter who it was that decided to do it.

I don't want to argue, but I think some issues of the forms would be what parts were in Arabic and what parts in English and blah blah blah. I don't think this means the study is fraudulent by any means, but I am incredibly annoyed that it has allowed certain people to take shots, when up to this point many of their arguments have been ad hoc or almost disposable. Start with one thing, move to the next. Quite depressing. Frankly, I'd like an explanation, and I find that the press release doesn't address the explanation, and I assume there was one.

I also wish LancetStudy would just say who they are, although I support anonymity on the web, in this case if one is arguing professionally, I think it is OK to argue in the open, especially if there is any insider information. Also, the dialogue would change because specific questions could be asked and answered.

David is calling everyone a liar again?

here is a first answer from Burnham:

Because of the difficulty of carrying out research in Iraq during the war, Burnham and his team partnered with Iraqi doctors at a university in Iraq. Burnham, working out of Jordan, said he made it clear to the doctors that they could collect the first names of children and adults, to help keep the information straight, but that last names could not be collected.

When the surveys came back to him in Jordan, it appeared that some had last names. Many were in Arabic. Burnham said he asked his Iraqi partners and was told that the names were not complete, which he accepted. But Hopkins, in its investigation, found that the data form used in the surveys was different from what was originally proposed, and included space for names of respondents. Hopkins found that full names were collected.

from [here.](http://www.baltimoresun.com/news/local/bal-researcher0223,0,2317874.sto…)

As always, I am a defender of Burnham.

David Kane, if ever i shoud be looking for some defence, you will be my very first choice!!!

If Burnham is lying about that, why do you believe him about anything else?

David Kane was caught in outright lies when this all boiled up. I stopped believing him about anything, afterwards.

On the other hand, Hopkins has found nothing that makes them believe the results of the survey are wrong ...

When a research team lies to you about item X, you should immediately wonder whether they are telling the truth about Y and Z.

I think Kane has a new definition of "chutzpah".

David Kane will, no doubt, now apologize to Burnham.

sod-

Thank you- this makes much more sense.

i think that this is a good example of a "throw enough mud, some will stick" tactic.

please notice how people who always claimed that the Lancet study didn t collect enough data (Kane) will claim that this is what they always said.

This brings up something rather fishy related to the other recent discussions about main street bias. The L2 authors have refused to release their lists of main streets to anyone (Laaksonen, the MSB authors etc.) even though this would be crucial to assessing what their sample design was. Burnham claimed this in 2007:

"The interviewers wrote the principal streets in a cluster on pieces of paper and randomly selected one. ... The team took care to destroy the pieces of paper which could have identified households if interviewers were searched at checkpoints."

Read this again. We're to believe the teams had to destroy a list of streets from which they would have selected one to sample, even though it's kind of a stretch to see how such street lists (including both sampled and unsampled streets, and with no household information) could be used to identify which houses were interviewed.

Now we learn that the field teams, while "taking care" to destroy those street lists, ostensibly to protect the identity of those interviewed, were carrying around (such as through checkpoints) the names of the people they interviewed!

So what is going on here? If the field teams told this to Burnham about the street lists they were either lying to him or they must be unusually stupid. I doubt the latter. Or did the field teams never say this and Burnham was just making this up?

Here i can't help but recall the old Nature article on L2 where the L2 authors were caught making up something else about the field work, that the teams used locals to find streets away from the main streets, but then being contradicted by a field team member who said they didn't do this. Both cases are assertions from Burnham and Roberts attempting to side step the MSB criticism. Going back even further, I also recall Roberts side stepping an early criticism from Steven Moore in the WSJ by claiming (falsely, as revealed years later here on Deltoid) that L2 did collect demographic data on their households.
Burnham and Roberts seem to have a habit of just making stuff up. And this usually seems to be when doing so will help them either deflect valid criticisms of their work or keep information necessary to evaluate their work hidden.

JHU also points to something new (at least to me) along the lines of the latter: "The original protocol explicitly stated that no names of study participants or living household members would be collected. The protocol also included an appropriate script to secure verbal consent from study participants, rather than a written consent process that would have included participantsâ signatures."

So the "original protocol" (which the study did not follow in the case of the data entry form), included a consent script which was deemed appropriate. But many researchers have requested L2's consent script (including AAPOR) and have all been refused. Why is nobody allowed to see the consent script for this study? Maybe the script that was approved wasn't actually used, just like the data entry form. Or maybe there's some far fetched reason why it had to be destroyed to protect the innocent.

I also smell a weasel-worded statement from JHU here:
"The review found no evidence that the violations caused harm to any individuals involved in the study"

What did JHU's attempt to find such evidence entail? If it didn't include anything that could reasonably be expected to find such evidence, such as re-contacting the L2 respondents, then the statement should read that their review did not really do anything to find such evidence and they did not find any.

The "update" has this to say:

"In an interview today, [Burnham] said he was gratified that the Hopkins investigation, as well as independent reviews, have verified his results. "I think that strengthens our conviction on the quality of the data and its relevance,""

Maybe JHU needs to investigate whether Burnham has started smoking crack.

The Lancet 'troofers' will have some fun with this. There are endless "what if"s , "or maybe"s for them to roll in.

The point was to find if deviation from the protocol possibly caused harm to respondents relative to the proposed protocol. JHU likely asked the authors to explain exactly how the information on the forms was used, stored, distributed. In the case that the information was inert, or essentially undistributed yet stored in a documentable, secured fashion, then I would propose that JHU would express that the changes did not explicitly harm respondents (above and beyond the original protocol).

There's a section of the review which Tim has omitted (above), but which seems relevant:

The Bloomberg School of Public Healthâs IRB acted properly in determining that the original study protocol was exempt from review by the full IRB under federal regulations. The original protocol explicitly stated that no names of study participants or living household members would be collected.

And then there's the bit below (which Tim does include):

Use of the form and collection of names violated the study protocol submitted to the IRB and on which the IRB determined the study was exempt from full human subjects review.

And then the Baltimore Sun account (presumably taken from Burnham):

Burnham, working out of Jordan, said he made it clear to the doctors that they could collect the first names of children and adults, to help keep the information straight, but that last names could not be collected.

When the surveys came back to him in Jordan, it appeared that some had last names. Many were in Arabic. Burnham said he asked his Iraqi partners and was told that the names were not complete, which he accepted. But Hopkins, in its investigation, found that the data form used in the surveys was different from what was originally proposed, and included space for names of respondents. Hopkins found that full names were collected.

What's not clear is whether the IRB would have exempted the study from full human subjects review if "use of the form" included any recording of names (eg just first names). I find it difficult to believe that JHU would suspend Burnham if the only mistake was the one described in the Baltimore Sun (ie that Burnham accepted the word of his "Iraqi partners" that the names were "not complete" - which doesn't sound like much of an ethical violation in itself).

David:

Sure, I was referring to the comment that "A good rule of thumb is to never trust the research team again."

That's a huge leap to make based on a failure to conform to the IRB protocol.

Again, I'm not downplaying the severity of ignoring human subjects requirements, but a breach of protocol is very far from forging results. The Hopkins review found compelling evidence of the former, but did not indicate finding evidence of the latter.

It is, therefore, a massive leap to assume the latter.

And, if they'd had evidence of fraudulent results, that would have resulted in much more serious disciplinary action.

Robert:

I dunno, at least at my university, full exemption requires that research by *very* low risk. I don't know for sure how that's defined for other areas, but for mine (user studies as part of electrical engineering/computer science work) it involves observing users doing typical activities and asking non-identifying questions about them.

Speech recognition work that involves recording -- rather important for later processing -- is not always considered exempt, even if names aren't recorded. And I've been working on a voice-controlled assistive device, and that isn't exempt. I think in my case, they worry about vocal strain, even if we're doing, say, a 30-minute user study.

So first names alone may well have been off-limits under the exempt ruling. But they also seem to have changed a form without approval, and that unquestionably would have required approval.

Each university's IRB is different, of course. But even if accepting the translators' word was ok, changing the form wasn't. And in a high-profile review, you'd better believe that they'll be strict about compliance.

So some first names and some last names were recorded on the sheets when they were not supposed to be. When Burnham asked about this he was told that the names were not complete and therefore would not represent a threat to respondents (something that the investigation agreed with). On the subject of the actual gathered data, the investigation had this to say:

The review concluded that the data files used in the study accurately reflect the information collected on the original field surveys.

That is, the results in the paper are totally consistent with the data recorded.

And now David Kane and others are once again trying to claim fraud?

Weak.

"The review concluded that the data files used in the study accurately reflect the information collected on the original field surveys."

This certainly settles the burning question of whether the process of coding the written forms into a spreadsheet was fraudulent.

"The original forms have the appearance of authenticity in variation of handwriting, language and manner of completion. "

It also says that, Raymond.

I have to say that at this stage I'd like a little more detail from this JHU investigation.

Aly wrote: "That is, the results in the paper are totally consistent with the data recorded."

Data recorded by Lafta is X. It then gets passed through some (unknown) signal processing box Y by US L2 team in which (among others) they scale up to the entire population. That produces data Z (i.e. 650,000 plus/minus...).

Aly's statement that Z is totally consistent with X is clearly misguided, without knowing Y.

Y, we are told, accounts for MSB, deduces R is near 1, and discards it. So what is this magic Y?

Kane cried fraud from the very beginning, with no basis to do so. Fraud has never been established in fact. Never. Kane has been wrong all along. Burnham made a mistake, one that is not at all a matter of fraud, and he is now bearing responsibility for it. He is an honorable man. Kane has never taken responsibility for his false claims of fraud. And now he revels in an honorable man's assumption of responsibility for an error. Kane is an irresponsible and dishonorable man.

LancetStudy

The issue which lead to censure was that in some cases names (i.e. extra pieces of information) were recorded when they shouldn't have been.

This has no bearing on the validity of the study.

Hence your X,Y,Z comments are meaningless, as you would see, if you weren't grinding on your axe so hard.

Cheers

Sam: It is beyond dispute that Burnham committed fraud when he lied to the audience at MIT in February 2007. Now, we just have to get to the bottom of how much fraud there is.

See latest entry for a full discussion of what we now know and don't know. Burnham is now calling Lafta a liar. Who should we believe? {Please leave any comments here at Deltoid, not at my blog, since this is the best forum for discussion.)

David Kane, in the course of some more blogwhoring: Burnham is now calling Lafta a liar.

I presume this is just another of your misrepresentions, David? I've seen no report of Burnham calling Lafta anything of the sort. Do you have a link, other than to your own blog?

Data recorded by Lafta is X. It then gets passed through a signal processing box Y by US L2 team, producing result Z (i.e. 650,000 plus/minus...). Hence Lancet article.

Process Y accounts for MSB, deduces R is near 1, and discards any possible effects of MSB.

What is Y?

David Kane: "Now, we just have to get to the bottom of how much fraud there is".

Exactly. I would like to know how much fraud was committed by various members of the Bush administration through serial lying and the manipulation of facts that paved the way for the invasion of Iraq. This lying led the US to violate international law, the UN Charter and the US Constituion, in committing the "Supreme international crime" and has resulted in utter carnage and desperation in Iraq. So when are you going to start pursuing the real criminals, David? If you are correct, there may thus 'only' be 100,000-300,000 corpses in Iraq as a result of the illegal war. Is this an acceptable total? Does it satisfy the requirements of a 'humanitarian' war and occupation? Why has this salient fact be left out of this debate?

I am not a statistician of an epidemiologist, but I was wondering why Burnham and colleagues only stoked the ire of academia and the corporate media when they published their Lancet studies. Why was there hardly a whimper raised when their results on the death toll from other conflicts was publicly cited by figures including Colin Powell? David, would you have expended so much effort if the study had been used to estimate the death toll in Afghanistan by the Russians during the 1980s? Why is this any different? Why now? Because it sheds bad light on the carnage? Please fill me in.

Whatever one can conclude from this sorry episode, is that it has shifted attention away from a humanitarian catastrophe that has resulted from the US invasion of Iraq. Here's some figures that few dispute: 75 per cent of Iraq's doctors have fled the country. The war resulted in at least 2 million internally dispalced people. And most importantly, it led to mass death of the civilian population. Historian Nir Rosen said that "Iraq has been destroyed, never to rise again". Let's get back to the real issues!

David Kane:

Is your quote from the MIT talk your best piece of evidence that Burnham said names were not collected? What you have quoted is not convincing because there is a missing word:

...we had criticisms that we could not produce a record showing which houses were visited and what the names of the were...

What is the missing word between "the" and "were"? Could it be "streets"?

David Kane, don't you get a little tired of "arguing" here? Everyone but you at Deltoid desperately wants the Lancet study to be true, and they will defend it and the authors to the ideological death. They want the truth to be that a zillion Iraqi's died as a result of the war, and that living under Saddam was much better than the small chance at freedom and democracy they have now, which they wouldn't have had in three generations otherwise.

I have never seen a single post here by the "regulars" even for a moment posing an honest question that goes against the grain of Lancet or AGW. Not for a moment, and not on any aspect. That seems a tad unreasonable to me.

I like Jeff Harvey's response ... should we discount the Burnham study because JHU gave him a slap on the wrist, and start believing the proven liars?

For the last time...

Data recorded by Lafta is X. It then gets passed through a signal processing box Y by US L2 team, producing result Z (i.e. 650,000 plus/minus...). Hence Lancet article.

Process Y accounts for MSB, deduces R is near 1, and discards any possible effects of MSB.

What is Y?

The polyonymous sockpuppet: "For the last time..."

Another name change coming up?

Ben spews
"Everyone but you at Deltoid desperately wants the Lancet study to be true"

No mister rat, we don't want it to be true. We desperately want it all to be false. The invasion of Iran, the bungling of the occupation, the torture of the detainees, the bankrupting of the country, the failure to regulate that led to the current depression, in short: the âelectionâ of the moron. We desperately want to wake up some morning and discover that it that it was all a bad dream.

But unlike the people on the other side, we are the reality-based community. We donât have the option of pretending the earth is flat. We believe that the only way to make rational decisions is to see the world the way it is rather than how we wish it was. And in the lies of Kane and Ben, and what ever the sock-puppet is calling himself today, we see exactly the cause of our 8-year long national nightmare, and we are revolted, and determined to prevent that evil from rising again in this once great country.

Kevin asks:

I've seen no report of Burnham calling Lafta anything of the sort. Do you have a link, other than to your own blog?

Don't you check the links on my blog? This is from Baltimore Sun reporter Stephen Kieh. (But maybe he is just another member of the neocon conspiracy.)

Burnham said he asked his Iraqi partners and was told that the names were not complete, which he accepted. But Hopkins, in its investigation, found that the data form used in the surveys was different from what was originally proposed, and included space for names of respondents. Hopkins found that full names were collected.

Full names were collected. Burnham claims that "his Iraqi partners" told him the names were not complete. Ergo, Burnham is accusing his Iraqi partners of lying. Now I believe (but don't have a link) that Lafta delivered the papers to Burnham and was the primary/only contact for him. It seems obvious that Lafta (his co-author!) is one of the "Iraqi partners" that Burnham is accusing of lying to him about whether or not the names are complete.

David,

Are you really so clueless about the Arab world that you donât know what Burnhamâs Iraqi partners probably meant by a âcompleteâ name? Taking the example given by Wikipedia, a manâs complete name, as far as Riyadh Lafta is concerned, might be Abu Karim Muhammad al-Jamil ibn Nidal ibn Abdulaziz al-Filistini. Hopkins might reasonably regard just a fraction of that as a âfullâ name.

What mattered to Burnham is whether any individual could be identified and you have produced no evidence that Lafta misled Burnham on that score, nor that Burnham accused him of doing so. But then absence of evidence has never restrained you from smearing people, has it?

Rathergate writ small.

Clearly the fact that some surveyers collected on "many" occasions some extra information indicates that the data collected is not reliable.

Clearly a self-described ["professional statistician"](http://scienceblogs.com/deltoid/2007/07/david_kane_on_lancet_confidenc…) who [doesn't know what a confidence interval is](http://scienceblogs.com/deltoid/upload/2007/07/KaneLancet.pdf), who [cannot calculate crude mortality rates](http://scienceblogs.com/deltoid/2007/08/robert_chung_on_david_kane.php), who [disseminates false documents](http://rds.yahoo.com/_ylt=A0oGkmTjNqRJYjQA3DNXNyoA;_ylu=X3oDMTEyZW5oMmY…) and who attempts to use apologetics for mass murder as vehicle for self promotion is the appropriate person to make accusations of fraud.

Kevin,

I am happy to walk you through this slowly. The Baltimore Sun reported that "Hopkins found that full names were collected."

Do you believe that this is a true statement? I hope so. Your claim then is that, while the survey form said (probably for hundreds of names!) things like "Nouri al-Maliki" or "Iyad Allawi," that Lafta might claim (truthfully?) that these names were not "complete" and that Burnham might accept this claim even though the names were clearly enough to identify specific survey respondents?

I just want you to explain the exact time line to me. Lafta comes to Jordan from Iraq with 1800 forms. Burnham looks at them and says, "Hey! These seem to include people's names. You weren't supposed to do that." Lafta replies "No worries! The names are not complete." And then Burnham says . . .

I just can't figure out how you think the conversation went.

The defining cognitive trait of the denialist is cherry-picking. In particular, they will seize upon any error, omission, or inconsistency (and virtually any study has some flaw, if you look closely enough), no matter how irrelevant to the conclusions, as a rationalization for rejecting conclusions they don't like. One may recognize the denialist in the wild by his characteristic cry, "If this was wrong, how can we trust anything from the same source?"

I tell you, it is now unquestionable that the US invasion is an unqualified success. Why? BushCo's policies only increased deaths by 400k, not 650k.

THAT, my friends, is successful policy we can believe in!

Best,

D

What Dano said. Remember back when the war supporters would attack Iraq Body Count's numbers as too high? Now we've got surveys and polls that suggest numbers from 2 to 10 times higher than IBC.

I am, though, doubtful we'll ever know the true number. Maybe one of the experts could tell me if it would be worthwhile doing another survey now. As a layperson it seems to me that with 4-5 million people displaced it'd be pretty difficult to do a survey that could determine the death toll now.

Boris at #34: Good question. The transcript is not accurate on that point. Here is the key sentence.

There were limitations to record keeping; we had some criticism that we could not produce a record showing which households were visited and who names of people were, and so forth.

I will correct my own post soon. Don't think the quote is accurate? Check the video yourself.

Burnham told us that names were not collected. We believed him. But names were collected. What else did he tell us that wasn't 100% accurate?

David Kane: I just want you to explain the exact time line to me.

Fine, here it is, by comment number:

#30: You claim Burnham is now calling Lafta a liar.

#31: I ask for evidence.

#40: You refer to a story which Tim has already quoted in his update.

#41: I point out that this story does not support your claim.

#42: You wriggle and twist, as is your custom in such cases.

Just to spell it out for you, David â on the off-chance that you really are as obtuse as you pretend to be; the position Burnham is in is this:

(1) Riyadh Lafta, who knows about Iraqi names, told him that the names on the forms were not sufficiently complete to identify the individuals concerned;

(2) The Hopkins investigation, presumably after seeking advice from people who can read Arabic, judged that âfull namesâ were collected.

Maybe Burnham thinks Lafta did the dirty on him. Maybe he thinks Hopkins got bad advice. Maybe he has no opinion on the matter. We donât know. We do know that he has accepted responsibility for what happened and, so far as I know, he hasnât made any attempt to shift the blame onto Lafta or anyone else. Which means that your claim, that Burnham is now calling Lafta a liar, is simply false.

Burnham told us that names were not collected. We believed him. But names were collected. What else did he tell us that wasn't 100% accurate?

Interesting that the only Burnham quote you can produce in support of this is "There were limitations to record keeping; we had some criticism that we could not produce a record showing which households were visited and who names of people were, and so forth."

I can't help noting what that quote does not say. In particular, it does not say that no names were collected whatsoever.In fact, it doesn't actually make any claim whatsoever about what Burnham does or does not have; it reports what critics said. I can see multiple ways in which Burnham's statement could be 100% accurate:

1. Some full names were collected, but not all, making it impossible to assemble such a record.
2. Names were collected, but some were not full names, making it impossible to assemble such a record.
3. Some names were collected, but not associated with households.
4. Complete name and address information was collected, but Burnham felt unable to make it public due to ethical obligations to the study subjects.

You see very strongly motivated to make Burnham out to be a liar.

Kevin,

1) So, you agree that I am reasonable in asserting that "Iraqi partners" includes Lafta? (I just want to confirm that we agree on that.

2) We also agree that we can rely on the Baltimore Sun reporting that:

Burnham said he asked his Iraqi partners [read: Lafta] and was told that the names were not complete, which he accepted. But Hopkins, in its investigation, found that the data form used in the surveys was different from what was originally proposed, and included space for names of respondents. Hopkins found that full names were collected.

You agree that "full names" were collected?

3) So, if we agree that "Iraqi partners" include Lafta and we agree that the Baltimore Sun can be trusted when it reports that "full names were collected," then I am not sure what the quibble is. Do you really intend to make a big distinction out of the difference between "complete" names and "full names?"

In other words, the name "Kevin Douglas" may not be "complete" since it misses your middle name, but it is more than enough (95 times out of a 100) to identify you if I am driving around your neighborhood.

You claim that "Riyadh Lafta, who knows about Iraqi names, told him that the names on the forms were not sufficiently complete to identify the individuals concerned." Let me grant that for a moment. Isn't this clearly a lie? If I am walking/driving around a neighborhood and I a sheet of paper with names like "Nouri al-Maliki" or "Iyad Allawi", isn't that more than enough information (at least 95 times out of 100) to identify specific individuals in that neighborhood? So, to the extent that Lafta said that it wasn't, he was lying.

Of course, once you are out of the neighborhood and/or out of Iraq, a name like "Nouri al-Maliki" or "Kevin Douglas" may not be enough to find the person. But that was not the concern expressed in the protocol, obviously! The concern was that militia or soldiers would intercept the interviewers in the field and attack respondents.

trrll: Are you serious? I guess it depends on what the meaning of "is" is.

Anyway, consider another example. Let's go to the official Q&A from the Hopkins page, since removed for obvious reasons.

Approval specified that no unique identifiers would be collected from households visited by researchers, including complete names, addresses, telephone numbers or other information which could potentially put the households at risk. While household demographics were collected in both Iraq studies, personal information such as the date, location and cause of death was collected only for deceased household members. Research regulations do not consider a dead person to be a human subject and informed consent is not required for uniquely identifying information on the characteristics and circumstances of death. Informed consent was obtained from the principal respondent in each household before interviews were conducted.

"Personal information" was, we now know, collected for some living household members. This was a lie.

Now, it is true that Burnham's name does not appear on this Q&A. Perhaps some low level Hopkins' functionary was just making things up! My bet, however, is that this Q&A was run by Burnham.

David Kane at 50.

Regarding complete name vs. full name, are you dense? What do you make of the name Robert ap Robert? In the USA, this was my English prof's full name, in Welsh it translates as Robert son of Robert, which is similar to the conditions in the Muslim world where full names contain the geneological history of the person and in that manner are considered complete names. Bring that to the local level, even in small towns in Wales, there are several Robert ap Robert. That is the distinction.

With that distinction in mind, there is no issue with Lafta's claim of not including complete names, and the Johns Hopkins finding that some full names were obtained. My interpretation is that the issue centers around the fact that any names were collected at all, and if that is the case, then clearly Burnham is at fault, for making that field decision to include first names. But that does not make Lafta out a liar.

Mike

Kane is a political hack and internet troll masquerading as an academic.

The only long-term answer to a dittobot of his low ilk is, frankly, Greasemonkey killfiling. Period. The which I've done.

And this comment exchange establishes that nicely. I've seen far better discussions with Young Earth Creationists.

And for an academic and public fraud like him to call Burnham a liar, repeatedly, is simply proof that he sees research as an extension of political campaigning by other means.

David,

Youâre just squirting squid-ink. Your contention is that Burnham is calling Lafta a liar. You think the Baltimore Sun report implies that Lafta lied to Burnham. But you arrive at that conclusion by a chain of reasoning which is peculiarly your own. There is no reason whatever to suppose that Burnham thinks as you do.

We now know one thing for certain.

Burnham is far more credible than John Lott.

He actually has raw data which wasn't eaten by a computer, or a ratel.

Burnham's actions were ethically dubious, but they confirm the trustworthiness of his science.

Claiming an exemption from human subjects review on account of no names being collected, then allowing the collection of names, is exactly the kind of oversight-evading switcheroo that will make an institutional committee come down on you like a ton of bricks.

It's a big risk to take, and I think it confirms the claim that Burnham was striving to get high quality data. If he were just making shit up, like John Lott has been repeatedly accused of doing, there would have been no reason for him to keep anything more than the minimal record.

Two scenarios.
1) Some names were collected. When these forms get to America, Burnham ignores them as they are not valid data for the study. He is ethically prevented from using them if they exist in some cases. Thus in public descriptions of the study he states that no identifying data was taken. This is true in the sense that any potentially identifying data must be ignored or destroyed.
2) Some names are collected. Burnham gleefully wrings his hands at sneaking this past the IRB, lies to everyone, and keeps this data in a secret stash that he can look at every night, gloating over the fast one he has pulled on the world.

Which scenario seems likely? Which will David Kane promote?

And what does this have to do with the results of the study, which presumably do not depend on any of the names that might have been collected?

Upon reflection, I think that Kevin Douglas (and others) are correct. If Burnham were participating in this thread, he would not claim that Lafta lied to him. I retract my (implicit) claim to the contrary in #40 and elsewhere.

Tim writes in #14: "David Kane will, no doubt, now apologize to Burnham." I am honestly confused. What should I apologize to Burnham about?

David, your seizing on this names business and the farcical Burnham called Lafta a liar bit is a real mask slipping moment. Just fyi. Beforehand, I hadn't paid sufficient attention to this debate to know whether you or your claims were worth taking seriously. Now, well, as consistent Kane's Law of Credibility by which one instance of unprofessionalism implicates every other assertion one has or will make, I can safely add u to the killfile. Bye now.

Burnham is now calling Lafta a liar. Who should we believe?

David the defender? Making this claim about Burnham? i am shocked!

So when are you going to start pursuing the real criminals, David? If you are correct, there may thus 'only' be 100,000-300,000 corpses in Iraq as a result of the illegal war. Is this an acceptable total?

the problem with this is, that David thinks that the real number of excess dead in iraq is 100000 +-100000.so yes, he really considers the possibility of ZERO excess deaths to be a real one. (he has to, his bogus fallujah claims fall apart if he doesn t)

and yes, this means that he claims that IBC is doing an OVERCOUNT of dead in newspapers. (everybody with any knowledge will tell you, that they do an UNDERCOUNT)

ouch David. let me remind you of your post #1:

As always, I am a defender of Burnham.

and now i scanned you blog post, just to find this part:

Why did Burnham lie about this to so many news organizations?...I thought that this was just going to be one of many examples of where Burnham lied,

is this your very special way, of being a defender? always?

i can t shake the feeling, that many pretty common terms (lie. defender. always.) have a very special and different meaning for you. might explain a lot of our different opinions. is everything OK in your own little world?

uuups, missed this one: (the TITLE of David s post)

Burnham Sanctioned ... I declare victory!

i actually have problems of getting this together with the image of an adult person.

but getting it together with person who claimed to be "always ... a defender of Burnham", just 14 hours ago, is simply impossible.

We are, once again, reduced to spending a lot of time arguing about intessentialities, by which I mean points that pro and con would be blown away tomorrow if Obama would today turn to the Iraq commander and say "Can I have the Army's estimate of the number of Iraqi deaths?"

The best argument in favor of the Lancet figures has always been that if the American government had really thought the figures were vastly inflated they would have shown them up by doing their own calculations. The fact that they didn't do this - or wouldn't admit publicly that they had - shows that they believed the figures were in the ballpark. And they ought to know, within limits (that is, I'm sure that any estimate they have would be deliberately calculated to be very much low end; I just don't think they would be able push it very much below half a million even so).

The fact that Obama hasn't asked for the figures to be released is disappointing; if I was an Obamatron I'd speculate that he's using it as blackmail to stop the army undercutting him on the withdrawal.

Not to derail the thread, but could someone help me out with this topic.

The Bloomberg statement includes:

The paper in The Lancet incorrectly stated that identifying data were not collected. An erratum will be submitted to The Lancet to correct the text of the 2006 paper on this point.

I am confused about how this is going to work. The only relevant statement that I could find in the paper was:

The survey purpose was explained to the head of household or spouse, and oral consent was obtained. Participants were assured that no unique identifiers would be gathered.

For all we know, this is a true statement. Perhaps this is what participants were told, albeit incorrectly. If that is so, then a hard core Lancet defender, like Les Roberts, might maintain that no correction is necessary. In fact, don't all the authors of a paper need to agree on any correction? I suppose that a single author, like Burnham, can say whatever he wants, but I wouldn't think that a single author has the right to make an official change unless the other authors agree. At most, a single author can just demand that his name be removed from the publication. And, say what you will about Les Roberts, but he is a head-strong fellow. Not that there is anything wrong with that! What if Burnham wants to make a correction but Roberts doesn't?

Perhaps there is some other reference in the paper that I have missed.

Moreover, wouldn't this also be a good time to correct the mistaken description in the paper of the sampling scheme. After all, even Roberts/Burnham admit that it is not accurate.

Anyway, I don't have a strong opinion on what will happen. I am just curious what others predict . . .

... Maybe the Lancet erratum will also include a clear statement of the magic procedure Y used to dismiss any street bias (see below):

Data recorded by Lafta is X. It then gets passed through a signal processing box Y by US L2 team, producing result Z (i.e. 650,000 plus/minus...). But somehow, this mysterious process Y allows the deduction that there is no possible street bias in X, by quantifying it as negligible using some unknown quantitative model, and hence allows L2 to ignore even mentioning the possibility of street bias in the Lancet article.
So what on earth is this miraculous Y data procedure?

Shorter David kane:

Perhaps if I offer enough imaginary vile scenarios, all qualified by the word "perhaps,' no one will notice that I am doing character assassination by making shit up.

LancetSTudy (and all your imaginary friends)

You keep spamming that straw-man question many times all over Lambert's blog,so I'll go ahead and repeat here the answer I gave in the other thread.
----
Lancet Study ( and your various sock puppets):

You keep asking what "marvellous Y data procedure" ... "allows the deduction that there is no street bias."

That is a deeply disengenious and dishonest question. The fact is that THERE IS NO EVIDENCE THAT THERE IS ANY STREET BIAS!!!!!

You guys are making this shit up.

N does not equal 10. Or anything close to it. As I've aid over and over now - the lancet authors stated as a design criteria (not a desired goal, but a criteria upon which they based their sampling design) that every house in Iraq should, to use your language, be in the sampling frame.

Only if one ignores this statement (or acknowledge it and assume Lancet2 to be either massively incompetent or fraudulent), and only if one ALSO goes out of one's way to exclude obvious main streets from main street category, and ALSO does so in a way that is clearly designed to exclude massive quantities of homes from the survey - only if one does all that, can one get a value for N that has any appreciable impact on the Lancet2 analysis.

Stop being despicable.

DK:
>Tim writes in #14: "David Kane will, no doubt, now apologize to Burnham." I am honestly confused. What should I apologize to Burnham about?

Accusing him of lying.

Tim,

You and I often do a good job of getting to the bottom of empirical issues. Recall our work last year on the topic of demographic data and L2. So, perhaps we can get to the bottom of this as well! Here is my current summary of the facts. Are there any mistakes? Highlight:

-----------------

Consider Burnham's February 2007 talk at MIT. Burnham claimed that:

There were limitations to record keeping; we had some criticism that we could not produce a record showing which households were visited and who names of people were, and so forth.

We intentionally did not record that, because we felt that if the team were stopped at a checkpoint, of which there are lots of checkpoints, and the records were gone through, some of you may have had this experience, where you stop at a checkpoint, people go through all your papers, read everything, and they find certain neighborhoods. That might have increased risk, which we didn't want to do.

That's a lie. And Burnham knew at the time that he was lying to his MIT audience. There are other examples. Consider the "The Human Cost of the War in Iraq" (pdf), the official companion piece to Burnham et al (2006). It claims that:

The survey was explained to the head of household or spouse, and their consent to participate was obtained. For ethical reasons, no names were written down, and no incentives were provided to participate.

Since names were written down, this statement is not true. One might defend Burnham's misstatement at the MIT lecture as being an honest mistake, made while speaking off the cuff. But here we have a formal report with a statement that Burnham knew was both untrue and important. (If it were a trivial issue, then Hopkins would not have sanctioned him and no correction to the Lancet would be necessary.) How many other untrue statements are there?

-----------------

Do you dispute any of the facts/quotes here? And, if not, is your objection to the word "lie"? You would prefer that I, say, accuse Burnham of "purposely misleading" his audience?

As always, if I have made a mistake here, I am eager to correct it.

David Kane lost his credibility about the same time I lost my virginity, and I turn 55 tomorrow.

Kane and the sockpuppets are vile. Göbbelesque.

Lee is right.

That is a deeply disengenious and dishonest question. The fact is that THERE IS NO EVIDENCE THAT THERE IS ANY STREET BIAS!!!!!

the evidence supporting that people getting killed in "mainstreet areas" are living there, does not exist.

the Spagat paper contains a hole, that allows oil tankers to pass through it...

ChrisB: Excellent post. You hit the nail on the head. The real fear among those who defended the invasion are that real, reliable figures of the death toll in Iraq will become publicly available. Once we have an accurate figure, then it will create outrage, because it will reveal the true extent of the horror. But as long as it 'may be, could be, probably isn't, is too high, too low, etc.' then the media and government PR machinery can manage the critics.

Note how Iraq Body Count was heavily criticized by many who supported the war until the Lancet studies were published. I remember reading bitter attacks on them early during the conflict by many pro-war pundits. Then suddenly, overnight, the IBC estimates became 'gospel' in the eyes of Bush, Blair and many of their supporters in the corporate media. This is because the IBC counts suddenly made the conflict look less bloody than the Lancet estimates. Again, it all relies on the tried and trusted PR strategy of 'managing the outrage'. Edward Bernays would have been proud, I am sure.

Most of us would post the conclusions first, then bore us with the basis of the conclusion.

Tim, not being trained as a scientist, bores us with the trivia, thereby hoping we don't notice some inconvenient facts.

Are Tim (and other members of the Deltoid community) upset that Burnham lied to us. This is Burnham writing to Tim.

As far as the survey forms, we have all the original field survey forms. Immediately following the study we met up with Riyadh (in this very hotel I am now in) and Shannon, Riyadh and I went through the data his team had computer entered, and verified each entry line-by-line against the original paper forms from the field. We rechecked each data item, and went through the whole survey process cluster-by-cluster. We considered each death, and what the circumstances were and how to classify it. Back in Baltimore as we were in the analysis we checked with Riyadh over any questions that came up subsequently. We have the details on the surveys carried at each of the clusters. We do not have the unique identifiers as we made it clear this information was not to be part of the database for ethical reasons to protect the participants and the interviewers.

Is there a single person at Deltoid who, after reading that paragraph, thought to himself: "Well, they may not have "unique identifiers" but I bet that they have the names of the participants."

I doubt it.

If I were Tim, I would be getting somewhat upset about now . . .

David, for the millionth time, I am not upset that Burnham 'lied' to us. I don't think he did; besides, I don't believe there has been a huge human cost because of the publications of Lancet's Part I and II.

I AM extremely upset, however, that Bush, Blair and their inner circles brazenly lied to generate public support for an illegal war that led to carnage and unimaginable levels of destruction of the civilian infrastructure in Iraq; lies in pursuit of a brazen political and economic agenda that should be obvious to anyone.

My advice to you: get your priorities straight. Whose lies have caused the most suffering by a country mile?

To Lee and company. I have a new question:

**Data recorded by Lafta is X. It then gets passed through a signal processing box Y by US L2 team, producing result Z (i.e. 650,000 plus/minus...). But somehow, this mysterious process Y allows the deduction that there is no possible street bias in X, by quantifying it as negligible using some unknown quantitative model, and hence allows L2 to ignore even mentioning the possibility of street bias in the Lancet article. So what on earth is this miraculous Y data procedure?**

Hmm, I now realize it is the same question as always. Since you never answer it, and yet it addresses what makes L2 valid or not scientifically, maybe you should.....

David, Burnham was mistaken about the collection of last names. This does not make him a liar. The Johns Hopkins report did not find any dishonesty on the part of Burnham. Are you now claiming that the report was a cover-up?

A special message to people who agree with Lee:

Lee claims there is no evidence for street bias. There is certainly no evidence that there is elephant bias, or taco bias etc... But that is because elephants and tacos were not involved either in the design, survey implementation, or the violence itself, as far as I know.

However, streets were involved in all 3. Jody tells us that the design would have incorporated any STREET bias effect (in some way that she does not explain, or know..), the survey involved people walking on a STREET to houses on one STREET or another, and the violence occurred on a STREET. And the common word is...? Yes, STREET.

So the doubt about street bias is reasonable, and it exists, and the issue should be addressed by the scientists which dismiss it. Just saying 'no, it doesn't exist' is not a scientific argument.

I have to disagree with something Tim said earlier. I can't believe that LS is one of the study's authors, unless he/she is deliberately feigning cluelessness while in sock-puppet mode.

LS keeps asking what "Y" is. Wonder if LS ever heard of Occam's Razor? "Plurality should not be posited without necessity."

One should verify that something exists before trying to quantify it. LS believes that "Y" represents the discrepancy between what the survey team said they did, and what they actually did. I'd suggest that LS first needs to provide hard evidence that this discrepancy exists.

I believe that "Y" exists, but that it actually represents the discrepancy between what LS thinks he understands, and what he actually understand. As evidence for the existence of "Y," I submit to you... all of LS's posts.

Bruce wrote: "..LS believes that "Y" represents the discrepancy between what the survey team said they did, and what they actually did. .."

No. See my post 77:
It is what the US L2 team did to the raw survey data X, to produce the Iraq population-level conclusion Z.

...by which I mean:

**What quantitative analysis did Burnham/Roberts do to correct for (or discard) possible street bias during the survey?**

Tim writes:

David, Burnham was mistaken about the collection of last names. This does not make him a liar. The Johns Hopkins report did not find any dishonesty on the part of Burnham. Are you now claiming that the report was a cover-up?

I do not think that the report was a "cover-up."

Just so we have the facts straight. We agree:

1) Burnham wrote to you that "We do not have the unique identifiers"

2) At the time he wrote that, he had 1,800 plus survey forms in his office. Some of those forms included "full names" (per the Baltimore Sun).

Your claim is that Burnham, at the time he wrote you, was "mistaken" and thought that none of the forms had full names. It was only later that he realized his error.

This is a reasonable defense and, if correct, means that I should retract my claim about "lying." But doesn't it come down to the details of just what sorts of names were on those forms? For example:

If most/all the names were written in Arabic, you are correct. Burnham was simply mistaken because he does not read Arabic.

But wouldn't you concede that if most/all the names were written in English (and were names like "Nouri al-Maliki" and "Iyad Allawi") then I am most likely correct? If Burnham, at the time he wrote you, had hundreds of survey forms which featured the full names of Iraqis in English, then he must be lying (not simply mistaken) because any reasonable person would conclude that such names identified specific people, especially in conjunction with the specifi cluster?

My goal here is to determine your distinction between lying and being mistaken ahead of time. Then, once we find out the actual details, we will know the answer.

My guess (and I have seen no clear evidence one way or the other) is that Burnham had hundreds of survey forms with full English names. I have not proved that yet, but it is worth it to be to get to the bottom of that if it will cause you to change your judgment from mistake to lie.

LS,

It is a simple matter to hypothesize the possibility of bias. One need only make the assertion. To demonstrate a plausible mechanistic theory that such a bias actually exists is something more difficult. You haven't even made an effort. You ignore the issue as if it was immaterial, but it is the material issue.

Burnham, et al., make the explicit correction for unknown and unknowable biases by the large range of uncertainty given in the study.

Your whole argument is a tautology. What is known as suffering simultaneously from diarrhea of the mouth and constipation of the brain.

"My goal here is to determine your distinction between lying and being mistaken ahead of time. Then, once we find out the actual details, we will know the answer."
This is untrue. Kane's MO is to make the baseless, ideologically-driven, assertion first ("fraud"! "liar"!). And only after those assertions have circulated does he start to consider whether they are empirically accurate. And his assertions are never empirically accurate, which he never admits.
He will never recognize any of this and apologize. He will simply continue to play the game. He should just shut up...

LB wrote:

1. "You haven't even made an effort."

JPR did.

2. Burnham, et al., make the explicit correction for unknown and unknowable biases by the large range of uncertainty given in the study."

That is ludicrous!! How can you correct for biases you haven't even recognized as possible!!

L2 addresses biases at the larger scale of clusters, but not at the street level.

So....

What quantitative analysis did Burnham/Roberts do, either post-survey or pre-survey, to correct for possible street bias during the survey?

street, **street**, _street_, STREET....

I'm more bothered by the fact that names were collected in any level of detail than by Burnham's statements at the MIT talk. I don't understand why they would have done something so stupid, but once it was done, it's not clear to me that you should announce this to the world. It might not have been safe for the people back in Iraq (both the team and the interviewees) if it was known that names were taken. For all anyone in Iraq knows, Lafta might still have the information stored somewhere. Partial names or full names, someone with an electric drill might get ideas.

So David, the only justification you are able to offer for your accusation against Burnham is a "guess". You really are not doing yourself any favours with this sort of behaviour.

1) Well, you are probably right that I should tone down in my language a bit. I will update my post accordingly.

2) If we are going to have a conversation, it would be helpful if you made your position clear. How many full names would need to be present in the forms for you to agree that Burnham is lying and not simply mistaken? Or is there nothing that Burnham could have known when he wrote to you that would make you think him a liar?

3) Perhaps "guess" is not the right word. Perhaps we should have a wager?

I bet that a majority of the names were in English and not Arabic.

[The interviewers were all fluent in English. The forms (I think!) were in English. The interviewers knew that the Doocy/Burnham did not speak Arabic. The Sun reported that "Many [of the names] were in Arabic." I think they got this info from Burnham and that if a majority were in Arabic, he would have mentioned that.]

I bet that there were hundreds of names listed.

[Johns Hopkins talks about "many of the records" having names. If this only happened a few times, they might have ignored it. Once at least one team is writing down names, it would make sense that they would do this all the time, not just for random clusters.]

I bet that hundreds of names were "full names" (from the Sun), meaning enough of the name to identify specific people.

So, although I do not have proof, I would wager that Burnham had hundreds of Iraqi names like "Nouri al-Maliki" and "Iyad Allawi" on the forms in his office. If that is, in fact, the case, would you agree that Burnham lied to you?

I'm one of "David Kane's" strongest defenders on this blog, but I wonder how he accounts for possible concerns that he was boiling children he had previously drained of blood for Satanist/Marxist rituals when he posted his "analysis". I also have not heard a satisfactory explanation of the possiblity that "Kane" cannot do simple arithmetic, believes in the Hollow Earth, and is an albino.

Just the facts, please!

The interviewers knew that the Doocy/Burnham did not speak Arabic.

So if the interviewers knew that Doocy/Burnham did not speak Arabic and also didn't want the interviewers to record names they would have written the names in English?

Hey, as long as you get to cry "fraud!" it seems likely, right? That's the important point to remember.

Marion, i think you are too fast.

shouldn t we first make our point clear? how many children could Kane boil, without us calling him a liar?

One again, LancetStudy (and your army of mousy sock puppets) - bullcrap!

"Only if one ignores this statement (or acknowledge it and assume Lancet2 to be either massively incompetent or fraudulent), and only if one ALSO goes out of one's way to exclude obvious main streets from main street category, and ALSO does so in a way that is clearly designed to exclude massive quantities of homes from the survey - only if one does all that, can one get a value for N that has any appreciable impact on the Lancet2 analysis."

See that? Ive posted it several times now in response to your spammed-all-over straw man question.

If N has a value much smaller than 10, then street bias does not alter the Lancet2 conclusions in any appreciable way. This is true also of several other of their parameters, which have been adressed in other threads.

Lambert's post that started the other thread is an answer to your oft-spammed question. Lancet 2's authors statement that a sampling design parameter was that every house in Iraq be in the sample frame, ADDRESSES YOUR F****** QUESTION!!!!!!!!!!!!

Lee, DNFTT please. It encourages them.

David Kane @75, quoting Burnham:

Kane, that quote supports Burnham. It describes them takign the database, crosschecking it against that field entries, abd then Burnahm taking the database, and when necessary checking with Lafta for further details.

And Burnham says had he didn't have unique identifiers because they were not entered into the DATABASE. Please, Kane, read the stuff you post.
--
I think this violation of IRB approved details is a big ethical error, and needs to have real consequences - and it has. Those confidentiality requirements were there to protect lives.
But his has NOTING AT ALL to do with the actual data and its analysis, or the conclusions of Lancet2. At the data/analysis/results level, it is yet another way that Kane and others can use character assassination to try to discredit Lancet2.

sod, just to be clear, i am not raising the claim that Kane, a fine researcher and a credit to his community, is doing that. I am simply mentioning concerns and hoping politely that they'll be addressed.

As the great Peggy Noonan said:

Was Mr. Clinton being blackmailed? The Starr report tells us of what the president said to Monica Lewinsky about their telephone sex: that there was reason to believe that they were monitored by a foreign intelligence service. Naturally the service would have taped the calls, to use in the blackmail of the president. Maybe it was Mr. Castro's intelligence service, or that of a Castro friend.

Is it irresponsible to speculate? It is irresponsible not to.

Lee writes: "If N has a value much smaller than 10, then street bias does not alter the Lancet2 conclusions in any appreciable way..."

That is your analysis, or Tim's. Not the L2 authors. The question is: Why didn't they do a street-bias analysis? Where did they do it. Do they reach the same conclusions as Tim? If so, how prior to JPR, if they didn't have the JPR formula? Did they derive the JPR formula and plug in numbers? What numbers did they plug in? Which map? Which streets? The streets that Lafta generally picked?

Questions, questions.... 'irrelevant' questions? No.

Lee, my friend, grab one of your tacos from your main-street-defining-taco-trucks, and read the following very, very slowly while munching...

The survey involved people walking on a STREET to houses on one STREET or another, and the violence occurred on a STREET. So how, where and when did Burnham/Roberts manage to quantify and discard any street-bias effect?

and the violence occurred on a STREET

Citation please, for this extremely odd assertion that violence in Iraq occurred exclusively on streets. What dataset is the basis for this claim?

The survey involved people walking on a STREET to houses on one STREET or another, and the violence occurred on a STREET. So how, where and when did Burnham/Roberts manage to quantify and discard any street-bias effect?

this is getting slightly stupid. all house to house polls happen (along) streets.

you are still using the term "street-bias" as if it was a fixed term. it isn t. it is a phenomenon, that most likely doesn t exist.

dsquared wrote: "...Citation please, for this extremely odd assertion that violence in Iraq occurred exclusively on streets.."

Oh dsqaured ... you fool. No 'exclusively' word appeared in my post.
(Anyway, do you have details about events in trees or in the air....?)

> Lee, DNFTT please. It encourages them.

Marion Delgado is onto something - mocking seems like the only useful response to Kane et al.

Lord is our Spaghetti stupid. Will this do, clown Or maybe this, which clearly happened on a non-main street

That second photo is a Main Dirt Street. Expect arguments expounding on the MDSB in L2 shortly.

It has been pointed out that the hanged guys were hanged by Sadaam. However Eli wishes to point out that it was still a death in the air in Iraq. There are other examples, including a youtube clip of Sadaam dangling.

Were the survey forms filled out in pencil or pen? I can find no information regarding this in the public spehere.

Why is Burnham hiding this information? If a pencil, was it HB?

How can other researchers have confidence in these results if we don't even know something so basic about the study methodology. Without this information, it's impossible to correct for HB bias.

If we don't know how Burnham accounted for potential HB bias, the Lancet Study is a fraud.

This much is clear.

Could someone perhaps lure Harold R Pierce over here? This thread could use a little sanity.

Could someone perhaps lure Harold R Pierce over here? This thread could use a little sanity.

Post of the week!

Guys, do the decent thing and read my sidebar that helped launch this Hopkins investigation. The sidebar can be found at
http://news.nationaljournal.com/articles/databomb/sidebar2.htm.

It includes this line from Burnham;; "...The survey "was carried out as we designed it," Burnham told National Journal..."
Does anyone want to try to make that statement comport with Deltoid's preferences and Hopkins' conclusions?

Yours, waiting for the next shoe to drop.

Neil Munro

Given his conduct, [Neil Munro is hardly in a position to talk about doing the decent thing](http://scienceblogs.com/deltoid/2008/01/flypaper_for_innumerates_natio…).

You mean this one?

Inclusion of identifiers did not affect the results of the study.

How did the collection of some first names (as apparently Burnham believed was the case at the time) affect the *design* of the survey, which I at least would interpret to mean the way in which households were chosen, the structure of the survey questions, the protocol, and analysis of results?

> Yours, waiting for the next shoe to drop.
>
> Neil Munro

Always a sensible policy when slithering around in the mud.

Tim in #110: What a great thread you link to! Shall we revisit all the items that you got wrong and Munro got right? I recommend starting a new thread devoted to that topic. A fun time is guaranteed for all.

Guys, do the decent thing and read my sidebar that helped launch this Hopkins investigation. The sidebar can be found at http://news.nationaljournal.com/articles/databomb/sidebar2.htm.

Neil, i just took another look at your article. it is still misleading, full of baseless accusations and false claims.

starting with the very first paragraph:

Three weeks before the 2006 midterm elections gave Democrats control of Congress, a shocking study reported on the number of Iraqis who had died in the ongoing war.

the "number of iraqis who have died in this war" has been confirmed by the IFHS study.

but i did only scan your article. and i find it interseting, that you are another one, who feels confirmed by the Lancet review. (and another one, who started the investigation..)could you please point out the part about NAMES in your writing/article? i must have missed it!>

and i would like to hear your opinion on this part (you missed it so far?) of the review:

The original forms have the appearance of authenticity in variation of handwriting, language and manner of completion. The information contained on the forms was validated against the two numerical databases used in the study analyses.

Sod, setting aside the question of whether or not it says anything about the results of the survey, Munro's sidebar does mention the names, about halfway down:

"Burnham and his colleagues have frequently said that their Iraqi surveyors did not record names. But the Iraqi researcher who directed the survey may have used an answer form that had spaces to record names, with or without Hopkins's approval. In May 2007, the chief of the Iraqi survey team, Riyadh Lafta, gave a copy of what he said was his form to Ali Mohamed, a United Nations official who tracks deaths in Iraq, Mohamed told National Journal. On the form that Mohamed provided to NJ, the top line has spaces to record the location of the survey and the 'name of householder.' The form also has spaces to record the names of infants and deceased people."

Regards,
Bruce

thanks Bruce,

his link doesn t work for me, and the article didn t contain that information.

[here](http://news.nationaljournal.com/articles/databomb/sidebar2.htm#) is the article. Munro did indeed raise the question about names. my apologies to him.

i still don t see, why somebody who "even faked responses" would violate ethic rules and add names, but i guess that is just me..

Hey David Kane, you are aware aren't you that full names do not constitute a unique identifier? Full name + date of birth is a unique identifier, as is Full Name + address.

There is nothing in this review of the study to suggest any level of fraud. Burnham certainly appears to have collected data in a way which is inconsistent with the IRB application, but this is not itself a sign of fraud or maliciousness, and I suspect that if one were to audit all human research studies one would find a lot more small inconsistencies like this.

I am, however, quite surprised that the JHU IRB exempts a study from full Human Research Ethics approval even if no identifying data is collected. It's still research on real people, with the attendant risk that it is frivolous, hurtful or exploitative. Do other people here think that is a strangely lax IRB?