More comments on dodgy National Journal piece

wow, both are very good pieces.

the commentary on the Munro article is a master piece.

John Tirman,

Your comments are interesting and important. But, despite Tim's best efforts, a blog is not an ideal medium for scientific dispute. So, why not a debate?

I would be happy to arrange a debate at Harvard between you and me on the accuracy of the Lancet surveys. Or, if you would like to have someone else to speak for the Lancet side, that would be fine. (Any of the authors would be great choices, but I realize that they are very busy.) Or, if you prefer, we could hold the debate at MIT. The room that Burnham used for his talk last year was excellent and the video production and distribution facilities first rate.

The very heart of the academic enterprise is open discussion and debate, especially between folks who disagree.

Any interest? My schedule is wide open.

Your comments are interesting and important. But, despite Tim's best efforts, a blog is not an ideal medium for scientific dispute. So, why not a debate?

Hilarious, a debate is even less suited for scientific dispute.

So, here we are, another ploy right out of the science denialist playbook. Just as Kane plays the "free the code" card thumped so heavily by McIntyre.

Tim, do you have a "you might be a crank if you ..." bingo card available?

I agree with many of John Tirman's comments. But his comment t23 on page 9 (which I am unable to cut and paste) is wrong. The authors have not "provided data sufficient to determine if the survey was conducted properly." But, opinions might differ on that. The next part, however, is simply false. (And, who is this "obsessive critic" that you are complaining about?) At least 5 people (that I know of) do not think that the L2 data is release is sufficient: Me, Stephen Feinberg, Michael Spagat, Safaa Amer and Fritz Scheuren. There are no doubt others. If we can't trust Tirman to report accurately on matters of public record, how can we be sure that he is correct about other issues that we can not independently verify?

If someone else could place that comment in this thread, I would appreciate it. My pdf skills are not what they should be.

David: I agree with dhogaza. In fact a blog is a much better forum for an exchange of ideas such as this where responders can think and research before they respond.

Of course the best place for an exchange of ideas would be to publish in a quality peer-reviewed journal.

John

In fact a blog is a much better forum for an exchange of ideas such as this where responders can think and research before they respond.

Which, of course, is why lying scum greatly prefer the debate format to any other.

David Kane: "If someone else could place that comment in this thread, I would appreciate it. My pdf skills are not what they should be."

OK...

"Comment [t23]: They have provided data sufficient to evaluate whether the survey was conducted properly, leaving out only the unique identifiers to protect the Iraqis interviewed. Many people, including critics, have seen this data, and essentially only one --- an obsessive critic --- has made a complaint about this matter."

It's two sentences. You don't need PDF skills. You just need to be willing to do fifteen seconds worth of typing.

David, mark this:

In the ethical review process conducted with the School's Institutional Review Board (IRB), Burnham and Roberts indicated that they would not record unique identifiers, such as full names, street addresses or any data (including details from death certificates) that might identify the subjects surveyed and put them at risk.

Have I not been trying to tell you this for a long time now? You can bet that in at least some instances, identifying data included the cluster number itself. Now do you understand why this data is "not available"?

And lo! and behold! on the next part of the release:

The researchers will not release data at the household, street or neighborhood levels, which could identify and put study participants at risk.

And then in the related link:

These data will be released on request to recognized academic institutions or scientific groups with biostatistical and epidemiological analytic capacity.

Which confirms my repeated claims to you that not all academics will have equal access to the data.

SG writes:

Which confirms my repeated claims to you that not all academics will have equal access to the data.

You have no idea what you are talking about. The data release has been identical to all outside groups who have received access. The IFHS folks have the same L2 data that I have. No more, no less. Don't believe me? Check with L2 author Shannon Doocy.

The issue is, has only "one critic" made a complaint? No. Tirman is misleading his readers. If we can't trust him to get these details (which we can easily check) correct, how can we trust him on other matters?

By the way, I agree that a debate is less than optimal. The problem with the blog is that it is easy for Tirman (and others) to refuse to respond to specific questions. They just ignore them. In a face-to-face debate, that becomes much harder to do.

Simple example: Why have you refused to make the L1 data available to the same degree as the L2 data? Another example: Why don't you release the computer code from the models? I have a long list . . .

But I could imagine a blog seminar of some sort, ideally hosted by Tim at Deltoid, in which we could thrash out these issues in some depth. I am ready, as are other L2 critics. Are the L2 authors?

A debate would only be fruitful if both participants were intellectually honest.

Without meeting that requirement, it would be nothing but a circus, which is probably what Kane is hoping for.

Not that I seriously expect you'd take him up on his offer, but just in case: it's a trap!

Tim does not link above to the Burnham & Roberts letter to the National Journal, some of which I agree with (like the Soros stuff). But some is just false. My Zombie Army is on the march!

David, no debate, sorry, one reason being that I am not competent in statistics. And yes, you are the main critic. When we published the survey, I polled many statisticians and epidemiologists, and many have seen this data since, and virtually all say (sometimes with mild qualifiers) that they have confidence in the method, researchers, data, and outcome. Newspaper reports generally found such people, too. There are individual nits to pick, as with some of those quoted by Munro, but I do not know if they actually have the critical bent, shall we say, that you do. The WHO committee that reviewed the data, with Lafta present, seemed to come away satisfied.

My interest is not in scoring points on arcane matters of number crunching, but in seeing the causes of violence clearly and hoping perhaps that such knowledge will help prevent more violence, in Iraq or elsewhere. Without such knowledge, the U.S. military could be led into more foolish exertions of power. The scale of the carnage is important, and the ways in which people are being killed, and where, is also crucial to understanding the nature of the insurgents' violence in particular. Only large-scale death really explains an insurgency--really, many insurgencies-- that is decentralized, without apparent ideology, linked to no known groups (the AQ portion is small), and remarkably persistent. The most logical explanation for this is that many people have been roughed up and they are fighting back, believing that they are defending their tribes, towns, etc. This is not a brief for the insurgents, mind you, but a very plausible explanation. And the higher the violence, the more it makes sense. (See opinion polls of Iraqis and this view--and high mortality--is even more convincing.) There is much corroborating evidence for high mortality, not least the 4.5 million displaced (some of whom, perhaps a million, were pre-2003)and a reported 500,000 war widows, among other indicators.

These considerations, this set of puzzles, is what's important to me. Are there sampling errors in any of these surveys? Quite possibly. Is there sloppiness in data entry? It would be surprising if there weren't. Were survey managers too cavalier or too scared by street violence to be tightly disciplined about protocols? I can't blame them if they were.

But, overall---and the MoH survey really does confirm this---we see mounting evidence that mortality is very, very high, much greater than political discourse has allowed. So I am not very patient with the nitpickers. Very very few knowledgeable people see fraud in L2. Knowing the researchers, I find it extremely improbable that they would countenance it, or not detect it. That's good enough for me, particularly when there is so much supporting evidence. And so I'm quite willing to concentrate on what I see as the principal challenges---convincing the political elite in this country to take the high mortality figures seriously as a problem of diplomacy, military strategy, humanitarian planning, and morals.

By the way, I agree that a debate is less than optimal. The problem with the blog is that it is easy for Tirman (and others) to refuse to respond to specific questions. They just ignore them.

Simple example: David, why have you nothing to say about the response rate of this paper?

I appreciate John Tirman's response and (surprisingly?) agree with almost everything he says above. Too many Iraqis have died. If I could snap my fingers and make things better, I would. But my main focus is on how science is conducted, and so I will continue to hype the issues of data availability, replication and so on.

Now that I have pointed out that many more than "one" critic has "made a complaint" about the matter of data availability and reliability, I hope that John will correct that comment in his piece. It would certainly be fair, for example, to say "only a few." Also, I would appreciate it if he cited me by name and, ideally, linked to my blog. That will allow readers to judge for themselves the quality of my complaints. And, what is the point of being obsessive if you don't get the credit! ;-)

"But my main focus is on how science is conducted, and so I will continue to hype the issues of data availability, replication and so on."

And continue to avoid answering questions about whether the high response rate for the IFHS study is proof of fraud.

But my main focus is on how science is conducted...

I damn near lost my coffee reading this one.

But my main focus is on how science is conducted, and so I will continue to hype the issues of data availability, replication and so on.

And, yet again, right out of the climate audit playbook ...

Regarding this, though ...

The researchers will not release data at the household, street or neighborhood levels, which could identify and put study participants at risk.

And then

These data will be released on request to recognized academic institutions or scientific groups with biostatistical and epidemiological analytic capacity.

This can be read to support Kane's position that everyone's been given the same data, i.e. without the revealing information included.

It also seems clear that withholding that information was a decision taken with the ethics committee before the survey was taken, as a condition of having the survey project approved.

Therefore, Kane's bitching and moaning essentially boils down to wanting data released that, as part of the ethical review process leading to approval of the project, was promised to be kept secret. In which case the L2 crew are bound not to release that data.

Am I reading this correctly?

You may well be - in which case it'd be nice to have a clear definitive statement to that effect from the authors so we could move on.

Also, I would appreciate it if he cited me by name and, ideally, linked to my blog. That will allow readers to judge for themselves the quality of my complaints.

well David, i would really recommend every person with at least an average IQ and some basic math education to take a look at your page.
your arguments are weak and obviously driven more by some personal agenda than by scientific interest or knowledge.

on the other hand, a majority of the right wing blogosphere and commentators are preaching to a crowed that does NOT fulfill these requirements.
they get easily confused by numbers, especially when someone tells them that they support their views.

a simple look at your latest drivel exposes a look at this:

The statement on missing certificates is wrong. Three clusters did not have the presence of certificates noted, and in all there were 120 deaths in which the interviewers neglected to note their presence.

i interpret this as Burnham saying that about 120 cases in which the interviewers did not note the certificate and that in three clusters little or no certificates were noted.

you declare fraud:

So, what are BR talking about? Who knows? My guess is that Roberts wrote this (without really looking at the data) and then convinced Burnham to sign off

and find a grant total of 128 missing certificates and note that they are spread out over 20 (or 14) clusters.

i fail to see the major difference in the number of certificates, and i don t see a contradiction in the numbers you present to what Roberts said.

(like if i say: i found 130 errors (when in reality it were only 122) in the posts of David Kane, and 3 of his posts didn t contain a single true sentence.)

so what is your basis for this allegation again?

John Tirman is not doing perfect troll management, but he's getting there.

Spagat and Kane are no better than our worst internet trolls - let them sink out of sight after their falsehoods have been called out and debunked.

"Three clusters did not have the presence of certificates noted" is just flat out wrong. The number is 20 clusters, or at least 14, depending on the meaning of "presence of certificates noted." I do not know why they made this mistake. Sloppiness? A desire to pretend that interviewer forgetfulness was concentrated in just three clusters? I really don't get it.