IBC vs Media Lens on Iraq Casualties

The BBC has a report on the dispute between the IBC and Media Lens about Iraqi casualties. (My previous post on this is here.) IBC's John Sloboda trots out Kaplan's fallacy:

Some critics of the Lancet study have said it's like a drunk throwing a dart at a dartboard. It's going to go somewhere, but who knows if that number is the bulls eye.

Unfortunately many many people have decided to accept that that 98,000 figure is the truth - or the best approximation to the truth that we have.

Yes, some critics have said that, but not ones that are experts in statistics.

Sloboda says:

We've always said our work is an undercount, you can't possibly expect that a media-based analysis will get all the deaths. Our best estimate is that we've got about half the deaths that are out there.

OK, then why does the IBC page say "Iraq Body Count: Max 38661"? That's not really the maximum possible number of deaths, is it? Why not report their estimate that the true number of deaths is 70,000 or so?

The BBC also has replies from Media Lens and John Pilger. The IBC has a more detailed defence at their website which claims that my analysis here is incorrect. I will deal with their claims in another post.

Tags

More like this

"OK, then why does the IBC page say "Iraq Body Count: Max 38661"? That's not really the maximum possible number of deaths, is it?"

No. It's the maximum number of reported civilian deaths, as is stated on their homepage, counters, database..etc., and based on the methodology posted openly on their site.

I find it hard to believe that someone who has produced dozens of lenghty pages dissecting the intricate details of the Lancet study would be incapable of reading the description of IBC's count directly above the number he's citing.

"Why not report their estimate that the true number of deaths is 70,000 or so?"

Because that would be another guess that most people would not trust. Obviously when there is a divisive issue like Iraq, most people are not going to respect a figure like the Lancet study that has a margin of error greater than that of an opinion poll. Yes, we all know the 100,000 figure was their "best guess", and they say the figure becomes less likely the further away you get from that central figure on either side. But nevertheless, it's still a guess.

The authors of the Lancet study themselves said their work was of "limited precision" due to the uncertainty of the interviewing process in the middle of an insurgent conflict. Common sense, really.

Lastly, there is also the issue that many people, like the Medialens editors, want to use the Lancet study for pure propagandarist purposes. They have no interest in telling the truth and just want to use this excess deaths figure to simplistically claim all of this number is solely the fault of Bush and Blair, editing out the matter that it does not include all direct causlties of war and glossing over the fact Al Qaeda and insurgent bombers, the people they support, also have moral agency. Al Sadr didn't even start his uprising until a year after the invasion; is that Bush's fault too?

Medialens are propagandarist polemicist fanatics and the media are wise to be wary of them.

"Because that would be another guess that most people would not trust. Obviously when there is a divisive issue like Iraq, most people are not going to respect a figure like the Lancet study that has a margin of error greater than that of an opinion poll. Yes, we all know the 100,000 figure was their "best guess", and they say the figure becomes less likely the further away you get from that central figure on either side. But nevertheless, it's still a guess."

I think it's true that people will always challenge these estimates and raise doubts about them. That is the advantage of having a baseline of documented deaths which is impossible to deny. And this does not preclude others making estimates.

John's reference that "Some critics of the Lancet study have said it's like a drunk throwing a dart at a dartboard. It's going to go somewhere, but who knows if that number is the bulls eye." is actually an improvement of the "dartboard" references I've seen previously.

It is kind of like a drinker in a bar throwing a dart at a board. He's not just throwing randomly. He's trying to hit the middle and making his best effort, but the wider the CI (the more drunk he gets) the less likely it is that he'll hit it.

Media Lens are "glossing over the fact Al Qaeda and insurgent bombers, the people they support..."

Oh, come on Ryan

Are you joking? John Pilger, the celebrity journalist they have chosen to front their campaign againt IBC, openly supports the violence in Iraq. He is pro-war and so are they. They are hypocrites.

Yes, that could benefit from a qualifier like *reported* killed or documented or something along those lines.

The BBC's full page on Iraq Body Count is much better than is typically the case in soundbyte references:
http://news.bbc.co.uk/1/hi/world/middle_east/4525412.stm

And how accurately would you say the Lancet study is referenced in the media for most readers Tim?

"Are you joking? John Pilger, the celebrity journalist they have chosen to front their campaign againt IBC, openly supports the violence in Iraq. He is pro-war and so are they. They are hypocrites."

I'd suppose they take the view that Iraqis have a right to resist foreign aggression, even if many of the Iraqis who do that are rather unsavory themselves.

Agree with that or not, but calling people who take that view pro-Al Queda is kind of like Goebbells smearing people as pro-Stalin or as Commies for supporting Russia's right to resist aggression and conquest, regardless of whether Stalin was himself a bastard.

Ryan, you suggest that Slobada is reluctant to present "another guess that most people would not trust". That doesn't sit very comfortably with the fact that he says: "Our best estimate is that we've got about half the deaths that are out there."

He is claiming that the IBC figure is something better than a lower bound for the true figure. But in truth that's all it is.

By Kevin Donoghue (not verified) on 28 Apr 2006 #permalink

joshd, I clicked on about twenty of the links at the IBC's [page on media mentions](http://www.iraqbodycount.net/coverage.php). Almost always the IBC number is given as the number of Iraqi civilians killed.

If I were in charge of that page I would be gravely concerned that what the IBC is counting is **almost always** misreported and I would change the way the number is presented.

Change it to what Tim?

You mean instead or in addition? The latter might have some potential, the former would be disagreeable imo.

And if we were going to do that we'd probably want to do a much more thorough analysis. What we did in this paper was merely followed Les Roberts lead using his analysis as a model, but improved on certain flaws in his analysis of IBC, to arrive at a more realistic appraisal. The intent and design of this exercise was to demonstrate that the claims of our critics were wrong and based on these previous erroneous calculations. We believe it successfully did that, but that exercise was not itself designed specifically to produce the definitive estimate, or the best way to turn IBC into an estimate (which it currently is not). There may be still more appropriate adjustments that need to be made for time-frames or other issues, in addition to those we added to Roberts' original analysis.

For us to recommend that the media should quote IBC as a definitive estimate, would be rather presumptuous of us and I think would demand another going over, with much more detail to all possible factors, and with that specific goal in mind as the purpose and basis of the analysis. (I realize the media often use it as an estimate anyway, but that is not because of our recommendations. It's because of intentionally or unintentionally ignoring them.)

Everything we have said so far has been to the effect that we are *not* that, and are instead a total of those deaths that have been documented in media or official sources and which therefore are definitely known to have occurred.

Also, we are not convinced that turning IBC into an estimate is the way to deal with the issue of media wrongly using IBC as an estimate (and we are less convinced of the extent of the "damage" being done by these misuses than are many of our critics). IBC is a new kind of methodology and the media seem accustomed to getting figures from survey estimates or demographic estimates. And many seem to present IBC in the same way out of habit and to fit it into a soundbyte. It may be much more appropriate to continue trying to educate and inform about this form of methodology, rather than changing it into an estimation to conform to media presentation habits.