Innumerate BBC

Lenin points to some complete innumeracy at the BBC as they make excuses for not using the Lancet estimate. They write:

We do not usually use the Lancet's figure in standard news stories
because it is so far out of line with other studies on the same issue.
There are also some questions over the validity of the Lancet study in
the case of measuring casualties in Iraq. The technique of sampling and
extrapolating from samples has been criticised in this case because the
pattern of violence in Iraq has been so uneven.

Of course, it isn't far out of line with other studies---it agrees quite well with the ILCS survey. And while it is possible to criticise the estimate because the pattern of violence has been so uneven, such a criticism that was grounded in an actual understanding of statistics would be that the Lancet underestimated the death toll because of the exclusion of Falluja and the tendency for cluster sampling to underestimate deaths when they clump together.

As Lenin says:

the BBC are rejecting the report's surveys on two false grounds: one, that it is 'out of line' with other reports (it isn't); two, that its method is somehow inappropriate because uneven patterns of violence skews the results. Both criticisms are identical to those deployed by the UK government, and both are ridiculously wrong. Even the right-wing Economist did better than the BBC.

Tags

More like this

Lenin on the IBC attack on the Lancet study I had anticipated that the team behind Iraq Body Count would react to the latest survey on Iraqi mortalities published in the Lancet by trying to minimise their import and undermine their reliability. I was not wrong. The reason is fairly simple: they're…
A misleading sidebar on this BBC story on Iraq only presents the IBC count of civilians killed in Iraq (which is guaranteed to be a significant undercount) and omits to mention the Lancet estimate of roughly 100,000 excess deaths. Gabriele Zamparini wrote to the BBC seeking an explanation. The…
Stephen Soldz has posted his discussion with Jon Pedersen about the new Lancet study: [Pedersen thinks that the] prewar mortality is too low. This would be due to recall issues. ... Pedersen thought that people were likely reporting nonviolent deaths as violent ones. These two have to go together…
Or at least 655,000 (± 140,000) of them. Before I get to the news reports, I think it's important to make something clear. These statistical techniques are routinely used in public health epidemiology and nobody complains about them. Critics of this estimate can't play the same game the…

*[Sorry, Seixon -- you have your own thread where you can post about Lancet statistics. You are not permitted to clutter up every discussion here with your innumerate nonsense. Tim]*

Well, this relates to the chronic problem of the innumerate media's never reporting a confidence interval for any research result, when in truth that's as important as the point estimate. Even the Beeb, sadly.

OK, so I can't post anything about the Lancet study in a thread about the Lancet study. Gotcha. I know why you deleted my comment. I'll take that as a sign that you know your previous claims cannot withstand scrutiny.

The BBC can reject to talk about the Lancet study based entirely on the confidence interval alone. I doubt you could find a mortality study that has a wider confidence interval than this one. So if you have one Lambert, I'd love to see it.

tim,

you have to understand the domestic issues affecting the BBC here.

they are under severe threat because the government, riled up over the whitewashing exercise termed "the hutton report", and egged on by rupert murdoch, is threatening the bbc with the removal of its charter and a subsequent throwing to the commercial wolves, which would be a disaster on a massive scale - it is by far and away the best broadcaster (both in terms of quality output and trustworthiness) in the UK and probably the world.

as a result, there are a lot less stories on the bbc that are challenging what the government is saying, sadly. as you have said, the official government position is that the lancet study is b*ll*cks and to risk another row over it is pointless, especially with the lack of any real opposition to HMG in the UK right now - even over important issues.

>it is by far and away the best broadcaster (both in terms of quality output and trustworthiness) in the UK and probably the world.

You'd have a difficult time convincing me of that. Not that I think any of our mainstream media sources in the USA or Canada are any better.

In other words, it has just turned into the same intimidated craphouse as a lot of other once good broadcasters and newspapers.

Sorry - I know the reasons and I grieve - but they are telling lies.

Seixon, this thread is about the BBC... I'm guessing your comment had nothing to do with that.

I believe the confidence interval for the estimate of deaths in the Rwandan genocide was wider than that of the Iraq study.

Danny,

Gee, is that because the numbers in the Rwandan genocide were millions and not thousands? What is it with this blog and misleading statements?

Tim,

What exactly do you think the BBC should report about the Lancet study? Roberts summarizes his study thus:
"we reported three things: violence was up 58-fold, becoming the main cause of death; the one neighborhood visited in Anbar Province had 1/4th of the population dead, statistically suggesting almost 200,000 deaths; and in the other 32 neighborhoods we estimated 98,000 deaths."

I haven't seen any coverage that reports Roberts' take on his study which, you will notice, is quite complicated. Most press coverage just says "about 100,000 deaths," which is basically misleading. The issue of with Falluja vs. without Falluja really makes it difficult to summarize the study accurately.

I am not trying to solicit customers from your site, honestly I'm not, but 'Son of Duff' has just elbowed his way onto my site to publish a pretty picture summarising all the available statistics into one graph and thus demonstrating, apparently, that when it comes to killing people, Saddam was a pro and the coalition are mere dilettantes. You readers may care to have a quick peek and then return to the argument here.

http://duffandnonsense.typepad.com/

The graphs (which are barely legible) are so misleading as to be laughable.

For one thing, you acceptable uncritical the extremely high estimates for deatsh caused by sanctions.

Funny isn't it how the right were wildly pro-sanctions up until 2002 and ridiculed any suggestion that they were causing serioua loss of life then switched overnight to embracing the highest estimates for deaths.

Orwell would be proud.

For another, you fail to include the indirect loss of life attributable to the invasion. The highest estimate I've seen for pre-invasion infant mortality is 6 per 100 births. The current estimate from the Iraqi government itself is 10 per 100.

You also blithely assume that everythign's goign to be wonderful in Iraq for the next 20-odd years.

By Ian Gould (not verified) on 10 Nov 2005 #permalink

OK, it seems Lambert can't stand scrutiny of his previous posts, so I guess I will just have to not keep undermining him. Censorship, yes sir!

I actually had 4 points, but Lambert is uncomfortable with two of them since they go after his comments on here. So the rest of you will have to deal with scrutiny, while Lambert protects himself from it.

A. Duffy, your graph is laughable for reasons already stated.

B. Ian, the UN survey done in 1999 gave 108/1000 for infant mortality in 1999. Another one done by UNICEF gives a figure of 101/1000 in 1998. This one says 103 for 1998. This indicates 102 in both 2000 and 2003.
So I'm kind of wondering Ian, were you looking hard, or hardly looking when you found that 60/1000 was the highest rate pre-invasion?

Seixon,

I was working from memory and made a mistake.
I should have checked it before posting and I admit that.

But some of us have lives outside of obsessively posting on blogs.

By Ian Gould (not verified) on 10 Nov 2005 #permalink

Most press coverage just says "about 100,000 deaths," which is basically misleading

It isn't, not really. Particularly when the alternative that the BBC has pursued is to uncritically print the IBC and Health Ministry point estimates.

Gentlemen, I shall point "Son of Duff" in your direction so stand by for incoming!

Me? I couldn't pass a GCSE in 'sums', so I'm outta' here.

But some of us have lives outside of obsessively posting on blogs.

It took me a few minutes to round those up, Ian. Let's leave the "get a life" BS at home, mk?

It isn't, not really.

No, but writing "100,000 civilians dead" is. I would add that the IBC is usually printed with qualifiers, and that both it and Health Ministry records base themselves on facts and not a poll. Not that they accurately count every single death in Iraq, but they are at least based in fact, especially the latter.

Duff,

The problem is that your numbers for at least the pre-invasion period are based on very little.

In another matter, I am a bit taken back by the wildly different infant mortality rates in play between the previous UN surveys and the Lancet and UNDP studies.

If you compare the ones I gave earlier to these:

UNDP: 1991-2000 32.2/1000; 1994-1998: 32/1000; 1999-2003: 29 (female) - 35 (male) /1000; post-war: 22-47/1000 (depending on region)

Lancet: pre-invasion: 29/1000; post-invasion: 57/1000

What to make of all that?

Ian Gould: The figures I have used for the regime's death-tolls come from http://www.moreorless.au.com/killers/hussein.html and are minimums per that source. As I researched for the approximate start and end dates of the regime's various death-toll categories, so as to be able to spread the numbers across the corresponding years, I couldn't help noticing the source's figure of 500,000 coming up repeatedly for the infant mortality rate due to sanctions. What about the effect on the adult (and elderly) mortality rate? As I said, to favour the Stoppers the figures are minimums for the regime.

The Lancet figures are for excess deaths after the invasion, i.e. excess compared to the rate before the invasion (more on that later). They are not exclusively violent deaths, but rather they also include non-violent deaths due to all circumstances prevailing after the invasion. The total Lancet figure, violent and non-violent, has been attributed to the coalition, in contrast to your statement "you fail to include the indirect loss of life attributable to the invasion".

As for the future, well let's wait and see. The only sure way of getting the post-invasion death-toll back up to the regime's giddy heights would be to pull out of Iraq too soon.

Tim Lambert/Seixon: The Lancet figures are excess figures for the immediate post-war period compared to the immediate pre-war period, and I have attributed this figure in total to the coalition. The sanctions continued after the March war until June 2003 due to the bureaucratic inertia of the UN (and perhaps due to some of the "indefatigable" beneficiaries of the sanctions system and programs?) and in spite of the frenzied attempts by the US to lift them. This three month period is material to the Lancet post-war study period (18 months) and so a sizeable proportion of the Lancet post-war figure is due to a cause I lay squarely at the feet of the regime. But to favour the Stoppers I've blamed the UN/US and included it into the coalition's figure.

Perhaps what you're trying to say is that the background mortality rate inherited by the coalition - the amount above which the Lancet figure is an excess/surplus - should also be added to the coalition's figure? This background figure comprises two parts: -

(1) The mortality inertia behind the regime's policy during its period in office of inducing and enduring international sanctions. While I am prepared to add the effect of lingering post-war sanctions into the coalition's figure (see paragraph above), the inertia of prior-war international sanctions, induced and endured by the choice of the regime, is simply too unreasonable to include in the coalition's death-toll. Rather, this number should be added back to the regime, but I haven't done this to favour the Stopper's argument.

(2) The background mortality above which the regime's death-tolls are also a surplus. Remember the death-toll figures for the regime are also an excess number - murders, wars, etc. - above the background mortality rate for Iraq 1979-2003. Since the background mortality has not been included in the regime's figure I haven't included it in the coalition's figure either. If you want to add it to the coalition then you have to add it to the regime too. Either way, it doesn't affect the net difference between the two administrations - 874,500 more under the regime over 24 years.

The only way the Stoppers can wriggle out of the conclusion that there are 1,000 more Iraqis, Iranians and Kuwaitis alive every month precisely because of regime change by the coalition, is to attribute the sanctions deaths to the coalition rather than the regime. And that's a whole topic on its own; and the Stoppers lose that argument too.

Son of Duff

P.S. The [link again](http://duffandnonsense.typepad.com/duff_nonsense/2005/11/lies_damned_li…)

By Lawrence Duff (not verified) on 11 Nov 2005 #permalink

As for the future, well let's wait and see.

You don't wait and see. If you did there would be only two columns in your right-hand chart. What you do is forecast a glowing future and conclude, on the basis of this rosy scenario:

In other words, there will be 36,438 more Iraqis, Iranians and Kuwaitis alive per annum over the next 24 years, equals just over 1,000 per month, than would otherwise be the case if the regime had been left in place.

This also implies that without intervention, the wars with Iran and Kuwait would have been repeated. It's a bit much to tell other people not to speak of the future when you not only do it yourself, you also pronounce on what would have been.

By Kevin Donoghue (not verified) on 11 Nov 2005 #permalink

Lawrence, you don't seem to have understood what the Lancet study measured. It measured the **increase** in deaths after the invasion. However many people were dying before the war from sanctions and being murdered by Saddam, well, about 5000 more died each month afterwards.

And 500,000 dead from sanctions is [not a minimum estimate](http://www.reason.com/0203/fe.mw.the.shtml):

>Those who get past the initial frustrations of researching the topic usually end up on Richard Garfield's doorstep. His 1999 report -- which included a logistic regression analysis that re-examined four previously published child mortality surveys and added bits from 75 or so other relevant studies -- picked apart the faulty methodologies of his predecessors, criticized the bogus claims of the anti-sanctions left, admitted when the data were shaky, and generally used conservative numbers. Among his many interesting findings was that every sanctions regime except the one imposed on apartheid South Africa led to limitations of food and medicine imports, even though such goods were almost always officially exempt from the embargo. "In many countries," he wrote, "the embargo-related lack of capital was more important than direct restrictions on importing medicine or food."

>Garfield concluded that between August 1991 and March 1998 there were at least 106,000 excess deaths of children under 5, with a "more likely" worst-case sum of 227,000. (He recently updated the latter figure to 350,000 through this year.) Of those deaths, he estimated one-quarter were "mainly associated with the Gulf war." The chief causes, in his view, were "contaminated water, lack of high quality foods, inadequate breast feeding, poor weaning practices, and inadequate supplies in the curative health care system. This was the product of both a lack of some essential goods, and inadequate or inefficient use of existing essential goods."

Richard Garfield was one of the authors of the Lancet study.

Kevin: My comment "let's wait and see" is not an alternative or contradiction to my projection, but rather means "let's wait and see if my projection turns out to be more accurate than the interminable spin and lies of the Stoppers". I am sorry for any confusion.

"This also implies that without intervention, the wars with Iran and Kuwait would have been repeated".

Using the past to make projections about the future is, I know, intrinsically dodgy. Bertrand Russell sums it up pithily well: -

"Every day the farmer steps out to the henhouse and scatters chicken feed to the chicken. So the chicken would be very reasonable to believe that tomorrow the farmer will come and feed the chicken. But tomorrow, instead, the farmer goes out to the henhouse and wrings the chicken's neck for dinner. More refined views as to the uniformity of nature would have been useful to the chicken".

At best statistics and projections are only one component part of a wider judgement when trying to assess what's going to happen and which course of action might lead to a better outcome. But if you are going to use body bean-counting with projections then at least count both halves properly. To devote shrill and repetitive attention to only one half of the story, so the other half is blotted out, is to spin and lie, and this has been the disreputable technique of the Stoppers. And that is the real point of my post, as Father of Duff observed in his earlier comment here and in the comments section of my post.

Tim: I did say, not once but twice, that the Lancet figures are excess deaths over and above the mortality rate before the war: -

"The Lancet figures are for excess deaths after the invasion, i.e. excess compared to the rate before the invasion".

"The Lancet figures are excess figures for the immediate post-war period compared to the immediate pre-war period".

I then went on to explain that in comparing the lethality of the regime versus the coalition it is not reasonable to add the pre-war mortality rate - the figure, above which the Lancet's is an excess, which was inherited by the coalition and continued into the coalition's administration - to the coalition's figures. The reason for this is that the pre-war mortality rate, perpetuated into the coalition's administration but not included in the Lancet excess figure, comprises two parts, neither of which can be justifiably added to the coalition's toll: -

(1)The inertia of pre-war sanctions which continues to induce a lingering effect on post-war mortality rates. Since the regime's decision to induce and endure sanctions in the pre-war period was the cause of this effect, it is the regime's death-toll which should include these numbers, not the coalition. I have not added this death-toll back to the regime to favour the Stoppers case.

(2)The background mortality rate not due to the regime's murder, war, etc. figures given in the chart. The figures in the regime's side of the chart are excess figures above this background mortality rate. Since the background mortality rate is not added into the regime's chart-side it is also not added to the coalition's chart-side. You could add it to both if you wanted, but it wouldn't change the net difference in death-tolls: 874,500 more under the regime.

I don't think I can say it again any other way or more clearly.

As for the 500,000 sanctions deaths and the Garfield report, the Garfield 106,000 - 350,000 range only covers under 5's in the period 1991-1998. What about 1998-2002? What about the over 5's across the whole sanctions period? Garfield says:

"Studies from 1996 onward suggest that there was little decline in mortality rates at that time. Since March 1998 the oil for food program has greatly increased access to essential supplies and the mortality rate has surely declined, but data are not yet available to estimate the magnitude of that decline."

"Even gross estimates of deaths among those above age five are much more problematic to generate. Adult pathologies are less dependent on environmental and social conditions for which we have good estimates and depend more on clinical variables. No attempt will be made to estimate above five year old excess deaths here."

He doesn't know. I don't know. If you're comfortable with a minimum figure of sanctions deaths less than 500,000 over the full sanctions period, then be my guest. My feel for the surveys tells me that to do so would be to enter the realms of holocaust denial.

Son of Duff

P.S. My link
again
and a Garfield
link

By Lawrence Duff (not verified) on 12 Nov 2005 #permalink

Duff,

If you're saying "wait and see" then you have to apply that to yourself as well.

No comments on the amazingly inconsistent infant mortality rates between all kinds of surveys?

Back under the sanctions period, the surveys were finding 100/1000 as the common rate.

Now after the war, the UNDP and Lancet both found rates around 30 for the pre-war period.

I understand this might have been discussed here before, which in case I would appreciate a link. But can this be explained by recall bias alone???

Or does it indicate that Saddam was perhaps pressuring Iraqis to lie under interviews while he was in power to juice the numbers?

The discrepancy between the UNDP and Lancet post-war figures are also a bit odd (well, not considering the UNDP's immense precision in comparison...).

Forgot to add:

If the Lancet and UNDP figures for the pre-war infant mortality are correct, then that undermines the entire premise of Duff's graph, and also undermines the whole "500,000" thing that anti-sanctions activists were carping about. I'd say that would be big news to many.

Uh, I'm not sure about this bit from Son of Duff: "Since the regime's decision to induce and endure sanctions in the pre-war period was the cause of this effect, it is the regime's death-toll which should include these numbers, not the coalition".

Okay, maybe it shouldn't be attributed to the coalition, but it is entirely fair that these deaths are laid at the doorstep of the regime? I mean, considering th...

Oh, who gives a shit. All I know is that this whole "Son of Duff"/"Father of Duff" thing is weirdly gay.

Thank you Hugs. I'm glad someone here had the courage to speak truth to weird.

Now we can all get back to watching Sexion's head come to a point.
And his nose to a trunk.

Hmmmn! "Hugs O'Toole" sniffs out something he considers to be "weirdly gay". Well, considering that he chose for himself a 'nom-de-plume' like that we can only assume he would know! No doubt if you give him a kiss, he'll tell you.