Poor Reporting in the Washington Post

Peter Baker in the Washington Post writes:

The Iraqi death toll has been the subject of considerable debate. A group of British researchers and antiwar activists called Iraq Body Count estimates civilian casualties between 27,383 and 30,892, not counting Iraqi troops or insurgents, by tabulating incidents reported in media and human rights reports. Iraqi authorities have said that roughly 800 people die a month in violence there, a rate that if typical over the course of the conflict would come to 25,600.

An epidemiological study published in the British journal the Lancet last year estimated 100,000 deaths in the first 18 months since the invasion based on door-to-door interviews in selected neighborhoods extrapolated across the country, an estimate that other experts and human rights groups considered inflated.

Brad DeLong comments:

This is, I think, somewhat depressing. Baker wants to be adversarial--in a way that Harris would call "liberal" and "biased," and would not like. Baker is outraged at the way in which the White House has pretended ignorance as a way of avoiding answering questions about the impact of the war on civilian Iraqis. Baker wants to use the fact that Bush has a "30,000 civilian Iraqis dead" number in his head as a knife to pry open this particular oyster.

The problem, however, is that Baker is underbriefed. He knows that the Lancet published an article last year but he doesn't really know what the study said. He doesn't make the point that the Iraqi Body Count estimate that tabulates only reported casualties is--if the individual reports are accurate--to understate total casualties because there are, inevitably, unreported casualties. He doesn't say who the "Iraqi authorities" who report 800 a month are, or why anybody should trust their estimates.

It's worse than that. Other experts do not consider the estimate to be inflated. Lisa Guterman called

about ten biostatisticians and mortality experts. Not one of them took issue with the study's methods or its conclusions.

Nor is it true that human rights groups consider the estimate to be inflated. Mark Garlasco of Human Rights Watch was pressed for a comment by the Washington Post. He said that he had not read the study but that the estimate seemed inflated. He changed his mind after reading the study and talking to statisticians about it.

To my knowledge, none of the Washington Post's reporters who have written about the study have tried reading it and talking to statisticians about it. It is not as if these journalists have to understand statistics---they just have to talk to someone who does. It's just pathetic

Tags

More like this

Say, rather, that they don't know how to find a statistician--they know how to find pollsters and political hacks, but not how to find experts in any subject (not just statistics) who will give them a balanced view...

By Brad DeLong (not verified) on 14 Dec 2005 #permalink

Not only do they not bother to talk to statisticians, but they don't even bother to understand the general-purpose words they are applying. Words are simply assigned without regard to the facts of the case. A count of deaths from violence in published reports is not by the standard definition of the word an "estimate". It is a count. It is also misleading to state the range, which makes it look more like an estimate, without indicating that this range is due to discrepancies between published accounts of events.

This problem gets worse when dealing with the study. The neighborhoods were not selected, by the general definition of the word. It is misleading to state this without the qualification that the "selection" was random. The deaths were excess deaths, not plain old "deaths".

It is extraordinarily difficult to translate technical terms and ideas into "plain English". But it is a reporters job, more or less.

(Spot on in the last paragraphs. That quoted sentence alone in the article is surely due correction.)

One small curiosity I have about the study is how did the researchers manage to go around Iraq and conduct the study so nicely without getting their heads chopped off? I'd like to go do a similar study, but I'd be a little worried about my noggin.

The interviewers were Iraqis. Roberts accompanied them on a couple of trips, but he kept a low profile. Several were medical doctors, which might perhaps confer a bit of immunity from all but the worst kinds of thugs.

By Kevin Donoghue (not verified) on 14 Dec 2005 #permalink

There's been a lot of research done over the years (in many different cultures) that show people normally answer these surveys accurately.

In the Lancet survey, they asked a random sample of households reporting a death to produce a death certificate. Most were able to do s0 with the remainder providing credible reasons why not (i.e. the death had occurred very recently.)

In order to cut down on the risk, the researchers paired provinces they believed to be similar and then only did field work in one. So if Nineveh and Anbar were considered similar, one of the two was dropped from the survey and the other was allocated the total number of sampling clusters for the two combined.

This was one of the more controversial aspects of the survey and contributed to the very high uncertainty levels in the figures.

By Ian Gould (not verified) on 14 Dec 2005 #permalink

Interesting. I wonder how many Iraqi's would have had to die to have overthrown Saddam from within, or if it would have been worth it, in light of the history of Saddam's regime, to just live with the status quo? I guess we could have just waited for diplomacy to work, but I don't know of any examples where that has ever happened.

"It is not as if these journalists have to understand statistics—they just have to talk to someone who does."

Yes but then having talked to someone who does they would now be in possession of the truth, or something much closer to the truth than they had to begin with.
And then they'd have to choose either their own personal comfort or their own personal sense of integrity and self-worth, because possession of the truth would make it impossible for them to continue with both intact.
People go crazy from getting stuck in these kinds of binds.
Seeing them coming ahead of time and avoiding them is what passes for professional skill in many journalistic venues these days.
It portends trouble either way - but some of us would prefer to see trouble no matter how great, and face it as human beings, together, than take isolating refuge in self-deception.

"I guess we could have just waited for diplomacy to work, but I don't know of any examples where that has ever happened."

You mean apart from Pinochet's Chile, Franco's Spain, apartheid-era South Africa; Suharto's Indonesia; Taiwan; South Korea; Poland, East Germany....

By Ian Gould (not verified) on 14 Dec 2005 #permalink

Right, sorry, I guess I was only thinking of China, Cuba, Iran, Iraq, Libya, Syria, Sudan, Afghanistan, North Korea, Saudi Arabia, Zimbabwe, the Balkans and the rest of the list.

I guess they'll all have to wait until their dictatorships are crushed under the weight of those paper UN resolutions. Oh wait! Isn't Libya chairing their "human rights counci?" Why, I'm surprised those dictatorships haven't collapsed into flowers and sunshine already!

And hey, before I pull a fast one on myself, last time I checked, we, meaning mostly the US, fought a fairly bloody war to keep South Korea free. Where was the diplomacy there? And further, Taiwan? Give me a break. If it wasn't for Uncle Sam glaring down at mainland China, the Taiwanese would all be speaking Mandarin right now. Oh wait, they already do. Well, they'd be part of that wonderful Chinese flavored communism now. Great, yeah, diplomacy.

And one last time, I promise, East Germany? Wasn't that created out of diplomacy. Hah! And it only took them 50 years of, I guess you could call that diplomacy, to get them out of their situation.

Then we have South Africa. Now, if I recall correctly, while not a bastion of human rights, it wasn't a dictatorship either. In that situation, you could affect the substantial minority through sanctions, whereas Saddam, as the dictator, could just keep squeezing the people and building his palaces while they starved.

Which brings me to Poland. What exactly would have happened to the Iraqis under Saddam if they tried the stuff that Lec Walesa led his people to? They'd have been shot or worse. Good luck with that approach.

And lastly, we come to Pinochet. We were and are an awful lot harder on Castro, and he isn't going anywhere any time soon, except maybe into the grave, but then the Cubans get his swell brother. Yay!

And you'll recall Ben that after the UN/US fought this fairly bloody war in South Korea, it was ruled by a collection of autocratic dictators, and certainly wasn't a beacon of democratic light. It was only 40 years later that we got democracy in South Korea.

I could make a similar comment about Taiwan, but at least there were no massacres there (that I know of).

The point of this is not US bashing, but merely to point out that building an open, free society is a hard stop-and-go process and requires a lot (decades worth) of patience. Remember it took centuries for Britain and the American colonies to drift into democracy, the open society and universal franchise. It would be astonishing if it can be done quickly in Iraq, with or without the recent US assistance.

By Patrick Caldon (not verified) on 14 Dec 2005 #permalink

And you'll recall Ben that after the UN/US fought this fairly bloody war in South Korea, it was ruled by a collection of autocratic dictators, and certainly wasn't a beacon of democratic light. It was only 40 years later that we got democracy in South Korea.

Actually, I did not know this. Seems true to form though.

The point of this is not US bashing, but merely to point out that building an open, free society is a hard stop-and-go process and requires a lot (decades worth) of patience.

Fair enough, although it seems that with some (Pinochet, South Africa) the liberal world is not patient, and with others (Castro, Middle East), the liberal world has all the patience in, well, the world.

Ben,

Proof that not all dictatorships have been overthrown peacefully is not proof of your initial claim that no dictatorships have been,

By Ian Gould (not verified) on 14 Dec 2005 #permalink

Taiwan? Give me a break. If it wasn't for Uncle Sam glaring down at mainland China, the Taiwanese would all be speaking Mandarin right now.

Sigh

The first thing the Nationalists did on arriving in Taiwan after been kicked off the mainland was to massacre 100,000 indigenous Taiwanese who wanted independence from China.

Then they proceeded to set up a dictatorship which rivaled that of the PRC in its brutality.

By Ian Gould (not verified) on 14 Dec 2005 #permalink

Fair enough, although it seems that with some (Pinochet, South Africa) the liberal world is not patient, ...

Pinochet ruled for 20-odd years then stepped down voluntarily due to age and ill-health; Franco ruled Spain for about 30 years after World War II; Apartheid in South Africa was established on a national level in 1948, it took almost 50 years to end it.

Yeah, a terrible lack of patience there.

By Ian Gould (not verified) on 14 Dec 2005 #permalink

Proof that not all dictatorships have been overthrown peacefully is not proof of your initial claim that no dictatorships have been

This is true.

Then they proceeded to set up a dictatorship which rivaled that of the PRC in its brutality.

No kidding? I did not know this. How did that change and why? I seem to have made a few gaffs here.

In any event, I'm loathe to leave these matters to the UN. Among the reasons for this, Cuba is a brutal dictatorship, and what has the UN done about it? Oh right, nothing. Pinochet, who is at worst as bad as Castro, and arguably better, is arrested and harassed whenever he leaves Chile, and now inside Chile as well. Castro on the other hand...

"Pinochet, who is at worst as bad as Castro, and arguably better, is arrested and harassed whenever he leaves Chile.."

Arguably better? Leaving that aside for the moment... most of Pinochet's legal problems originate from Chileans. And it was Chileans who have restored democracy to Chile, not "the left" outside of Chile. Should Castro retire, or be forced out of office, don't worry, there will be plenty who want to make him pay.

http://en.wikipedia.org/wiki/Taiwan#RepublicofChina_era

The ROC administration announced October 25, 1945, as "Taiwan Retrocession Day." Reportedly, they were greeted as liberators by the island residents. However, the ROC military administration on Taiwan under Chen Yi, was extremely corrupt. This corruption, compounded with a period of hyperinflation, unrest due to the Chinese Civil War, and distrust due to political, cultural and lingual differences that had developed between the Taiwanese and the newcomers, quickly led to the loss of popular support for the new administration. This culminated in a series of severe clashes between the ROC administration and "native" Taiwanese, in turn leading to the bloody 228 incident and the reign of white terror.
...
During the 1960s and 1970s, Taiwan began to develop into a prosperous and dynamic economy, becoming one of the East Asian Tigers while maintaining an authoritarian, one-party government. Because of the Cold War, most Western nations and the United Nations regarded the Republic of China government on Taiwan as the sole legitimate government of China until the 1970s, when most nations began switching recognition to the People's Republic of China.

After Chiang Kai-Shek died in 1975 his Vice-President, Yen Chia-kan, briefly took over from 1975 to 1978. During the presidency of Chiang Ching-kuo, from 1978 to 1987, Taiwan's political system began a gradual liberalization. Martial law, which had been in effect since 1948, was lifted in 1987. Upon Chiang's death, Vice President Lee Teng-hui succeeded him as president of the ROC and chairman of the KMT, and effective one-party rule was ended in 1991.

The short answer is that as Taiwanese became richer and better edcuated they demanded increased civil rights.

At the same time, the US' support for Taiwan became more conditional and the US was lessing willing to overlook Taiwan's failings.

Chiang Ching Ko deserves credit for dismantling the repressive regiem set up by his father.

Jack Andersen - the Washington reporter who had a history for breaking major stories but also for publishing some pretty extraordinary claims - reported that Chiang Chnig Ko's brother was involved in serious and well-advanced plans for a military coup following his brother's death.

By Ian Gould (not verified) on 15 Dec 2005 #permalink

Given the apparent desire of the USA to spread peace and democracy around the globe, I eagerly await their invasion of Burma. Its as bad and bonkers as any other dictatorship, and its involved in the drugs trade. Plus its not exactly very powerful, so it could be democratised for very little cost.

It is all just a gigantic mess, isn't it? I just think we should turf the UN and start over. Don't let any petty dictatorships in and don't let Libya and the like chair so-called "human rights" commissions. Actually stop genocides when we know they are occuring, and generally stop putting up with the actions of petty dictators outside their countries.

On the other hand, I'm a bit reluctant to want trade and the like to end with these countries, since it often seems that, as Ian pointed out wrt Taiwan, prosperity and liberty grow hand in hand, as in China and not in Cuba.

ben, you seem to be laboring under the assumption that the US is, under the current Administration and previous ones, in the democracy export business. In fact, however, US record on promoting democracy in the world is extremely spotty at best.

The high point is of course post-WWII Germany and Japan. Low points, on the other, include supporting dictatorships in Iran, Guatemala, Chile, Iraq, Cuba and many, many other places. Note that I am not talking about passive support, I am talking about being instrumental in propping up the dictatorship and on occasion overthrowing (or attempting to overthrow) a democracy. This is still going on: e.g., Haiti and Venezuela.

The same goes for human rights. For a country (US) that is committing war crimes (waging aggressive war, mass killing of civilians, torture, use of chemical weapons, and more) on an ongoing basis to complain about other countries' human rights record (say, Lybia) is hilariously hypocritical.

On a more personal note, ben, let me say that I admire the fact that unlike most of us you are readily willing to admit a mistake. I would guess that this is a result of your young age. I'll assume that my guess is correct and take the liberty of giving you advice that I wish someone had given me when I was young.

Your world view is still not set in stone, although it is already heavily influenced by the common wisdom we are all awash in. I suggest that you take a step back and re-evaluate what you think you know about world politics. Try to start without pre-conceptions about who are the good guys and who are the bad guys. Come up with objective metrics, preferably quantitative, for measuring "good" vs. "bad" and then look for data - real hard data, again, preferably quantitative data - to find out objectively who is good and who is bad. Try making your arguments as airtight as you can make them, checking and double checking every assertion.

Try having the metrics reflect your true values. For example, according to my values (but not according to common wisdom), the right the rich to accumulate wealth is less important than the right of the poor to food. You make up your own metrics, but be ready to doubt them and revise them if need be.

This may seem like a lot of work, but since we are talking about a world view that should serve you for the rest of your life, it is well worth the effort.

Best wishes.

Who appointed you ben? It strikes me you combine a lack of knowledge with a feeling of certainty in exactly the proportions that made these messes. And, btw, while you are at it, how about adding Mexico and Mississppi to the list, although arguably the former has come further.

So I'm assuming that Ben will be lobbying for the end of trade sanctions against Cuba?

As a non-American I have some difficulty understanding the US attitude to Cuba - it's a small country with a small communist dictator who on the scale of baddies in the history of the last 50 years really isn't that high up on the list.

"As a non-American I have some difficulty understanding the US attitude to Cuba "

Well, the Florida panhandle (i.e. America's dong) contains the refugees who fled Castro's takeover; and unlike the average refugee, this group has a median wealth higher than the American average, as well as a solid understanding of how to play the political game, an extreme rightwing philosophy, and a hatred of Castro that knows no bounds. As seen in 2000, antagonizing the Florida power structure is not a way to win elections.

For other examples:
http://en.wikipedia.org/wiki/Orlando_Bosch
http://www.guardian.co.uk/usa/story/0,12271,851913,00.html
and of course, who could forget:
http://www.cubdest.org/0002/c0024cu.html

Er, young? Well, if 33 is young. It's not that I haven't formed a coherent world view, it's that my knee-jerking on this forum is just as bas as every other knee-jerk's work everywhere else.

See, I have about 3 minutes of free time every day, so the 5 minutes I spend here is entirely in haste. I throw out some bunk that easily stirs the pot, but then that way I get a lot of interesting material back. Some of it I learn from, i.e. John Lott, and global warming, and some of it I heartily disagree with, i.e. welfare-state or socialist economics and that the US is committing war crimes in Iraq. Some times I get people calling me an idiot, and other times not.

Interestingly, my father-in-law thinks Castro will be remembered as the greatest leader of our age. Even more interestingly, we've never had an argument about that. There's one pot I can do without stirring.

Ben,

Its no use disagreeing with the FACT that US forces have committed numerous war crimes in Iraq. Anyone with a semblance of reality would realize that the US has not only committed numerous war crimes in Iraq (the evidence is overwhelming, even if our media ignores it) but in other conflicts as well over the past 50 years. Read any of Dahr Jamail's reports and its blatantly evident. I find your denial amusing and sad more than anything else. Similarly, its hard to actually swallow that some people cling to myths, long since demolished, that the US-UK axis support human rights, freedom and democracy.

By Jeff Harvey (not verified) on 16 Dec 2005 #permalink

Jeff Harvey- You are letting your emotions, and in particular hatred, cloud your judgement. I have read Dahr Jamail's posts from time to time. I have even looked at some of the ghastly pictures on his site. I even accept the Lancet study results. Nonetheless, my opinion on the Iraq War has not changed one iota.

I can only suggest that you delve further into history. If you do you will note that hegemony has never been pretty. Secondly, think about what it would be like to live in a world under Soviet or Islamo-fascist hegemony. It ain't bloody pleasant to contemplate. For one thing, you wouldn't have the option of whining on forums such as this.

By Steve Munn (not verified) on 16 Dec 2005 #permalink

Steve Munn, I'm not sure what point you're making. You're right--it's much better to live in the US or any prosperous democracy than in some dictatorship. But that doesn't mean a democracy can't be guilty of major war crimes--look at the history of almost any powerful democracy (I'm assuming, perhaps wrongly, that weak ones don't have as much opportunity to step on people) and you'll usually find a bunch of skeletons is hidden in their closet.

So when a democracy is guilty of war crimes it is the responsibility of the citizens of that country to point them out and try to stop them. I can't tell if you agree or disagree.

By Donald Johnson (not verified) on 16 Dec 2005 #permalink

"So when a democracy is guilty of war crimes it is the responsibility of the citizens of that country to point them out and try to stop them. I can't tell if you agree or disagree."

That would kind of be the point of living in a democracy, something which seems to elude the "don't criticize the president during a war" folks.

I regard the UN as the best of a bad job. If you got rid of it and started all over again, keeping out countries with dictatorships etc, you end up with a small self referential clique. If you let them in, you stand a chance of altering attitudes etc. I think part of the reason the USA is getting so much stick over Iraq (apart from the lack of legitimate reasons for the invasion, and no, "to spread freedom and democracy" is not enough) is that the idea of human rights and how bad genocide etc is has spread globally. Maybe we've learnt something from the 20th century? I'd like to think so.

A problem is that even if you got rid of the dictatorships from the UN, your still left with the question of who shall commit the resources needed to stop genocide? its all very well Ben saying that Darfur should have been stopped (with which I agree) but due to various circumstances, most of Africa is unable to provide the necessary resources, meaning that countries like the USA and UK etc have to be willing to spend the money. I dont recall anyone from either country really agitating for said expenditure. (I am a UK citizen).

"I regard the UN as the best of a bad job."

Indeed. I would assume it's designed to give a forum where disputes can at least stand a chance of being argued out instead of battled out. Eliminating it because of the squabbling is akin to eliminating lawsuits because of all the squabbling that goes on there. Which come to think of it, really is another brilliant rightwing goal.

I regard the UN as the best of a bad job. If you got rid of it and started all over again, keeping out countries with dictatorships etc, you end up with a small self referential clique.

Yes - and one which would probably expel the US for its continuing use of the death penalty and refusal to sign up to various huamn rights and weapons control treaties.

By Ian Gould (not verified) on 16 Dec 2005 #permalink

One other small point for Ben, West Berlin was created out of diplomacy. Although the division of Germany was settled at Yalta, it would have been impossible to turf the Soviet Union out of the rather large areas that they had occupied by defeating the Germany armies. You might also remember that it was diplomacy which got the Russians out of Austria in 1955.

Ben:

Er, young? Well, if 33 is young.

Somewhat older than I thought, but still young enough to be taking some advice.

By the way, if the objective of your posts is to get some "interesting material" back (an objective that makes you quite different than the standard pot-stirrer who has no use for information), why don't you simply ask?

Sorry to be posting on this so late. I don't know if many people will read this comment.

Anyhow, I telephoned Peter Baker on the day this article came out and asked him about the paragraphs in question. He emphasized that they weren't original reporting on his part, and simply represented a summary of news clippings about previous estimates or counts of Iraqi dead.

This brings up a conundrum. Once the paper reports something about the Lancet study, future reports referring to the study will not likely alter the claims about it, unless they constitute new reporting on it, rather than just summarizing old articles. Yet what is currently newsworthy that would also correct the record?

In addition, Baker told me that the Iraqi government study was conducted by the ministry of the interior a few months earlier and that I should be able to find it by checking news clippings (I could not, unless he was referring to the Iraqi Living Conditions Survey).

When I asked which human rights groups considered the number inflated, he mentioned Human Rights Watch as quoted in the Post. He did not name any other human rights groups or "other experts", but referred me to news clippings from the time the report came out.