More on ORB survey that found one million deaths

ORB and IBC are counting different groups. IBC attempts to count civilians only. ORB counts all Iraqis that died violently because of the war, which would include soldiers of the Saddam regime and current Iraqi government and insurgents. They don't necessarily contradict each other.

Roberts is wrong in claiming that IBC attributes more deaths to car bombs than gunshot.

Dangit. Go offline for a long weekend and Tim beats me to the punch (sorry, Tim, I read Kleiman before you).

I suggest that the screeching chimps who wish to deny Iraqi deaths look at the chart Wimberly made before posting your chimpchatter here.

Best,

D

I have referred to this article a couple of times. It could be subtitled "How genocide can be committed with small arms, particularly AK-47s". With a small number of men, using only Kalashnikovs, Carles Taylor (now on trial for war crimes) was able to murder 1,000 people in a Ghanaian village in a single day.

http://www.grip.org/bdg/g1782.html

So for a high Iraqi death rate, you do to have to appeal to ari strikes or car bombs. Plain old bullets will do just fine.

Note the date on the article - 2000. How little has changed.

The recent ORB poll suggest that there have been more than a million Iraqi violent deaths because of the invasion, (and about 800,000 of those in Baghdad, based on 206 interviews in that city.)

This is newsworthy if true, and ORB has issued a press release to that effect. They've missed an even bigger story. Based on the same 206 interviews, in theory selected entirely at random from the Baghdad population of about six million, Baghdad is mostly Christian: 37% Orthodox, 13% Catholic, 9% Protestant and 1% Christian (page 46).

One possible explanation is that the ORB interviews were entirely random, and that the Baghdad population is actually about 60% Christian, plus or minus random sampling error.

Another, which I prefer, is that it is very difficult to do a true random sample when your next planned interview is on the other side of a checkpoint manned by heavily armed locals with a dim view of outsiders who want to ask possibly inconvenient questions.

This would explain another puzzle in the ORB results. They imply that almost one in two Baghdad households have lost a family member. Both media reports and previous polls have indicated that the Iraqis being killed are overwhelmingly adult males: they are both more likely to be targeted and more likely to be exposed to attack. Iraq Body Count estimates that 90% of the civilian deaths are adult males, and including soldiers and insurgents would make the ratio even more extreme. Iraq is also a young country, with about half the population under eighteen. One would expect that if the Baghdad households sampled are like typical Iraqi households, those deaths would significantly impact the ratio of male to female adults answering the poll. This does not occur in the Baghdad poll results.

It seems likely that something decidedly unrandom has happened to the Baghdad sample. Or some Baghdad respondents are using an expansive definition of household that includes, say, everybody in their apartment building. Or both.

Will McLean:

I'm not sure what it is, but I know for a fact that the "206 interviews" number that you quote is not the total number of interviews done by ORB in Baghdad.
According to these charts put out by ORB, 23.4% of 5,019 total interviews = 1174 were carried out in Baghdad.

So, 206 is only 18% of the 1174 total done in Baghdad.

Here's the breakdown ORB gives for the interviews for the entire country by religion (from the same ORB charts presentation)
Religion
Sunni Muslim42%
Shia Muslim50%
Muslim (unspec.)7%
Other 1%

Christians would fall in that 1%, of course.

If 60% of the interviewees in Baghdad had actually been Christian (as you indicate above), that would mean that a minimum of (60%)(23.4%) = 14% of the total interviewees (for the entire country) would have been Christian, which does not jibe with the 1% given by ORB for the whole country in the ORB charts that I linked to.

Something is clearly the matter with the numbers you give, but I'd be willing to bet that the problem is with something other than ORB's random sampling procedure.

Even if we assumed that of the interviewees, all the "unspecified" were also Christian and that all of the "1% other" were also Christian, that would only make a total of 8% Christian -- ie, still not sufficient to make 14% (see above)

JB:

You are looking at a different, earlier ORB survey. They have done several.

The one that yields the 1.2 million dead estimate was taken in August 2007 and released in September.

highly unlikely that "Orthodox" in context means Greek or Russian Orthodox Christian. Might it not be a self-description of some Muslims?

They used the term "Orthodox" in the earlier poll which JB refers to, but with just a tiny number of respondents, as one would expect. My guess is that they just screwed up when they were compiling the tables. Paging David Kane!

Will, you're right, that's very strange. According to the table on page 43 most of the "Other" religion respondents were Arabs, and there aren't very many Arab Christians in Iraq. Either something is wrong with the sampling or the coding of the results.

The survey seems to overestimate the percentage of Christians in the whole country - according to Wikipedia, Iraq is 97% Muslim, 3% Christian. The survey reports 82% Muslim, 18% other. The Baghdad numbers skew the national figures.

It gets the ethnic composition roughly correct, reporting
76% Arab, 15% Kurd, 7% other against Wikipedia values of 80% Arab, 15% Kurd, 5% other.

Since Christians are mostly Assyrians or Chaldeans, the accuracy of the ethnic composition numbers suggests an error in the religious composition reporting. Unless, of course, there is another reading of the survey that I am missing. I would certainly like a comment from ORB.

It has also been pointed out that this survey reports a higher level of violence in Baghdad than the Johns Hopkins teams found. These discrepancies do not necessarily invalidate the results.

Hmm, the ORB poll earlier this year had 3% of Baghdad being Christian, which seems more likely. I've emailed ORB.

Apologies, the survery actually reports 12% Kurd and 12% other - mistake on my part. The Other category is some what inflated, the figures for Arab and Kurds look roughly correct.

Differences reflecting population shifts within the country?

According to Wikipedia, Baghdad always had a large Christian community, so its no surprise if most Christians would live there.

I barely recall stats class as I was still in a drunken stupor at the ungodly hour of 8 a.m.
How the hell do they get 1,000,000 deaths from a survey of less than 2,000 peeps?

"I barely recall stats class ... How the hell do they get 1,000,000 deaths from a survey of less than 2,000 peeps?"

You are in serious danger here. Wise choice to use an alias, cause otherwise, your stats teacher is going to throttle the life out of you (if they have to rise from the grave to do it).

Seriously people, do you know how much this sort of thing humiliates the instructor? The most important thing that you are supposed to get from a stats class and he couldn't steal a clue

Wildlifer,

To answer your question, it's roughly equivalent to doing something like measuring the height of 2,000 adults from around the US. If you do it right, when you're done you'll have a pretty good idea of the average height of the several hundred million people you didn't measure.

gz

Clear evidence that an alcoholic stupor is deadly to learning and knowledge. If you didn't get an F, you should get one now, retroactively.

And what's frightening is that he's not the worst commenter on these types of threads, not by a long shot.

wildlifer, if you don't believe in random sampling, next time the doctor orders a blood test, tell 'em to take it all.

Either ORB screwed up royally or they have got cold feet. All reference to the study has disappeared from their "Newsroom" page.

Really in this day and age a polling outfit should know better than to just pull the item off their website. An explanation is called for even if it's only "We're looking into it".

That's pretty bad. You don't make a public statement about something this important and then quietly pull back without explanation.

Weird. The study page is still on their website, it just doesn't come up on the main "Newsroom" page anymore.

I still found the survey at this link to the newsroom so ORB have not backed away from it.

http://www.opinion.co.uk/Newsroom_details.aspx?NewsId=78

Like Tim I mailed them about the Baghdad figures yesterday. No reply as yet.

It may be something as simple as misclassfying Muslims as Christians. Certainly, a lot of Christians in the Baghdad sample could explain why this survey shows violence in Baghdad to be so high. Christians have been the victims of both Shia and Sunni death squads, and many have fled.

http://www.topix.com/world/2007/05/iraq-baghdad-christians-flee-as-viol…

I accept the Lancet/ Johns Hopkins surveys, but as scientists we should be healthily sceptical of all empirical results until they have been confirmed a few times. The ORB poll looks to be the first confirmation of the earlier surveys, however we need any apparent discrepancies to be clarified.

Correction. If you click on this link, the poll has been removed.

www.opinion.co.uk/newsroom.aspx

Correction. If you click on this link, the poll has been removed.

www.opinion.co.uk/newsroom.aspx

Toby, if you had waited a little longer you wouldn't have needed to correct yourself - the report is back on the Newsroom page, together with a new item concerning plans to poll rural areas which were previously neglected:

"While, for obvious reasons, we cannot boost our representation of people living in Iraq's most violent areas we have decided - following feedback from readers of our poll - to conduct a more extensive survey of rural areas to see how this may impact on our estimate. We are in the process of conducting additional interviews in rural areas of Iraq. Once this data has been verified and merged with our current data set we will post it here on the ORB website. We aim to be in a position to release this data within ten days i.e. first week of October."

The religion question may have suffered from the same bias that resulted in so many attributions of deaths to car bombs. I suggested in my post that respondents might not want to get into dangerous areas in conversation with strangers - and revealing religion would be one. Ethnic cleansers might pose as harmless pollsters. They surely wouldn't go to the trouble of visiting houses pretending to be doctors, as with the Lancet surveys.
(sniff) I wonder why the trolls haven't gone for my chart yet.

Wimberly:

In a country where churches get bombed and Christians beheaded, do you really think somebody is going to falsely identify himself as Christian rather than just saying "Muslim" or "No comment"?

To me the most likely explanation for the puzzling number of Christians is a coding cockup. Can anyone explain to me how the unweighted number of 143 "Other" (denominations) becomes 262 except by mistake?

Kevin writes:

"Can anyone explain to me how the unweighted number of 143 "Other" (denominations) becomes 262 except by mistake?"

Looking on page 46, you will see that 251 of the weighted "other" count comes from Baghdad. Since the raw responses from Baghdad are more than doubled to get the weighted count, that part doesn't seem to be a coding cockup per se.

They're professionals and have had the problem pointed out to them, and a coding problem is an obvious explanation. I'm sure that they are double checking that or have done so. If it's coding, they'll fix it and report it.

Based on their update, I think they've looked for a coding problem, but not found one.

The survery is back up and now gives Baghdad as 95% Muslim, 3% Christian, Others too small to quote.

National figures give 97% Muslim, 3% Others.

Ethnic composition gives 78% Arab, 12% Kurd, 6% Other, 4% Don't Know, same as before.

There are no other changes as far as I can see.

Coding error?

They've put up a revised chart on the ORB site. The old version is still up at: http://www.opinion.co.uk/Documents/TABLES.pdf

Those interested may want to save a copy before it vanishes down the memory hole.

Thanks for the response Will, and thanks for the update Toby. I suppose it could have been something as simple as faulty translation.

Looks like something over a hundred responses from Baghdad only (weighted to 200+) got their religion changed, with changes in every category, and all Muslim categories and unspecified Christian showing gains.

Will: My suggestion doesn't apply to self-identifying as Christian (I agree this would be risky and you would expect under-reporting), but it could explain lots of "others", subsequently miscoded.