More on surveys of violent war deaths

Do you have any thoughts on the recent Burke et al. paper Warming increases the risk of civil war in Africa. It claims to have found a small but robust correlation between war-deaths recorded for sub-Saharan Africa in the PRIO dataset and country-scale temperature increases in the period 1981-2002 and says that 'this historical response to temperature suggests a roughly 54% increase in armed conflict incidence by 2030, or an additional 393,000 battle deaths if future wars are as deadly as recent wars.'

I'm no statistician but ...

The period under review neatly straddles the end of the Cold War, a continent-wide 'external forcing' that the study totally ignores. Here's an annual tally of African conflicts listed in the PRIO database. Is there a clear post-Cold War jump? Perhaps not. But I bet the apparent jump is more 'robust' than any correlation with temperature.

I haven't had time to read both papers yet, but this looks really interesting. There are some thoughtful comments accompanying [another Gelman article at Statistical Modeling, Causal Inference, and Social Science](http://www.stat.columbia.edu/~cook/movabletype/archives/2009/12/update_…).

The PRIO battle death data looks fascinating. The Obermeyer paper says that the PRIO "compiles and reconciles
passive reports on violent deaths related to war globally
from 1900 onward," but that doesn't appear to be at all an accurate description, based on the [PRIO data's documentation](http://www.prio.no/sptrans/1555324504/PRIObd3.0_documentation.pdf).

"The regression that used . . . " maybe s/b "The regression that they used . . . "?

Thanks for keeping this awful stuff in the public eye.

A recent paper from Spagat, Johnson, et al (on conflict deaths in Iraq, Afhganistan & Columbia) made the cover story of Nature last week:

http://www.nature.com/nature/journal/v462/n7275/covers/index.html
http://dissident93.files.wordpress.com/2009/12/gourley_nature_cover.jpg

Knock on wood, but in Big Lebowski terms, it seems our Donnie on this issue has stopped wandering in and saying "I am the Walrus."

However, it also looks like the registration system at pharyngula is causing the insane to spill out all over scienceblogs, the way the crazed homeless went out after Reagan closed the mental health facilities in California.

Tim

There is indeed evidence that people misremember past deaths. [See](http://www.ingentaconnect.com/content/apl/spub/2003/00000031/A062s062/a…)

Like you, I found it odd that individuals might misremember the death of siblings. But remember that the WHO surveys have recall periods going back 40 odd years. (The SMART protocol states flatly that recall periods should not exceed ONE year because of concerns about recall bias.) Remember too that many of the individuals who were asked about the violent deaths of siblings may well have been children when the wars were being waged. It wouldn't be so surprising if they misremembered the death of an infant brother or sister.

I should note though that the recall bias issue is peripheral to our critique of Obermeyer et al.

On population health surveys and excess death estimate more generally, the Human Security Report Project has a study due out on January 20th which examines the use of surveys to examine all excess war deathsââthose from disease and malnutrition as well as violence. It makes the case that, while such surveys are a vitally important source of data in war-affected poor countries, major challenges arise when they are used to estimate excess deaths.

The challenges lie not so much with surveys themselves, but with how to figure out the baseline mortality rate. Here the need is for pre-war trend data, not just a point estimate. The counterfactual 'what would have happened had there been no war' has to be taken into account in estimating excess deaths, but this requires understanding not just the mortality rate in the immediate pre-war period but the trend in mortality over several years.

Peacetime mortality rates are declining in almost all poor countries where most wars are waged today and if this isn't taken into account in estimating excess mortality, the death toll will be underestimated.

But, as our report points out, the practical challenges of "measuring from the slope" of a declining (or increasing) mortality trend line are almost insurmountable.

Census data can sometimes be used to estimate excess deaths from all causes, but incident reporting of the sort undertaken by PRIO and Uppsala can't be. There is no way of determining from reported deaths whether those who succumb to disease and malnutrition in wartime wouldn't have died anyway had there been no war.

Andrew

You folks sure like to think about who died... why did they die... should the memories of survivors count? Right through the New Year, you guys keep thinking for the unwashed, bloody sheep... Help us please with the wounded numbers in all these and more conflicts, would you please? The wounded divided by the dead. In most wars the ratio runs from 3-5 for each person killed. Where can I find these numbers please? In Iraq? In Gaza? In all areas of the world I have watched as the number of wounded has shot up over the years. In "Intifada"(to shake off), 1 & 2; this ratio got to well over twenty to one. To me this says that people are being wounded for sport by soldiers. Is this a war crime if proven to be true? It is not because of improved medical care. In most cases this only applies to troops in the area of operations. Not civilians. So please, do not bring up this as a red-herring. Also why don't the type of wounds count in your reports. How may shot in the hand? Arm? Head? Center of mass? Leg? You all surly see how important this data will be, we all can... Where can I get that info? Thanks to all.

Gelman writes:

We last heard from Dr. Spagat when he shot down the notorious article in the Lancet by Burnham et al. that estimated post-invasion deaths in Iraq using a sample survey.

Tim: Would you agree with Gelman's assessment? If not, you should explain where he goes wrong. I, of course, agree with Gelman that Burnham's article was "notorious" and that Spagat succeeded in shooting it down.

Tim's claim that you'll always know of an overestimate because of one obvious cluster seems like a baseless claim, to put it mildly.

Tom

I don't think there any good answers to your question.

Research organisations like the International Peace Research Institute in Oslo and Uppsala University's Conflict data program don't try and collect injury data because they believe the reports are just too unreliable. Getting good data on fatalities is hard enough. Plus what constitutes an "injury" can vary dramatically from place to place. A minor wound that didn't prevent an individual continuing in a combat role might counted as an injury in some contexts -- not in others. But there is little room for ambiguity when someone is killed.

One of the few serious studies we know of notes this:

"The ratio of the number of people wounded to the number killed ranged from 1.9 to 27.8. Two additional articles gave the proportions of people wounded who eventually died in major conflicts since 1940, without giving absolute numbers. Total deaths were never more than 26% of all casualties, a wounded to killed ratio of 2.8."

This is from a lit. review in 1999 by Robin Coupland and David Meddings, both then at UNHCR. See:http://www.ncbi.nlm.nih.gov/pmc/articles/PMC28193/

I think that there is actually quite a lot of evidence that the ratio of injuries to deaths is much higher in modern armed forces like those of the US because excellent first aid facilities prevent soldiers from dying who would certainly have died in earlier periods of warfare.

In most poor country wars people bleed to death from what would otherwise be non-fatal wounds because there are no medics to stop the bleeding, or medevacs to get them quickly to a field hospital. So here the ratio of deaths to injuries is low.

I think quick access to medical services would likely also explain the Palestine case where political violence takes place in urban settingsââas against remote jungles as is the case in many conflicts in the developing world. Civilians get treated as well as fighters here. Moreover Palestinians have health services that are far superior than those in most wars in Africa and elsewhereââand lots of practice in using those services to treat victims of violence.

Andrew

David Kane: "I, of course, agree with Gelman that Burnham's article was "notorious" and that Spagat succeeded in shooting it down."

Here's what appears to me to be the most relevant quote from Gelman at the link: "P.P.S. A reporter called me about this stuff a couple months ago, but I'm embarrassed to say that I offered nothing conclusive, beyond the statement that these studies are hard to do, and for some reason it's often hard to get information from survey organizations about what goes on within primary sampling units."

I feel this shows David Kane to be a person of dubious character and that Andrew Gelman may be prone to loose language on occasion. I've no doubt whatsoever that Kane can take Gelman's comments far further than Gelman has and far further than he would want them taken. That's what people like David Kane do, the devil finding work for those with idle minds etc etc.

Anon, I don't think it's fair to criticize David, when he's quoting Gelman accurately. But I think you're probably right about Gelman, since his "shot down" link goes to a [March 2008 post](http://www.stat.columbia.edu/~cook/movabletype/archives/2008/03/ethical…) in which he admits that he hadn't read Spagat's paper. Presumably he's read it since then, and agrees with it... but if so, he ought to have said something to that effect.

DK: "Would you agree with Gelman's assessment? If not, you should explain where he goes wrong"

I've already explained, in detail, where Spagat goes wrong. One of the posts is even linked in my post above. Gelman has not offered any assessment of the Lancet studies.

Alas, we didn't make it through the movie before the child started speaking.

I recall Les Roberts, Gilbert Burnham and Medialens (who were, IMO, Roberts's inept proxy for attacking Iraq Body Count) touting the false claim (wrt the Obermeyer study) that:

"But a study of 13 war-affected countries presented at a recent Harvard conference found that more than 80 percent of violent deaths in conflicts go unreported by the press and governments."

http://web.mit.edu/humancostiraq/further-reading/burnham-roberts2007.ht…

http://tinyurl.com/yfdh3l2

http://www.medialens.org/forum/viewtopic.php?t=2613

Robert,

Medialens were the "inept proxy" in your frankly inept opinion. I think Medialens did a great job explaining exactly why the mainstream (= corporate) media downplayed studies reporting the horrific toll of the illegal invasion of Iraq; had the Lancet studies been based on body counts as a result of the military activities of "officially designated enemies", and had these had been juxtaposed with those of Iraq Body Count, then I am sure the media would have focused on the larger toll for purely propaganda purposes. The bottom line is that the western media constantly downplay or ignore crimes committed by 'us' and our proxies whilst focusing laser like on crimes committed by official enemies. This is hardly suprising, since the msm is generally owned by commerical elites who clearly see the media as a means of promoting elite explanantions.

The IBC toll of the carnage in Iraq - almost certainly way too low - was only given full atterntion by the msm when the Lancet studies were published. The reason for this should have been obvious; in public relations terms the media were desperate to 'manage the outrage' that may have resulted had it been made clear that the real death toll of the Iraq war was many hundreds of thousands or even more than a million. Suddenly IBC came in from the cold because 30,000 deaths sounded 'acceptable' whereas 600,000 plus did not. What I found most annoying about IBC was that, when Bush and Blair began to downplay the death toll in Iraq using IBC figures, IBCs response to this abuse was apparent silence. They should have spoken up and argued that the war was illegal and that the death toll was horrific and that the war parties should not be using their own figures to defend the war. But I did not see that happen. The war party was allowed to abuse IBC figures to downplay the human toll of the war. The media went along for the ride.

I appreciate the enrormous efforts made by David Edwards and David Cromwell and Medialens to expose the media's cupability as 'vital cogs in the machinery of industrial western killing' as well as the hypocrisy of the IBC. As far as I am concerned, Robert, you should be expending more effort in exploring the real reasons for US military adventures and occupations in Asia and the Persian Gulf and its human toll, instead of attacking those who expose western actions and the actual motivations behind them (e.g. imperial expansion and efforts to prevent the rise of 'near-peers'). For that I salute Medialens, Pepe Escobar, William Engdahl, Paul Street, Tom Engelhardt, Andrew Bacevich, Noam Chomsky, John Pilger, Robert Fisk, and others who are saying it like it is.

Thanks for the lecture, Jeff. I too "salute" Noam Chomsky and Robert Fisk (or rather I greatly admire their work) but, unlike Medialens, they're generally careful with the facts, and they don't conduct smear campaigns.

If you're interested in why I find Medialens inept on this topic (I doubt, somehow, that you are), see my ZNet article which lists their many errors on IBC:

http://www.zmag.org/znet/viewArticle/22309

See also the following piece on their heavy-handed censorship, which also mentions how ZNet recently refused to publish their attempt to smear George Monbiot:

http://dissident93.wordpress.com/2010/01/04/medialens-embarrassing-arch…

Robert,

Methinks you are resorting to the word "smear" too easily here. Medialens were annoyed with IBC for the same reason that I alluded to above. IBC appeared unconcerned that their estimates of deaths in Iraq were being (ab)used by members of the war party to downplay the carnage. Bush was asked what he thought the death toll of the Iraq invasion was (back in December 2005 or 2006 I think it was) and he responded with the 30,000 estimate that appeared to come straight from the pages of IBC. The next day I expected those at IBC to respond angrily, accusing Bush of misinterpreting the intentions of their survey. Did they? Did they hell. Their only response was silence, as if they were pleased to see the msm and western leaders who waged or supported the war citing their work. Their silence spoke volumes in my opinion.

You also appear unconcerned at the much more egregious smear campaign that was launched at Roberts, Burnham, Gilbert et al. after the publication of their Lancet studies. Given the results of these studies (subsequently supported by the Opinion Business Research survey) apparently showed western culpability in mass murder, a finding which contradicts the carefully cultivated image of our basic benevolence, it was hardly surprising that the corporate media did everything it could to marginalize them.

But I digress. As I said above, given the underlying actual causes of the conflicts in Iraq and Af-Pak, I am bemused as to why you expend such efforets at defending IBC. Fiddling while Rome burns is my take on it. As for Medialens, I admire their tireless efforts to expose the msm for what it is: subservient to commerical elites and powerful vested interests. Their two books, "Guardians of Power" and "Newspeak in the 21st Century" are outstanding reads. They have criticized George Monbiot in the past for some of his views, and good for them. I admire Monbiot but the so-called "liberal" media, which is hardly liberal as far as I am concerned, needs to be held to account. We know where the right wing press stands, but as David Edwards and David Cromwell have shown, it is the allegedly liberal press which appears equally beholden to power. Along with the likes of Robert McChesney, Richard Falk, Fairness in Accuracy and Reporting, Democracy Now and other outlets, it is good to see those media sources apparently championing liberal causes to be held to account.

"Apparent silence", "appeared unconcerned", "fiddling while Rome burns" - the way Jeff Harvey frames his "criticisms" of IBC reminds me of Frank Luntz's manual - or perhaps of witch-hunts from earlier times.

Yes, of course, one has to be really "unconcerned" to devote nearly seven years of one's life, as a volunteer, to corroborating and documenting violent deaths. It takes so much more concern and integrity to lecture people on web forums about how they should spend their time, and on what premises they must accept to avoid the accusation of "being silent". Of course, of course.

Robert,

Your rather shallow posts shed a considerable amount of light on the way that you think. All of the gibberish you wrote in post 19 could be equally (if not more so) applied to people like Les Roberts and Gilbert Burhnam who are both professional epidemiologists. Yet I have never detected a scintilla of concern from you over the way these scholars have been treated and dragged through the mud. Have they not dedicated their lives to uncovering the scale of the carnage in Iraq? Where are your sympathies for them? After all, the IBC investigators are not professional epidemiologists but (as far as I can tell) have careers in other fields like the arts.

The IBC team has not, as far as I have seen, responded to the clear abuse of their 'estimates' by defenders of the war party, including those who carried it out (e.g. Bush and Blair) nor of the intent of those in the media who routinely cite IBC estimates as 'facts' whilst downplaying or ignoring other studies that have produced much higher estimates of death. Care to speculate why the IBC team have remained silent? Or why the corporate/state msm often treat IBC estimates as reliable while ignoring or attacking the Lancet findings? I have my own suspicions. In fact, it should be patently obvious. But I have yet to see you discuss this rather important point on your web cite or elsewhere. Why not, Robert?

In my opinion you are full of hot air. Lots of bluster but little substance. And especially little when it concerns the Anglo-American *modus operandi* or *casus belli* with respect to the illusory 'war on terror' or of the consequences of our ongoing military and economic policies in the Middle East and Caucasus region. Its just the same 'attack Medialens for daring to criticize IBC' refrain.

To give just one example of BBC bias, today their is an on-line discussion asking the question:

"Can the UK and US prevent extremism in Yemen?"

Again, all of the usual assumptions are packed into this one question. First of all, it assumes that the intentions of the US and UK governments in the region are purely benign; second, that our governments are anything but extreme in the policies they pursue. In other words, western policies are always packaged by the msm into heart-rending tales of concern for human rights and freedom, irrespective of the actual facts on the ground. Crimes committed by the west are gnerally ignored, or forgotten. The real agendas are seldom, if ever, disclosed. This is the kind of media hypocrisy that Medialens challenges on its web site. And I for one think that they do a fantastic job.

Jeff writes:

Yet I have never detected a scintilla of concern from you over the way these scholars have been treated and dragged through the mud.

That shows only that you haven't looked very far. I once highlighted the contrast in BBC coverage between the Tsunami deaths (which BBC covered daily) and Iraqi deaths (eg the Lancet 2004 estimate, which they headlined on one day, but then forgot about). In fact, Medialens (no less) once used my material (correspondence with the head of BBC news) in one of their alerts. I've also defended Les Roberts in the past from attacks by rightwing pundits. (That was before Les Roberts ran for Congress, before he started attacking IBC via Medialens, and before he said that the shock-and-awe campaign was "very careful").

Not that I should have to defend myself against accusations of "appearing silent" on issues of your choosing. When witchfinder-generals such as yourself start to accuse people of "appearing silent", you might want to take a look at yourself. (Incidentally, I can tell from your ignorant remarks about IBC that you haven't spent much time looking at what they've written. Clue: they have addressed Medialens's baseless accusations in full. But Medialens have "appeared silent" on that, which is possibly why you don't seem to know about it).

Robert,

You seem singularly obsessed over this IBC-Medialens issue. Get a life, pal. There's far more important issues to address in this world. It seems that you have dedicated yourself to the IBC cause. Are they not intelligent enough to take care of themselves?

I know that IBC have responded to Medialens criticisms. I just do not know why they have not expended so much effort to counter msm articles which cite their estimates for the death toll in Iraq as a means of downplaying the results of other studies that produce far higher estimates. Curious, that one. Or perhaps not.

Do you not think you are making a mountain out of a molehill? What about all of the other material medialens covers? Or does this IBC thing stick in your craw for some reason?

Jeff writes:

Do you not think you are making a mountain out of a molehill?

Well, I'm responding to accusations. I guess it depends on whether you consider your own (actually Medialens's) accusations to be mountains or molehills.

shorter Robert Shone:

Comment by Robert Shone blocked. [unkill]â[show comment]

funnier Marion Delgado:

http://fascistoar.blogspot.com/

Ater having read both the OMG paper and the Spagat/Mack/Cooper/Kreutz (SMCK) paper, I'd say that the SMCK paper makes a convincing case that there are significant flaws in OMG. Previously, I didn't understand at all how OMG arrived at their conclusion that war deaths in the most recent period were higher than they were in the 1965-1974 reporting period. After reading SMCK, I understand how OMG came to that conclusion, and I think their method was just plain wrong.

Like Tim, however, I have a few minor misgivings. Pointing to the [SMART Protocol](http://www.smartindicators.org/SMART_Protocol_01-27-05.pdf) as though it were some sort of definitive authority on what should and should not be allowed in mortality surveys strikes me as just plain silly. The introduction to the SMART Protocol states that "This manual is designed to be used in conjunction with the accompanying software. NUTRISURVEY for SMART which is free from www.nutrisurvey.de/ena/ena.html." I don't see why the software manual for a nutritional survey should be regarded as gospel for demographers who are trying to assess war casualties.

In general, yes, longer periods would lead to greater uncertainty, but a one-year limit is arbitrary and often unrealistic. The Protocol notes that the recall period is inevitably a compromise: "If the recall period is too short, a very large number of households will need to be visited and interviewed, which makes the survey unwieldy."

And while I agree that surveillance projects like the Iraq Body Count provide an absolutely essential metric, I'm only in partial agreement with SMCK's statement that these projects "reveal trends - whether things are getting better or worse - and can do so in near real-time." They can reveal trends, but they don't necessarily reveal trends. As SMCK points out in their own report: "Governments may forbid reporting of war deaths - particularly of their own forces. Journalists and other observers are sometimes banned from war zones, as in Sri Lanka in 2009, or may stay away because conditions are simply too dangerous. And in wars with very high daily death tolls - like Iraq in 2005 and 2006 - violent incidents with very small numbers of deaths may go unreported." Those factors can have significant effects on our perception of the overall trends.

Regards,
Bruce

Bruce wrote:

Pointing to the SMART Protocol as though it were some sort of definitive authority on what should and should not be allowed in mortality surveys strikes me as just plain silly. [...] I don't see why the software manual for a nutritional survey should be regarded as gospel for demographers who are trying to assess war casualties.

Well, I agree that it's not definitive gospel but I don't think it's silly to have good guidelines gathered together in an accessible document. Doing field work under crisis conditions is not a walk in the park so almost any advice you get from people who've actually done it helps you avoid egregious errors. I have no particular problem with the SMART protocol if it's viewed in the context of "best practices under way less than best conditions." However, like many manuals of the sort, it's limited in what it can and cannot cover. One of the topics that field manuals just don't have the room to cover is "how rotten will the estimates be if you don't follow best practices to the letter?" Of course, you *want* to follow best practices but often in the field you can't. Thinking that any deviation from best practices vitiates an entire study is the fallacy of the "Aha! I found a critical error" type. You can see lots of evidence of that in this topic. Some errors are far more egregious, and therefore consequential, than others.