LancetIraq
tlambert | October 30, 2006The BBC has Les Robert's answers to questions sent in by readers. Some extracts:
A research team have asserted in an article in Science that the second Lancet study is seriously flawed due to "main street bias."
We worked hard in Iraq to have every street segment have an equal chance of being selected. We worked hard to have each separate house have an equal chance of being selected. Realize, there would have to be both a systematic selection of one kind of street by our process and a radically different rate of death on that kind of street in order to skew our results. We see no evidence of…
tlambert | October 27, 2006This is a talk about the methodology of the 2004 study, but most of it applies to the new study as well. It's 46 minutes long.
Via lenin.
tlambert | October 25, 2006
Tim Blair isn't going to let go of his claim that Richard Garfield criticised the Lancet study.
He offered this quote:
"I'm shocked by the levels they (the investigators) reached," said Garfield. "Common sense, gut level, says it is hard to believe it could be this high. We don't know how many have died, we just know it's a lot. ... Right now, the only other option is to stay in the dark."
Garfield isn't criticising the study. Sven explained it for Blair:
Garfield is saying that 1) you can't depend on your gut for a measurement of this scope and scale, because it truly defies "common sense…
tlambert | October 25, 2006In my comments Iraq Body Count's Josh Dougherty throws a tantrum:
Tim, you're a bald-faced liar ... do you really need to be such a monumental fraud and liar to puff up this Lancet study?
Glenn Reynolds has studiously ignored the actual Iraq Body Count they've compiled. This however has him cheering Dougherty on. Reynolds is a law professor at the University of Tennessee. I wonder if he cheers law students who behave like Dougherty?
Oh the folks at this site, got a spray from Dougherty for their sins in citing the Lancet estimate. They seem to have figured him out:
But dear stranger, I…
tlambert | October 25, 2006Obviously anything Gregg Easterbrook writes about the Lancet study is going to be really stupid, and sure enough, he gives us this:
The latest silly estimate comes from a new study in the British medical journal Lancet, which absurdly estimates that since March 2003 exactly 654,965 Iraqis have died as a consequence of American action. The study uses extremely loose methods of estimation, including attributing about half its total to "unknown causes." The study also commits the logical offense of multiplying a series of estimates, then treating the result as precise. White House officials have…
tlambert | October 24, 2006
Tim Blair, whose reaction to the Lancet study was to reject the entire concept of random sampling offers us this:
Among other Lancet critics: Paul Bolton, a professor of international health at Boston University; Stephen Apfelroth, professor of pathology at the Albert Einstein College of Medicine in New York City; and mortality studies expert Richard Garfield.
Of the three, the only one who is an expert in mortality studies is Richard Garfield. What does he say?
The Lancet study cited two sources for the 5.5 pre-war mortality rate: the 2003 CIA Factbook entry for Iraq and a 2002 profile from…
tlambert | October 24, 2006In May I analysed the press coverage of the Iraq Body Count and found that the IBC numbers were usually misreported as the number of deaths and the IBC maximum was often reported as an upper bound on the number of deaths. I asked:
Why not contact reporters who get it wrong and set them right?
The answer from the IBC folks was that they didn't have the time to do that. What are they spending their time doing? Well, a few days ago 27 of Australia's leading scientists in epidemiology and public health signed a letter about the Lancet study, which said:
Last week, the medical journal The…
tlambert | October 23, 2006Sarah Bosely writes in the Guardian:
The critics argued that the Lancet paper does not indicate that the researchers moved far enough away from the main street. "The further away you get, the further you are from the convoys that roll down the streets and the car bombs and the general violence," said Sean Gourley. "By sampling only cross streets which are more accessible, you get an over-estimation of deaths."
But Prof Burnham said the researchers penetrated much further into residential areas than was clear from the Lancet paper.
The notion "that we avoided back alleys was totally untrue".…
tlambert | October 22, 2006The editors at Slate really don't like epidemiology. Not content with Christopher Hitchens' clueless attack on the Lancet study they've published another attack on the study. And this one is by Fred Kaplan, the man who made such a dreadful hash of it when he tried to criticize the first Lancet study. Kaplan writes:
The [first] study's sample was too small, the data-gathering too slipshod, the range of uncertainty so wide as to render the estimate useless.
So he's learned nothing about statistics since his botched criticism of the first study. Kaplan concedes that the new study has a…
tlambert | October 21, 2006Daniel Davies was on the radio talking about the Lancet study.
Richard Miniter interviews Gilbert Burnham.
Deena Beasley reports what experts in cluster sampling think of the study:
"Over the last 25 years, this sort of methodology has been used more and more often, especially by relief agencies in times of emergency," said Dr. David Rush, a professor and epidemiologist at Tufts University in Boston. ...
Rush, speaking at a meeting in Los Angeles on the medical consequences of the Iraq war, said that the relatively small size of the sample -- 1,849 households -- doesn't change the findings,…
tlambert | October 21, 2006I guess that the next time a new physics study comes out Science will ask epidemiologists what they think of it. You see, John Bohannon, the reporter for Science, decided that opinions from a couple of physicists and an economist were more important than getting comments from experts in epidemiology.
Bohannon report on the Lancet study (subscription required) states:
Neil Johnson and Sean Gourley, physicists at Oxford University in the
U.K. who have been analyzing Iraqi casualty data for a separate study,
also question whether the sample is representative. The paper
indicates that the survey…
tlambert | October 19, 2006The Washington Post has hosted a on-line discussion with Gilbert Burnham. Some snippets:
"One last point that is hard for many people to understand. The number of people or households interviewed and the number of clusters used does NOT depend on the population of the country. At a certain point, taking more samples from more clusters does not increase the validity of the answer--and we calculated those levels before the survey."
"Keeping bias out of sampling is a huge challenge, and we spend much of our time before a survey thinking about this. People living close together are more likely…
tlambert | October 19, 2006I asked Les Roberts to comment on Moore's piece. He wrote:
I read with interest the October 18th editorial by Steven Moore reviewing our study reporting that an estimated 650,000 deaths were associated with the 2003 invasion and occupation of Iraq. I had spoken with Mr. Moore the week before when he said that he was writing something for the Wall Street Journal to put this survey in perspective. I am not surprised that we differed on the current relevance of 10 year-old census data in a country that had experienced a major war and mass exodus. I am not surprised at his rejection of my…
tlambert | October 19, 2006river
The responses were typical- war supporters said the number was nonsense because, of course, who would want to admit that an action they so heartily supported led to the deaths of 600,000 people (even if they were just crazy Iraqis...)? Admitting a number like that would be the equivalent of admitting they had endorsed, say, a tsunami, or an earthquake with a magnitude of 9 on the Richter scale, or the occupation of a developing country by a ruthless superpower... oh wait- that one actually happened. Is the number really that preposterous? Thousands of Iraqis are dying every month- that…
tlambert | October 19, 2006Jim Giles talked to epidemiology experts about the Lancet study. (Nature subscription required):
Data from other conflicts show that such sampling is much more accurate than media reports, which usually account for no more than 20% of deaths. "Random counts force you to go to places that aren't convenient," says Jana Asher, a researcher with the Science and Human Rights Program of the American Association for the Advancement of Science in Washington DC. "The media don't wander off to distant locations. It's a very different type of data collection."
Death tolls from Iraqi health officials,…
tlambert | October 18, 2006It never ceases to amaze me the way the Wall Street Journal combines superb news coverage with a completely clueless editorial page.
To balance an excellent news article by Carl Bialik on the first Lancet study, we have an innumerate article on the editorial page by Steven E. Moore. Moore claims that the sample size for the Lancet study is too small:
However, the key to the validity of cluster sampling is to use enough cluster points. In their 2006 report, "Mortality after the 2003 invasion of Iraq: a cross-sectional sample survey," the Johns Hopkins team says it used 47 cluster points for…
tlambert | October 18, 2006Rebecca Goldin:
While the Lancet numbers are shocking, the study's methodology is not. The scientific community is in agreement over the statistical methods used to collect the data and the validity of the conclusions drawn by the researchers conducting the study. When the prequel to this study appeared two years ago by the same authors (at that time, 100,000 excess deaths were reported), the Chronicle of Higher Education published a long article explaining the support within the scientific community for the methods used. President Bush, however, says he does "not consider it a credible…
tlambert | October 18, 2006If you read the comment threads on the Lancet study you will know that David Kane frequently pops up with dark hints the authors committed some sort of fraud. Well now he has argued that the Lancet study is likely to be a fraud because the response rate was so high. [Update The post has been removed because the "tone is unacceptable, the facts are shoddy, and the ideas are not endorsed by myself, the other authors on the sidebar, or the Harvard IQSS"] Kieran Healy smacks him down:
Kane says, "I can not find a single example of a survey with a 99%+ response rates in a large sample for any…
tlambert | October 17, 2006Anthony Wells:
So, what could have gone wrong? The more excitable fringes of the US blogosphere have come out with some interesting stuff. Let's look at criticisms that don't hold water first.
Firstly, the turnout is unbelievably high. The report suggests that over 98% of people contacted agreed to be interviewed. For anyone involved in market research in this country the figure just sounds stupid. Phone polls here tend to get a response rate of something like 1 in 6. However, the truth is that - incredibly - response rates this high are the norm in Iraq. Earlier this year Johnny Heald of ORB…
tlambert | October 17, 2006Bill S.2125 was unanimously passed by the Senate and promotes relief, security and democracy in the Democratic Republic of the Congo. It states:
(5) A mortality study completed in December 2004 by the International
Rescue Committee found that 31,000 people were dying monthly and
3,800,000 people had died in the previous 6 years because of the
conflict in the Democratic Republic of the Congo and resulting
disintegration of the social service infrastructure and that
"improving and maintaining security and increasing simple, proven and
cost-effective interventions such as basic medical care,…