Daniel Davies has a new post on Lancet denial, with some particularly egregious examples. The worst example is by Harry of Harry's Place whose "discussion" of the study is to make a statement that he must surely know to be false:
Dsquared is a serial bullshitter who has never given a straight answer to any question.
Davies also links to a transcript by Seixon of the Hitchens-Galloway debate, where Seixon touts his own debunking of the Lancet study. Seixon's debunking fails because he makes basic errors in his statistics, but at least they are original, so let's look at where he goes wrong:
Dr. Les Roberts removed 6 provinces from being in the sample. That means that every single household in those 6 provinces was purposefully given a probability of 0% of being chosen for the sample. This violates the principle of randomness, thus violating the principle of statistics that you have a random sample. ... The study's results are based on a biased sample. Resting upon this fact alone, the study's results cannot be claimed to be accurate, nor should they be trusted as accurate. Dr. Les Roberts and anyone else cannot argue this simple point, because then they will have to take on the vast body of statistical literature and theory looming over the credibility of this study.
Unfortunately, Seixon does not understand sampling. The sample was not biased by the exclusion of six randomly chosen provinces since each household in Iraq was equally likely to be chosen by the sampling procedure. Seixon could just as well argue than all surveys are biased because after the sample has been randomly selected each person outside the sample has a 0% chance of being selected.
In response the study's statement that "Most individuals reportedly killed by coalition forces were women and children." Seixon offers this:
46% men, 46% children, 7% women, and 1% elderly. With this in mind, try to finish this sentence: Most individuals reportedly killed by coalition forces were _ and ____. In a world where scientists don't try to hoodwink their readers, the correct answer would be "men" and "children". Yet they chose "women" and "children", even though out of the 4 groups, women were #3, and they put two groups into their sentence. I guess they were hoping no one was going to actually read the rest of the study and find out that they are misleading liars. Seriously, what is the point of misleading in this fashion? Could it be... a political agenda?
So even though the statement is true Seixon insists that it is a somehow a lie. The fact that seems to have escaped Seixon is that Roberts et al grouped women and children together because they are likely to be non-combatants.
I think his confusion is not the confusion you attribute to him. The giveaway is earlier in his post where he says: "The reason is that all of the clusters from the one were transferred to the other, based on one random selection." He is misreading this:
Incidentally the guy's name is evidently George Gooding.
Oh boy, you screwed up, mostly because you took what I said out of context.
Look, the Johns Hopkins team did a real random sample to begin with. At that point, 17 of the 18 provinces had households distributed to them, a true random sample.
Then they decided that they needed to cut down the amount of provinces they were going to travel to, so then they paired up 12 provinces, so that 6 provinces were stripped of their households.
Thus, as I said, the sample is no longer random.
Kevin, you also don't seem to get this. They didn't redistribute all those households randomly individually. They took all the households in a pair, then randomly chose one of the provinces, and gave all the households to that one. Thus, they didnt randomly distribute each household, therefore no longer a random sample.
You can't have a random sample when you purposely remove 6 provinces from your original random sampling.
Tim, you screwed up most here:
"Seixon could just as well argue than all surveys are biased because after the sample has been randomly selected each person outside the sample has a 0% chance of being selected."
Well, obviously you didn't read the study. They did do a random sample to begin with. If they had left it, then I would have nothing to talk about. At that point, all households in Iraq had an equal chance of being selected. THEN, they started tampering with their random sample, and removed households from 6 provinces and put them in 6 others.
Try not taking things out of context, and try not misrepresenting what others are saying.
I understand statistics quite well, and the fact that no statisticians have been asked to comment on this, beside the one I quoted in my post who happens to agree with me, is quite a telling sign.
I'm sorry, but this Lancet study is all washed up, and the only ones in denial are the ones who still believe it is worth the time it takes to read it.
And Kevin, what in the world was the purpose of printing my name? Trying to scare me into silence for my dissent?
How did I misread that passage? I misread it because my name is so and so? Haha. Brilliant.
I could be wrong, but I imagine that reliable numbers are hard to get on this and that any study is bound to be imperfect. I would not be surprised if the study was fairly accurate. However, I am curious to know how "children" was defined. "Reportedly" is a pretty slippery word too.
Please tell me how, regardless of the methodology of the Lancet study, you can do the following:
Discount a study that has 20 times the sample of the Lancet one, and 26% more coverage?
Innumerates? Give me a break Tim, you are grasping for straws. Don't lump me in with all those who took Kaplan's word for it, I have already stated that his "debunking" was severely lacking. Anyone who believes his article blindly are innumerates. I have proposed something entirely different, and it is undeniable. You can't hide from it, so you try to steamroll right past everything.
I see children were defined as under 15, that seems reasonable.
Seixon,
A sample of households is random if, at the outset, each household has an equal probability of being selected. If my raffle ticket is blue and a red ticket is drawn from the hat, it is immediately apparent that my probability of winning has fallen to zero. That does not mean the draw was unfair. Notice that the probability distribution changes while the process of drawing the winning ticket is still going on. We started with all tickets having an equal chance of being the winning ticket. Now all red tickets have an equal probability and all other have probability zero. Finally the distribution collapses to a single ticket with probability one. Nothing is changed by the fact that the probability distribution alters during the selection process. It is nonetheless a random draw. Likewise the Lancet team's approach.
Apologies for dragging your name into it; I should have thought of the possibility that you would prefer commenters to stick to using your pseudonym. Since Tim Lambert has dealt with numerous Lancet critics, it seemed likely that your paths had crossed before you started your blog. No, rest assured that I don't expect you to be intimidated.
Kevin,
Again, you are misreading me.
The Lancet team did a random sampling to start off. Now, if they had simply left it at that, I would have no qualms about that aspect of the methodology. As you said, from the original sampling they did, all households had an equal chance.
The problem started when they decided to cut 6 provinces out of the sample entirely. They did this because they wanted to cut down the amount of traveling they would have to do, supposedly for security reasons (when in fact, they went to all the most dangerous areas of Iraq, purposefully going to Fallujah, for example, which later was excluded as an outlier).
At this point, they would remove all the clusters from 6 of the provinces, and place them in 6 other provinces.
This meant that Basra's 2 clusters were moved to Missan, based on one single drawing.
If they were to be truly randomly selected, keeping with the equal chance of being selected, they would have to do one drawing per cluster. They only did one drawing per grouping of clusters.
Thus, the entire sample became biased and 6 provinces were purposefully removed from the sample.
To put it more simply: there was 0% probability of households being in both Basra and Missan.
Another thing:
I said it was misleading the way they said "women and children" instead of "men and children".
Any objective reading of the statement gives the impression that women and children were most affected. The truth, that men were the most affected of all, gets completely left out.
Thus, the statement is misleading, not a lie, as I said. You know, if you actually read what I wrote instead of shooting off misrepresentations of my work.
Another example, cross-posted from CrookedTimber:
Let me do another example, Lancet essentially did the following
In a poll of 1,000 Americans, 67 from Texas are chosen to be interviewed via a random sampling of the nation. 114 are chosen in California, and so on.
Now Lancet decides that it wants to cut down on the number of states it wants to call. So they pair up California and Texas and decide that one of them will receive all the people chosen in the other.
A number between 0 and 53,000,000 is chosen at random (the combined populations of Texas and California). If the number is between 0 and 20,000,000 then Texas gets them. If it is between 20,000,000 and 53,000,000 then California gets them.
The number picked turns out to be 45,567. Texas gets all of California's 114 selected persons.
Now Lancet will call 181 people in Texas, and 0 in California.
You have got to be smoking some serious reefer if you think this results in a randomized sample, and a representative result.
This is exactly what Lancet did in Iraq. Try to weasel your way out of that one, sirs. I'm not in denial, I know what I'm talking about, and will continue to hand you sirs your arses as long as it is necessary.
Seixon,
Before the random numbers were generated, a household in Dehuk, Arbil, Tamin, Najaf, Qadisiyah, or Basrah had the same probability of selection as a household elsewhere in Iraq. That no such household got selected was due to chance; they were not discriminated against. The key statement here is: "the sample remained a random national sample." The fact that those Governorates were already out of the running when it came to choosing individual communities does not invalidate it. The mere fact that cluster assignment took place in two phases does not introduce bias.
I know it will take a while to convince you of this. I won't have much time over the next few days, so please don't interpret silence as assent to your position.
I would hazard that I know less statistical theory than the other people commenting here, but I can't see the problem that Sexion thinks is there. If I wanted to do a study on something in the United States, and I randomly chose either the twenty-five states alphabetically from Alabama to Missouri or the twenty-five from Montana to Wyoming, and then I do whatever else I'm planning to do to only one of those two lists, how does this invalidate my results? Of course it's possible that the twenty-five I didn't pick will be systematically different, but there's no reason to assume they will be.
I'm only going to make a meta-argument. The fact that Seixon thinks no statisticians commented on this is bizarre. Am I misunderstanding him? The Lancet is peer-reviewed and the validity of their statistical methodology would obviously be the focus of any reviewer's criticism. I also recall reading an article in the Chronicle of Higher Education about this article and I think the experts in statistics all agreed the methodology was statistically sound.
One could criticize the paper's conclusions or say that based on other evidence one thinks the true death toll falls somewhere in the lower part of their estimated range, but these technical objections seem unlikely to be important. I'm guessing the real experts would have identified these flaws right from the start. In my untutored opinion the biggest problem with the Lancet study and the UN study is precisely that they didn't focus on the worst-hit areas like Fallujah during the time period when the US was softening it up for the final assault. This, of course, is no one's fault. But in hindsight I wish they'd sampled several neighborhoods in Fallujah so we'd know whether the loss of life was as massive as suggested by the one neighborhood they did visit. The UN survey is no substitute--it didn't cover the period in question. Most likely we'll never know, unless someone does a study many years from now when it won't have any relevance.
Kevin,
You continue to obfuscate. Have you even read the study? It sure as hell doesn't seem like, and Mr. Lambert is sure very silent on the matter, even though I sent him an e-mail.
When you said:
"Before the random numbers were generated, a household in Dehuk, Arbil, Tamin, Najaf, Qadisiyah, or Basrah had the same probability of selection as a household elsewhere in Iraq. That no such household got selected was due to chance; they were not discriminated against."
Actually, if you read the study, clusters were selected in all six of those when they did a random sample. So "that no such household got selected was due to chance" is a gargantuan whopper. They DID get selected. Can you go read the study please?
I guess I need to quote from the study:
"Clusters initially assigned at random:
Baghdad: 7
Ninawa: 3
Dehuk: 1
Sulaymaniya: 2
Arbil: 1
Tamin: 1
Salah al Din: 2
Diala: 2
Anbar: 1
Babil: 3
Karbala: 1
Najaf: 2
Wasit: 1
Qadisiyah: 1
Dhi Qar: 2
Muthanna: 0
Basrah: 2
Missan: 1"
See? Can you read? Or does Lambert's immense tie to the credibility of this study keep you from firing up Adobe Acrobat to read the PDF?
Now, if they had left it like that, THAT would have been a random sample.
But NO! A random sample wouldn't do, because they didn't want to travel to all those provinces, for "security" reasons (which I am sure is also the reason they felt it was so necessary to venture into Fallujah).
Next came the "grouping process".
They chose 12 provinces, on a whim, and decided that, hey, 6 of these aren't going to have clusters any more, because we want to cut down our travel time. Why not all 18 provinces? Why not 10? Why not 8? Ah, that is left up to the God of Arbitrary, of course.
Now after this "grouping process", suddenly Basrah, Dehuk, Arbil, Tamin, Najaf, and Qadisiyah had 0 clusters. They originally had clusters assigned to them due to the random sample. After their tampering, they had none.
Of course they said "the sample remained a random national sample". If they didn't, then people like you would be like deer in headlights wondering what you were going to tell me when I challenged it.
To make this even more clear: let's say we continued with this "grouping process" that they did. Eventually, I could have all the clusters end up in Baghdad province.
I could just keep arbitrarily pairing up provinces, doing one random drawing, and give all the clusters assigned to each of the pairs to one of the provinces.
I would eventually end up having Baghdad with all 33 clusters, since Baghdad has the highest population, thus the highest probablity of winning this flip-of-a-coin-winner-takes-all nonsense.
I don't even know how to make this anymore clear to you guys. I'm starting to get the feeling that you know I have a point, but don't want to admit it.
washerdreyer,
You think you would get a representative opinion of Americans by only sampling 25 of the 50 states? Oh man... Take a statistics course, please.
Johnson,
No statisticians have commented on this "grouping process" that is in the study. Or, not that have appeared in the media, anyways. I have this from CNN:
"But Richard Peto, who is professor of medical statistics at Oxford University, cautioned AP the researchers may have zoned in on hotspots that might not be representative of the death toll across Iraq."
http://www.cnn.com/2004/WORLD/meast/10/29/iraq.deaths/
That is precisely what the result of the "grouping process" was. Almost all the safest areas of Iraq were ripped out of the random sample they already conducted, and the clusters were given to more dangerous areas.
Me thinks that none of you actually read my entire analysis. Mr. Lambert pretends in his response to me that there were not two phases in the distribution of clusters, and so does Kevin.
There was 0% probability that both Basra and Missan would end up with a cluster. There was 0% probability that Ninawa and Dehuk both ended up with a cluster. There was 0% probability that Sulaymaniyah and Arbil both ended up with a cluster. There was 0% probability that Tamin and Salah al Dinh both had a cluster. There was 0% probability that Karbala and Najaf both had a cluster. There was 0% probability that both Qadisiyah and Dhi Qar had a cluster.
Come into the light, stop being in denial.
Wonderful, the reply I get from Lambert?
"You don't understand sampling."
In my book, this is called stone-walling. Come on Mr. Lambert, show me the error of my ways, and quit misrepresenting my argument like you did in your original post.
I've got Kevin saying, "but, but, the study says so, so it must be true!"
Can I get more than a "you don't understand anything" reply? Usually when you rebut someone, you tell them HOW and WHY they are wrong, not just that they are wrong.
Absent this, as I said earlier, I cannot conclude anything other than that you know I have sunk your battleship and you are lost on how to spin yourself off the deck to save your credibility.
A gentle suggestion for seixon:
"It is better to remain silent and be thought a fool than to speak and remove all doubt."
Attributed to Abraham Lincoln among others
Actually, this is what I emailed Seixon
>You don't understand sampling and your email and comments you left
demonstrate this. If each household is equally likely to be chosen
then the sample is not biased by definition. You seem to have
accepted clustered sampling as valid -- this is just clustering at the
governate level.
I did tell him how and why he was wrong, he just seems to have ignored that part. There certainly wasn't a period after the word "sampling" in my email.
Mark,
Why don't you put your money where your mouth is? Am I going to actually get any rebuttals here, or are you are all going to HISS me out of here for daring to challenge your "obvious" superior intellect?
Enough with the ad hominem and stone-walling guys. Let's see you show how I am wrong instead of just sounding like a broken record.
Tim:
A number between 0 and 53,000,000 is chosen at random (the combined populations of Texas and California). If the number is between 0 and 20,000,000 then Texas gets them. If it is between 20,000,000 and 53,000,000 then California gets them.
The number picked turns out to be 45,567. Texas gets all of California's 114 selected persons.
"Now Lancet will call 181 people in Texas, and 0 in California.
You have got to be smoking some serious reefer if you think this results in a randomized sample, and a representative result.
This is exactly what Lancet did in Iraq."
Would you agree that if a poll was taken in this context the result would be as credible as asking New Yorkers who they are to vote for at the next election and extend that out for the entire US population? It's absurd right?
So if you agree it is absurd then you would have say the Lancet study does contain a few little problems.
One big gaping issue I have with your prized study is that in a previous post it was mentioned the reseachers conducted a household wide survey of all the inhabitants.I would be real interested in finding out how they eliminated double counting.
I think the best "critique" of all time was one (perhaps apocryphal) from the late 19th century which said of some statistical study "the results don't represent the entire population, but only a sample, and a random sample at that!"
100+ years later this is still the essence of most of the objections to the Lancet study including Seixon's, and the innumerable innumerates who object to extrapolating a sample proportion to an entire population because "it just doesn't seem right".
Joe and Seixon; I think your example about Texas and California would be more analogous if you were talking about grouping Wyoming and Montana, which is something that I would bet serious money at short odds a lot of pollsters do. Or North and South Dakota. Or grouping Oklahoma with Texas.
Secondly, it's just not true that clusters were systematically moved out of safe areas into dangerous areas; most obviously the city of Najaf was not sampled despite having been the scene of heavy fighting.
"The sample was not biased by the exclusion of six randomly chosen provinces since each household in Iraq was equally likely to be chosen by the sampling procedure. Seixon could just as well argue than all surveys are biased because after the sample has been randomly selected each person outside the sample has a 0% chance of being selected."
I'd like to weigh in on this. I think Seixon has a point as regards his criticism of the sampling, though I don't think he explains it clearly. There are biases, though it is debatable how profound their effects are.
Seixon thinks the sample was biased. Tim think it was not - on the basis that household in Iraq was equally likely to be chosen by the sampling procedure. But eqiprobability of each unit being chosen is not the only requirement for generating a random sample - independence is also a prerequisite.
Let me give an example. I decide to generate a sample by selecting a random starting point on a circle and then (a) toss a coin to see which direction to move in, (b) repeatedly throw a dice to see how many paces to move in, and (c) sampling at each point I stop at.
Each point on the circle has eqiprobability of being sampled - this is Tim's point. But I haven't made a random sample because the choice of points are not independent of each other. They are dependent, and any interdependence between the values at sampled points will bias the result. "Clumping" clusters together will have the same effect for the same reasons.
I would also like to make a point of my own. It is simply untrue that each household in Iraq was equally likely to be chosen by the sampling procedure. Locating each sample within a governorate was performed by: (a) drawing a map, and (b) generating a random coordinatate on that map.
This finds a random point in space: but does not find a random household. The method will oversample households in areas of low population density and undersample them in areas of high population density. Any link between mortality and population density will bias the result. Because there are marked differences in population density the sampling will be seriously skewed.
As to how serious these errors are in effecting the result - there's no way I can quantify this. But bias exists and we have to be careful in interpreting the result. I don't think this means the Iraq study is worthless, it clearly was and remains a substantial contribution to knowledge as the only study of excess mortality resulting from the Iraq War.
I do think some of you have assumed that because most criticisms of the study are innumerate, then all of them are.
"The researches of many commentators have already thrown much darkness on this subject, and it is probable that if they continue we shall soon know nothing at all about it" - Mark Twain.
It's probably an unreasonable expectation of the blogosphere but I'd welcome a bit more rising above the ad hominem from both sides here. I am a lazy bear of very little brain and am having a hard enough time counting on my fingers and toes so I can do without the finger-pointing, tu-quoques and outrage.
I have to say I don't find Seixon's criticisms of The Lancet study particularly persuasive. I don't think their methodology is unreasonable nor do I find their presentation of the data misleading or dishonest. The study is filled with caveats and seems to go out of its way to say that the data sample is limited, may be unrepresentative and calls for further verification.
In the absence of a larger and less extrapolated survey, this doesn't seem unreasonable. I'd certainly like to hear more about "the study that has 20 times the sample of the Lancet one, and 26% more coverage".
I also want to hear more about how this relates to the survey "Combat Duty In Iraq And Afghanistan" http://content.nejm.org/cgi/content/full/351/1/13 in the New England Journal of Medicine 1 July 2004, particularly Table 2 concerning responsibility for the death of noncombatants: http://content.nejm.org/cgi/content/full/351/1/13/T2
In this interview http://www.socialistworker.co.uk/article.php4?article_id=6271 Dr Les Roberts claims the NEJM results back up The Lancet study. Is this fair?
Wait, the "randomly" excluded provinces just happened to be in Kurdish and Shi'ite controlled areas where the infrastructure doesn't get bombed as much and acts of terrorism are few and far between?
[Wait, the "randomly" excluded provinces just happened to be in Kurdish and Shi'ite controlled areas where the infrastructure doesn't get bombed as much and acts of terrorism are few and far between?]
Jet, one of those "Shi'ite controlled" areas was Najaf!
There are certainly methodological criticisms one could make of the Lancet study if one was in a statistics classroom and didn't have to a) explain how the problem could have been corrected and b) revisit one's suggestions for a) after being reminded that there was a bloody war on. In principle, if there was a very high correlation between death rates and population density, the results might be skewed.
However, if anyone wants to seriously claim that this invalidates the study to such a degree that it should not have been published or should not be quoted, then I would be grateful if they could provide a single (real-world) example of a piece of empirical work of any kind ever having given seriously misleading results because of a problem of this kind and explain why they think that the Lancet study was analogous. I really think that blackboard critiques of this sort are reaching.
dsquared,
The violence in Najaf lasted around 3 months and was limited to contained portions of the city and did not result in major infrastructure damage which is what the Lancet study said caused most (2/3?) of the deaths.
And no one is saying the study is invalid. Only that serious problems exist with it that are ignored or shouted down. The biggest problem I'd like to see addressed is the obvious bias in the study. The points Seixon brings are do support the claim of extreme bias in the survey reports's wording, ie 100,000 civilians (didn't they say 1/3 of those were combat deaths?), even though women only accounted for 7% of deaths, grouping them in the category "women and children" so that that phrase can be used with the higher percentage of deaths of children giving the misleading impression that the women were dieing at the higher rates also, etc.
Nikolai, yes the pairing of the governates means that the samples were not independent, but that does not bias the result. It is just cluster sampling again at the level of governates and all that does is reduce the effective sample size.
The sample procedure does bias the sample towards households in relatively less densely populated and near edges of towns. Only if there are large density differences within a town *and* large differences in mortality in dense and less dense areas will that make a significant difference. (And it seem just as likely to bias things up as down.)
I have not assumed that because most of the criticisms are innumerate they all are. I have described Sexion's criticism as innumerate because it is.
*"I have describes Sexion's critism as innumerate because it is."*
I recall the old adage here that about half of statisticians graduated in the bottom 50% of their class. *Jeez*, and I thought my spelling was shocking, despite a mild case of lysdexia....
Interesting to see that Davies has also taken what I guess we should now call the party line on the '24,000' figure.
I think we can pretty much call the whole debate dead now. No new evidence will alter anyone's view. It is deceased; 'tis an ex-parrot; 'tis ceased to be......
I may be satisfied that the sampling procedure itself did not necessarily introduce a mathematical bias, but I believe Seixon may have a point that the pairing process could have introduced some unfortunate subjective aspects.
Looking at Seixon's map, I can certainly accept that a method that ended up excluding all of the northernmost and southernmost provinces of the country, including most of the Kurdish region and the until-recently significantly quieter area of the country patrolled by the British division, could well still have been a randomly derived result, but given what we know about Iraq's ethnographics and the recent patterns of violence it was at best an unfortunate one for a study that is asserting a national character. And when a study ends up including all 6 out of the 6 most violent provinces, and only 2 out of the 6 least violent (in terms of U.S. casualties, I grant), and bases that choice on an apparently undocumented assessment of relative levels of violence by the survey team, that would also seem to be an unfortunate aspect of the method.
The obvious example would be if the pairing process had somehow ended up excluding Baghdad (which would not have happened, because the survey team apparently concluded that there was no comparable governorate to pair Baghdad with, so it was not subject to randomized elimination). Of course, if Baghdad had been excluded, this would have led to justifiable criticism that the study was not inclusive of the country's major metropolitan area. But by the same token, a pairing method that guaranteed Baghdad's inclusion in this way, but introduced an extra chance for exclusion of anyone living in more "pairable" provinces, could well introduce a survey bias, regardless of the mathematical soundness of the pairing technique itself.
This is not to say that the actual number of casualties does not still likely fall within the Lancet study's probability range, but I believe you could make the case that the aspect of subjectivity introduced by the pairing process would justify the assumption that the true number was somewhere below the stated 100,000 midpoint.
"I have not assumed that because most of the criticisms are innumerate they all are. I have describes Sexion's critism as innumerate because it is."
OK. Well this innumerate has decided that the easiest way to break the stone-walling is to cut down my argument to a swift morsel of information. Here it is:
In their "grouping process", the Lancet used one random event to distribute multiple clusters. This violates the principles of random distribution, thus no longer a random sample. For example, for Ninawa and Dehuk, a total of 4 clusters were distributed according to a single random event. For the sample to remain random, one would have to distribute the clusters at one random event per cluster.
If you deny that this is correct, then you might as well distribute all 33 clusters with a single random event, which would most likely select Baghdad, and thus Baghdad would end up with 33 clusters, the rest of the country none. This is exactly what they did in their "grouping process", only to a larger degree.
If you keep repeating their "grouping process", you will most likely end up with all the clusters in Baghdad also.
Mr. Lambert, if you cannot respond to this, everyone can see that you are an emperor without clothes and you will have disgraced your name as being knowledgable about statistics.
Tim;
"the pairing of the governates means that the samples were not independent, but that does not bias the result. It is just cluster sampling again at the level of governates and all that does is reduce the effective sample size."
I admit I don't completely follow this, perhaps you could elaborate?
By my logic, dependent samples don't systematically bias the estimate in a particular direction, so much as destroy the basis of the "marbles-in-a-jar" logic needed for statistical inference. If clusters within governorates are more similar to, or more different from, each other than clusters in other governorates then estimates and confidence intervals are all thrown off by an unknown and unknowable amount.
Re: Density and Mortality.
To paraphrase: the point has been made that you need (a) density differences, (b) mortality difference between dense and less dense areas to (c) influence the estimate. There's been an implication that this isn't likely.
I just want to point out that it's been well established for a long time in geography that there are large differences in density within towns, and in public health that people in densely populated areas (i.e. the poor) have considerably higher death rates.
Dsquared says that "in principle [given] high correlation between death rates and population density, the results might be skewed."
It actually goes a bit beyond this. There can be no correlation, but if there is higher variability in death rates between high and low density areas (i.e. densely populated areas are more likely to see multiple people wiped out by a bomb) this will throw the basis of the inference off.
It isn't just the exact estimate I'm worried about, but how certain we can be of where it is. The reason for the paper was to make an inference of excess mortality was and say roughly how confident we are of where it is. Sampling problems destroy the basis for this.
"Most individuals reportedly killed by coalition forces were women and children."
Actually, the sentence is pretty badly worded, since interpreted literally with the usual logical operators, it would be saying that most of the individuals killed were immature females. It would have been clearer, had they said women ***or*** children.
Nikolai, clustering, whether it is at the individual level or the governate level, does not bias the estimate. What it does is increase the variance of the estimate and hence reduce the precision. As is explained in the article:
>This clumping of clusters was likely to increase the sum of the variance between
mortality estimates of clusters and thus reduce the precision of the national mortality estimate.
How likely do you think it is that Seixon has discovered a monstrous blunder in the statistics that was missed by the authors and the referees and all the expert statisticians who have looked at the study?
"You can't have a random sample when you purposely remove 6 provinces from your original random sampling."
Well, sure you can, unless your second selection is dependent on some (nonrandom) variable.
It reminds me of the observation:
"Life is unfair. But it's unfair for everybody; that makes it fair."
Gee, Mr. Lambert, I don't know, how likely do you think it was that blogger Brendan Loy was the first person to predict the destruction of New Orleans?
I read you because you have always addressed these kinds of debates on their merits. Arguments from authority (in this case, someone else's authority) are generally beneath you, and should stay that way.
Nine-tenths of the "refutations" of the Lancet study were utter crap, I concede. But before Seixon's post, I'd never seen the provinces chosen on a map, or his cross-reference of the provinces with the heaviest U.S. casualties vs the Lancet cluster provinces. These were both valuable contributions, regardless of whether the rest of Seixon's argument holds up.
Z, I believe this is the portion of the Seixon argument that holds up... that the selection of the 12 provinces from the initial 18 may indeed have been non-random. The study itself never asserts that it was random:
"To lessen risks to investigators, we sought to minimise travel distances and the number of Governorates to visit, while still sampling from all regions of the country. We did this by clumping pairs of
Governorates. Pairs were adjacent Governorates that the Iraqi study team members believed to have had similar
levels of violence and economic status during the preceding 3 years."
18 provinces were chosen. 6 were not paired, and thus included, whereas 12 were paired, and half of those left out. There is no indication from the above that the choice of whether a province ended up in the 6 or the 12 was randomized... indeed it seems to have been based on undocumented "belief" that there was a comparable adjacent province to pair with. From the above, Baghdad and Anbar clusters may have had no chance of being excluded, while Basra and all of the Kurdish provinces did. That seems an unfortunate choice of method, in retrospect.
Correction to my last. The first sentence, third para should read, "Clusters were chosen in each of Iraq's 18 provinces. Of those, 6 provinces were unpaired, and their clusters automatically included, and 12 provinces were paired with a similar neighbour province, and half of those substituted out."
Sorry, should have used preview.
In fact, reading Fred Kaplan's original critique of the study, he brushes against what I am saying here, although he does not explain it adequately. I quote:
Kaplan says it is unclear how they made this calculation. Actually, it isn't, and he should have read the study more thoroughly because it spells out how this was done.
Lambert, what you are saying ignores the fact that the clusters in 12 provinces were distributed via 6 random events, instead of one random event per cluster. This had the result of ensuring that around 25% of Iraq would be excluded from the get-go. This is how it went awry, and produced a sample bias.
Also, the last quote in my piece demonstrates that statisticians have talked about this, yet predictably they aren't getting much press coverage for their dissent. I demonstrate:
Now, here Peto isn't given any room to explain what he means by this, but he is talking about exactly the thing I have proven for you all right here, and in my piece back in May.
I think it is clear from one of Seixon's comments at his own blog that he has simply screwed up:
Unless I am much mistaken this is just a blunder. Let B and M represent Basrah's and Missan's percentages of Iraq's population. The expected number of cluster for each (1st round) is 33B and 33M, respectively. Basrah's expected number after the second round is:
[B/(B+M)][33(B+M)] = 33B.
Tim;
I think I perhaps wrongly used "bias" in a loose fashion. The estimate isn't biased in the sense of being systematically pushed up or down. But a dependent sample does damage the assumptions needed for statistical inference, and means we have less certainty over confidence intervals and the location of the estimate.
I'm afraid I'm still not sure what you mean by clustering "at the individual level or the governate level". The technique estimates mortality rates from clusters of individuals (in households); I'm not sure what clustering at the governate level is. If it means non-random allocation of clusters between governates then this could affect the result.
"How likely do you think it is that Seixon has discovered a monstrous blunder in the statistics that was missed by the authors and the referees and all the expert statisticians who have looked at the study?"
This is an argument from authority. Seixon flagged a glitch in the sampling procedure based on an intuitive understanding of randomness. He didn't express it particularly well, and I'm sure a lot of what he says outside this is wrong. But there have been some very clever people on this thread and others insisting that if all units are equally likely to be chosen then something is a random sample - and this just isn't true.
I'd bet that this wasn't missed by the authors and the referees and all the expert statisticians, but they appreciated the paper was important in spite of this, and that the sentence you quote was put in to try and at least reference the problem.
Ah.... going back and rereading the paper, I see what he's getting at (maybe?). The second selection was not strictly random, since it depends on the assignment into pairs, which was done on the judgement "that the Iraqi study team members believed to have had similar levels of violence and economic status during the preceding 3 years", and some of them were not paired. Thus, the unpaired Governates, and whatever (presumably somewhat of an outlier since they're not paired) levels of violence and economic status they represent as well as any other variables, get a free pass through the second selection.
Of course, this adds the complication of the accuracy of the matching by violence and economic status. Any systematic bias there would show up in the end result. This leaves us in the usual position of showing that there was an opportunity for bias to leak in, without demonstrating whether there was or was not any bias.
Aside from that, if we assume the matching was accurately done, how would this affect the results? Off hand I don't see an effect... it seems to me that, by definition, any dependency on violence and economic status would show no difference; similarly, any dependency on variables totally nondependent on violence and economic status would be affected quasi-randomly, which is not a bias. But I haven't quite proved to my self beyond any lingering question that this partial second selection, correctly done wouldn't result in bias.
Anyway, that's just my worthless opinion, which I hope somebody who knows what's what will explain to me.
Z, again with the caveat I don't accept the Greater Seixon argument, involving intentional duplicity, etc., there's still something to the Lesser Seixon argument of bias introduced through the selection of the 12 surveyed governorates.
The cross-referencing of violence levels as experienced by American and allied troops with the provinces chosen by the Lancet process is indicative (not conclusive by any means, though) that the matching based on relative violence levels was not, in fact, accurately done (and the Lancet authors seem to concede it was only based on a "belief".)
In practice, what they ended up with via this method was a survey of Iraq conducted entirely within the central 12 provinces of the country, and wholly excluding its northern and southern extremities. It's certainly arguable that any significant north-south variations in violence level in the country, assuming they were inaccurately controlled for, could have introduced some measure of error here. One can say that without accepting the larger argument that this was malicious.
Kevin,
Yes, I did make a mathematical error in that comment on my blog. It was late and I thought about it after going to bed and realized that I had goofed up a little bit.
I am now writing a new blog post that will make this very, very clear to all the hold-outs.
nikolai and z,
You are both overlooking that the second phase of sampling was carried out in a winner-takes-all one-off fashion. In order for randomness to be preserved, albeit with a higher resultant variance, each cluster would have to be assigned via its own random drawing. This was not done in the 2nd round, as the province with the highest population was most likely to win the one-off, and thus win all of the pair's clusters.
In statistical jargon, the expected mean or expected result of clusters for each province in the 2nd round would be either the total number of clusters between the pair, or 0.
For example, in the case of Basrah and Missan, their expected number of clusters would be (populationGovernate/populationTotal)*33 in the 1st round.
In the 2nd round, Basrah would have an expected number of clusters equal to:
(population(Missan+Basrah)/populationTotal)*33
This is because Basrah would be expected to win the one-off between it and Missan, and take all of their clusters. Missan's expected number of clusters would be 0, since there is only ONE random drawing for their 3 clusters, and Basrah would be expected to win it all.
Hope that clears it up... I'm still working on my blog post...
Missan's expected number of clusters would be 0, since there is only ONE random drawing for their 3 clusters, and Basrah would be expected to win it all.
This is wrong. The term "expected" has a particular meaning in statistics. In the notation I used earlier (comment no. 39) the expected number of Missan clusters before the first round takes place is:
[M/(B+M)][33(B+M)] = 33M
Now, once Basrah and Missan had their first-round clusters assigned to them, Basrah had 2 and Missan had 1. The expected number of Missan clusters now became:
[M/(B+M)][2+1] = 3M/(B+M).
This is roughly equal to one since Basrah has roughly twice Missan's population. In the event, against the odds, Missan got the three clusters.
It's an interesting problem. Assume for the moment that Coalition fatality figures are in fact, correlative with violence levels.
In that case, of the six provinces that were not paired, accounting for 14 of the 33 clusters, five were among the top seven provinces in terms of anti-U.S. violence (the sixth, Muthanna, is relatively peaceful, but did not have enough population to merit a cluster). One of those 14, the Anbar province cluster, would be ultimately excluded.
The remaining 19 clusters were in the paired provinces. The net effect of the choice in pairing here seems to have been relatively small, though... three clusters (in Qadisiyah, Tamim, and Dehuk) were moved to a significantly more violent province but two (those in Basrah) were moved to a significantly less violent one, while the other moves did not make a difference on the violence gradient, for a net non-violent > violent cluster-move of only 1 due solely to the effects of pairing.
However, the cumulative effect of not pairing the violent provinces (particularly Baghdad and Babil) while pairing the less violent ones, may still have been significant. Splitting up the provinces into thirds based on the anti-Coalition violence stat, and discounting Anbar/Fallujah, only six of the 32 clusters ended up being located in Iraq's six most peaceable provinces, while 18 clusters were located in the six most violent provinces.
Comment 43 has nicely demonstrated Comment 16.
Must go wash brain, it's been contaminated!
[New post at my blog](http://www.seixon.com/blog/archives/2005/09/slaying_the_kni.html).
Damn thing. HERE.
Oh, and Kevin:
You are again completely ignoring the fact that the clusters were being distributed by ONE random drawing, instead of ONE PER CLUSTER.
Your math assumes that there will be three separate random drawings, one for each cluster, and this is not the case when it comes to the Lancet study.
With Lancet it is:
Basrah: 66% chance of getting all 3, 33% chance of having 0
Missan: 33% chance of getting all 3, 66% chance of having 0
In a random distribution, it would be:
1st drawing:
Basrah: 66% chance of getting one
Missan: 33% chance of getting one
2nd drawing:
Basrah: .4356 chance of having 2; .21 chance of having 1; .1089 chance of having 0
Missan: .1089 chance of having 2; .21 chance of having 1; .4356 chance of having 0
3rd drawing:
Basrah: .287 chance of having 3; .144 chance of having 2; .072 chance of having 1; .036 chance of having 0
Missan: opposite of Basrah
Now let's compare the end results here:
Lancet:
Basrah - 66% chance of having all 3
Missan - 33% chance of having all 3
Random distribution:
Basrah - 28.7% chance of having all 3
Missan - 3.6% chance of having all 3
Are you telling me that those two are equal? Have a nice day in Denial Town.
So think of all those people claiming it was extremely likely that 100,000 "civilians" had been killed in Iraq citing the Lancet study and then think of Seixon's analysis showing why the spread was 100,000 +- 100,000. Looks like the right isn't the only side willing to whore and twist the truth to further their cause. Seixon wins this round.
Seixon's argument also gets a ton of ethos points since he's the underdog being attacked and called names for daring to bring this point up.
My analysis doesn't show that it is 100,000 +/- 100,000. My analysis shows that the finding of the Lancet study is completely meaningless because of the sample bias they introduced by violating simple principles of random sample distribution.
I realize I wasn't able to formulate myself clearly to begin with, but I hope that is now cleared up for all to see that I am, in fact, not an innumerate.
If anything, Mr. Lambert has (perhaps purposely) poor attention to details. Did Mr. Lambert always know about this flaw and hope that no one would find it? Did Mr. Lambert try to bluster me into conceding because he knew I was right? I sure hope not, that would be intellectual dishonesty of the worst kind.
Your math assumes that there will be three separate random drawings, one for each cluster....
Seixon, if I was assuming that I would have said so. Good night.
Well, because it seemed intriguing, I went back and checked Seixon's math on coalition fatalities in the surveyed areas, and I'm afraid the results weren't too impressive.
Seixon (he can correct me if I'm wrong, but I don't see how he got the numbers he did otherwise) evidently counted all fatalities, both combat and non-combat, and used a date range that extended into mid-2005. For comparison, I took only the set of 684 Coalition combat fatalities prior to Sept. 30, 2004, for which a definitive place of death could be established, which seemed to be the more useful comparator.
Across all of Iraq, with 24.4 million people, the rate of Coalition fatalities per million Iraqis was 28. Across the ten provinces where the Lancet study did its fieldwork (less the six excluded by pairing, Muthanna due to low population, and Anbar due to the Fallujah outlier), there were 459 Coalition fatalities in the period leading up the survey, while occupying a population of 17.03 million Iraqis... a rate per million of 27.
In other words, Coalition military activity in the areas where the Lancet team worked, in the first 18 months of war and occupation, was actually very slightly SAFER than Iraq as a whole... apparently discrediting the statistical backing for Seixon's "most violent provinces" argument.
The most that one can say on the available data is the non-random choice of which provinces to pair/not pair had the potential to introduce a bias into the Lancet study, but the one piece of evidence that it actually did doesn't appear to stand up to scrutiny, I'm afraid.
Perhaps that was too cryptic. Seixon, please consider the following case. An "honest" six-sided die is rolled, once and once only. The number of spots which can appear on the uppermost face is one of the following: 1, 2, 3, 4, 5, 6. Each has probability 1/6.
The number of spots which appears is a random variable. Call it X. What is the expected value of X, also known as E(X)?
Once again, good night.
Kevin,
You forgot that you have x number of clusters to distribute, not just one roll of the dice. You don't distribute ALL of them via the same roll of the dice. That isn't randomized distribution. If that was the case, then you could roll the dice on all 33 clusters at the same time, and Baghdad would win it and take em all. Would that be a randomized distribution of Iraq? Of course not.
BruceR,
It is quite possible that I didn't weed out non-combat deaths, but I still don't see how you are getting the results you are. As I said on my blog, it's a while since I compiled those numbers, and I should probably go through it again.
In any case, that doesn't discount the very fact that the "grouping process" led to a non-random distribution of the clusters. This by itself renders a bias into the sample, and thus undermines any number that pops out of the survey of those households.
John Quiggin Says:
September 22nd, 2005 at 7:53 pm
I think the best "critique" of all time was one (perhaps apocryphal) from the late 19th century which said of some statistical study "the results don't represent the entire population, but only a sample, and a random sample at that!"
100+ years later this is still the essence of most of the objections to the Lancet study including Seixon's, and the innumerable innumerates who object to extrapolating a sample proportion to an entire population because "it just doesn't seem right".
Would Quiggin mind explaining in his own words why he thinks Seixon is innumnerate. I don't mean cutting and pasting like he does with T. lambert's work on global warming. *[Rest of comment deleted. TL]*
I'm a neophyte in statistics, so here's my clusters for dummies interpretation of this.
Suppose Iraq had three provinces, A , B and C. A has a population of Na , B has a population of Nb and C has a population of Nc. A suffered Da war deaths and B suffered Db war deaths and C suffered Dc war deaths, so the actual number of war deaths is Da + Db. +Dc
You're going to do a lot of clusters but you decide for practical reasons you can only visit C plus either A or B. You assign a fraction Nc / (Na + Nb + Nc) to province C. Either Province A or Province B will get the remaining fraction (Na + Nb)/(Na +Nb + Nc) of the clusters. The chance that A gets these clusters is Na/(Na +Nb) and the chance that B gets them is Nb/(Na + Nb)
Okay, now suppose (because I'm assuming this is realistic) that the measured death rate you get in whatever province you measure is an unbiased estimate of the actual number of deaths in that province--or on second thought, to keep it simple, you just count the actual number of deaths in the two provinces you sample.
Then the expected value for the total deaths in Iraq that you'll get is
Dc + (Na/(Na +Nb))(Da)(Na+Nb)/Na) + (Nb/(Na + Nb))(Db)(Na +Nb)/Nb
which equals Dc +Da +Db.
Of course you won't actually get that. You'll either get
Dc + Da(Na + Nb)/Na or else Dc + Db(Na +Nb)/Nb
and if I weren't so lazy I'd calculate the variance. So if you did a perfect count inside a given province and did each province you'd get the exact answer, but if you pair two provinces arbitrarily as I did and only measure the death rate in one of them and extrapolate to both of the paired provinces, you'll get a death rate that is either too high or too low, but the expected value will still be the true value. So pairing provinces increases the variance in my toy model, since a perfect count in all three provinces presumably has no variance.
I assume pairing provinces in the actual paper increased the variance and that the authors are smart enough or had really good software that was smart enough to calculate it. But if my toy model is on the right track, the expected value of this approach would be the true value.
Yes, that's correct Donald. Clustering does not change the expected value, but it does increase the variance, which is what they said in the paper. I have some graphs in [this post on clustering](http://scienceblogs.com/deltoid/2003/09/cluster.php) that attempt to show you what happens.
[Seixon comments](http://scienceblogs.com/deltoid/2005/09/lancet40.phpcomment-page-1/#com…):
>3rd drawing: Basrah: .287 chance of having 3; .144 chance of having 2; .072 chance of having 1; .036 chance of having 0
In my universe probabilities add up to 1. Correct probabilities are 8/27, 12/27, 6/27 and 1/27 thanks to Mr Binomial Distribution. And yes, clustering gives you a different distribution, but if you use the correct probabilities the expected value is the same.
Yes I was wondering why they were not adding up to 1, thanks for that. The exact numbers aside, the point still stands.
If we were to have used the "grouping process" to begin with, we could end up with Baghdad with zero clusters by the mere chance a province it got paired up with won all of their clusters.
This would not give a correct result, as Baghdad wouldn't be sampled as all, and the other area would be vastly oversampled.
Your dice rolling example of clustering is nice, but it is a given that two dice have the same probabilities for their results. Two different provinces in Iraq are not like to equal set of dice, I'm afraid.
So yes, if you assume that Basrah and Missan are exactly the same, which the study did with a "belief" that they never substantiated, then it works out.
Unfortunately it is absurd to claim that two provinces are like an equal set of dice. That derails any notion of the real world.
Thanks for those probabilities though, I will double-check them later and update my blog post with them. Point still stands.
In fact, Lambert, clustering does give different probabilities. Like you showed, those probabilities are if you do a REAL random sample.
The Lancet probabilities are vastly different from these, and claiming that the results will be similar with an entirely different set of probabilities also is absurd.
Would Quiggin mind explaining in his own words why he thinks Seixon is innumnerate.
I can't speak for John Quiggin, but I will begin to credit Seixon with numeracy when he solves the problem I set him in comment no. 53: The number of spots which appears when an honest 6-sided die is thrown just once is a random variable. Call it X. What is the expected value of X, also known as E(X)?
Of course he could get the answer with a quick glance at any introduction to probability theory. But just getting him to look at one would be progress. It really is a bit much that somebody who doesn't know what these terms mean expects his critique of a scholarly paper to be taken seriously.
Would Quiggin mind explaining in his own words why he thinks Seixon is innumnerate (sic)
I think Seixon's own words do a better job. For example, those in comment #48 and #60.
In any case, given your past posts I know you have enough statistical knowledge to read a text on cluster sampling, so you know as well as I do that Seixon is talking nonsense.
Kevin,
Your question is a complete waste of time as it deals with something completely different than what we are talking about. Your attempts to derail the discussion instead of just answer to my challenge shows that this is nothing but a boondoggle.
As for your stupid little problem, the answer:
1/6*6 = 1
1/6*5 = 0.8333
1/6*4 = 0.6667
1/6*3 = 0.5
1/6*2 = 0.3333
1/6*1 = 0.1667
E(X) would thus be 3.5. So now you'll have to tell me why in the hell that is relevant to this discussion. I guess you'll start wandering off into talking about how provinces of Iraq are like equal sets of dice like Mr. Lambert did. Good luck on that.
E(X) would thus be 3.5. So now you'll have to tell me why in the hell that is relevant to this discussion.
As you see, it's not an integer. The fact that X cannot take the non-integer value 3.5 doesn't alter the fact that the expected value does so. Similarly, the expected number of clusters for Basrah given that there are 3 up for grabs and Basrah's population is double that of Missan, is 2.
I explained this earlier. You didn't get it. Do you get it now?
Yes, the expected values will be the same.
The problem is that with the Lancet method, there is no probability of 1 or 2, and the probabilities for 0 and 3 are much higher than with a real random distribution.
The clusters are no longer randomly distributed with the "grouping process". I'm not aware of any accepted cluster-cluster methodology, but please point me in the direction of one if there is any.
What the Lancet has essential done is cluster any number of clusters into a single cluster, and distributed this cluster-cluster via a single random drawing.
As you can see on my blog in the examples, this gives very absurd results if repeated numerous times, or if it was used in the initial sampling.
So now you'll have to explain why that method is only applicable in a 2nd phase and no other phase, and only for certain arbitrary provinces.
As I tried explaining, if we were to use this method for the initial sampling, then Baghdad would more than likely end up with all 33 clusters, or as I showed on my blog, if this method was used more than once, it would have tremendous effects on the result.
In my example of the USA, which you still have not responded to, you cannot simply say "well, Texas and Arizona are similar, so we'll only interview people in one of them and oversample it to make up for the other state". That will produce a bias.
However, for the USA, with election numbers and registered voters, if Texas and Arizona are found to be similar demographically, then this method could be acceptable, even though it will still produce a bias. Say we know that 45% of Arizona and Texas are Republicans, and about 35% of them are Democrats, and the rest independents.
If Texas and Arizona are within a few percentage points of each other on those demographics, or equal, then you can defend oversampling just the one of them as the Lancet did.
When we come to Iraq though, this provides huge problems. There is no way to know the violence levels in each province after the invasion. There is no census or registry that can lead us to conclude that two provinces are similar in this respect.
They paired the provinces up by "belief" and nothing else. That is not scientfically or statistically credible.
Thus, they undeniably introduced a bias into their sample that is impossible to gauge, and thus the conclusions of the survey are completely meaningless.
Thanks Tim. I went back to the paper and found the paragraph where the issue was discussed. This sort of thing is fun--it makes me go back and understand parts of the paper I'd just skimmed over on previous readings.
From my algebra exercise previously I gather that one would try to pair provinces with similar levels of violence in order to reduce the variance. It has nothing to do with the expectation value, which is unaffected. I don't know how this subjective decision affects the variance they calculate, but the description of how they calculated their variance is in the paragraph about bootstrapping, overdispersion in the regression, and other stuff that I know nothing about.
Seixon, are you saying that the subjective pairing of provinces changes the expectation value of the number of deaths, which is what I think is meant by the term "bias", or are you saying it increases the variance in a way that you claim is impossible to calculate? And what do you mean when you say the results of the paper are completely meaningless?
I haven't read your post yet, but I think what you're saying is that if there had been just one round in the cluster assigning procedure, Baghdad would have had the greatest chance of getting all 33 clusters, though I don't think it'd be "more than likely"--it'd be a chance of 5million over 25 million, or one fifth. That wouldn't change the expectation value for the number of deaths--it would just increase the variance, since you'd now be using one province to represent the entire country. They did what they did because they had limited resources and weren't stupid.
I think we may be at a situation where everyone basically agrees. Seixon accepts that "clumping" won't affect the expected estimate, others accept that the sampling wasn't random.
Donald gets it right when he says the the whole row has been around the word "biased". In the sense that bias is systematic in a particular direction, there isn't sample bias caused by clumping clusters. In the long run the estimate will be the same. In a looser sense, that a biased sample is one that isn't representative of the population, then dependency between clusters is a form of sample bias. I think the difference may be between common and technical language.
dsquared,
As it turns out, the city of Najaf was not included in the survey. It was included in the grouping of random governates to be paired and ended up not being selected for polling. The whole thing stinks. 6 very stable provinces were "randomly" excluded and then 3 hot-spots were also "randomly" excluded. Then when Fallujah didn't fit, it was excluded. It stinks of data manipulation (what a nice round number 100,000 is). But since all we have is their word on the "randomness", I guess we have to see how politically motivated they might have been. Their choice of pre-war child mortality numbers was suspect at best, and the accepted range of pre-war mortality numbers should have been introduced into the study (how utterly inconvenient to have posted a NEGATIVE number for casualties). The expressed politics of the Lancet group and the timing of the publication certainly don't help their credibility (I'll be looking for Lancet Iraq Part Deuce in late 2008). A CI sigma of 2 (or 4, don't forget Fallujah) and this study is taken as solid gold proof of 100,000 dead and anyone who disagrees gets the Lambert treatment (yes he's posted a gazillion good refutations, but no one likes the schoolyard bully to win).
Mon Dieu!
Was there ever a better blog post title than this one?
D
Seixon, I've posted the corrected Coalition combat fatality stats on my site if you want to check my work. I believe they confirm that your assertion that the provinces surveyed in the Lancet study were significantly more dangerous to Coalition troops than the national mean *during the actual period surveyed* is incorrect.
Tim:
Can someone you runs a website like you do also be a sock puppet on their own site? Pardon me for saying this but Kevin Donahue sounds remarkebly like a nice version of you despite the broguish name. Now this is too hysterical for words!
Having read this on the post about Hitchens,
Kevin Donoghue Says:
September 22nd, 2005 at 4:53 am
Joe C,
Even if the things you say about Cole are all true (I know that some of them are false and therefore see no reason to trust you on the others), they have no bearing on Cole's knowledge of Arabic. It's a red herring.
I can only assume you put on a sock on you own website. Why? Because language like hand writing is difficult to disguise.
I say this sincerely, as I think Donahue is you in disguise, it is also you in personality and to be honest, he is likable. He's nice, helpful and a teacher type. Please bring him out more.
I'd love to buy Kev lunch. Let me know if you are available one day Tim as I am always flitting around Sydney.
Sorry, I don't understand: how does supplying seperate figures including and excluding Fallujah stink of data manipulation?
I've read criticisms accusing the survey of being both too random and now of not being random enough. The study clearly states the limitations and difficulties of both the sample and the method of extrapolation. I can see how the variance of the data is affected, but that's some way short of completely invalidating the entire survey and the more huffing and puffing I hear, the more I suspect that its main sin is to propose a politically unacceptable reality rather than be misleading or fraudulent.
I suppose I should declare my grievous bias: I supported the war at the time, and with some reservations I still do. I certainly hope the war hasn't caused such a horrific number of extra civilian deaths but it doesn't seem impossible or fraudulent to suggest it has. The Lancet Survey seems the best available estimate, given limited resources and the unarguably hazardous environment.
I ask again: do the New England Journal of Medicine findings contradict or support The Lancet survey? Would it make any difference to you if they did?
Joe Cambria, your trolling has to be a little less blatant if you want somebody to bite. Commentors should read my [comment policy](http://scienceblogs.com/deltoid/comment/) on trolls.
Jet, you're basing your argument on Seixon's Coalition fatality numbers, which don't appear to add up. Running off the same database, but limiting my results to the same time period and excluding non-combat fatalities, I came up with quite different ones.
I think the case has been made that allowing a degree of subjectivity in the pairing process was not ideal. What hasn't been proven is that this made any difference.
14 clusters were unaffected by the pairing; 1 (Anbar) was ultimately treated as an outlier. That leaves 18, of which 8 in total ended up being transferred to new provinces in total. One cluster transfer (Sulimaniyah-Arbil) was between two undisputedly quiet provinces. Three (Qadisiyah, Tamim and Dehuk) were to probably more violent ones. Two (the Basra to Missan transfer) ended up moving to a less violent locale. That leaves the two that were transferred from Najaf to Karbala, which Seixon treats as equally violent provinces, I believe incorrectly (I think you can make the case, based on an accurate count of Coalition combat fatalities, that Najaf was somewhat more violent than Karbala in this period).
The upshot is that the net violent/nonviolent plus/minus is +1 cluster if you believe Seixon's theory, and -1 if you base it on the actual relevant Coalition fatality numbers. Regardless, once Anbar was excluded, the provinces surveyed seem to have had an average level of anti-Coalition violence close to the national average, for what that's worth.
Jet;
...since all we have is their word on the "randomness", I guess we have to see how politically motivated they might have been.
If your argument seems to be that the study is fraudulent (i.e. the results claim to be random, but are in fact fixed). If you really don't want to accept the results, why not use the slightly more charitable strategy of suggesting that they got unlucky by pulling an unrepresentative random sample? It's no wonder that the whole Lancet study debate has generated much more heat than light. Is there anything which would convince you to change your mind regarding mortality in Iraq?
I'm doubtful of the premise there is a correlation between coalition combat deaths and Iraqi deaths. September saw ~30 coalition deaths, ~130 Iraqi military/police and ~450 civilians. Coalition deaths happen in relatively large clumps scattered across the country. Iraqi deaths seem to occur more predictably in high profile target areas.
I'm sure too many people have died in Iraq, but I'm not sure the Lancet is skeptical enough about the 2002 2.9% infant mortality rate. To make the claim that a relaxation of sanctions resulted in a miraculous drop of infant mortality from 10.8% to 2.9% in 2 years, given that the average for 1989-1990 (relatively peaceful years, at least until the invasion) was about ~5%. Which is more likely, 10 occupation hating resistance sympathizing self-pollers lied about their babies, or Iraq dropped its infant mortality rate in two years from 10.8% to 2.9%, or half the 1990 rate of 4.7%?
Given that the Lancet study just used the 2.9% number instead of trying to assign some statistical magic to deal with the discrepancy is all you need to know to realize the study is seriously flawed.
nikolai,
The Lancet lost most of their Iraqi related ethos when they politicized themselves. I'd also be more inclined to give them some credibility if they hadn't decided to just accept the almost magical infant mortality rate they found. How can anyone believe that in 2 years Iraq went from Somalia infant mortality rates to Brazilian rates, and the evidence is a self-poll where the respondents have a motive to exaggerate their circumstance?
Oh, I meant they went from Somalia to better than Brazilian rates.
I'm starting to look like a one man show here, but before anyone brings up the ILCS as confirming the Lancet, it may confirm the pre-war Lancet number (but like I said before, that is a magical drop in deaths), but it certainly doesn't confirm the post-war Lancet number. The ILCS shows a more believable increase from 3.3% to 3.5%, while the Lancet self-pollers say it was 2.9% to 5.9%. The ILCS found an increase of 3 infant deaths per 1,000 and the Lancet found an increase of 30 infant deaths per 1,000. The ILCS doesn't help the Lancet at all.
I realized my "fun" comment sounds ghoulish, given what we're talking about. I was caught up in learning a tidbit about statistics I didn't know before. (Which would include most of statistics beyond the elementary level).
Nikolai said we're all in agreement. I can't tell for sure. Seixon has been talking about "bias" and "worthless" and it'd be nice to have clarification of what he means by those terms. What's odd is that, as dsquared pointed out at Crooked Timber, there's not really much doubt that there have been tens of thousands of deaths in Iraq, somewhat higher than what Iraq Body Count has found (which would be a minimum) and possibly as high as the Lancet paper suggests. And it's odd to see all this criticism of a paper for its bias in the colloquial sense when the most obvious bit of data manipulation, if you want to call it that, was when they left out the horrifying statistics of the Fallujah sample and thereby chucked out the Anbar province. And if I remember correctly,(my copy of the paper is at home) their sample in Sadr City or someplace where you'd expect a high death toll by chance turned out to have suffered zero violent deaths. So it seems that in practice the Lancet figure of 60,000 violent deaths didn't include any neighborhoods where people died in large numbers due to intense fighting or bombing in urban settings--in my perhaps naive way I would expect that that these comparatively small areas would account for a large fraction of the actual death toll and with a survey that only has 33 samples you've got a better than even chance of missing the really hard hit areas unless at least one in 50 neighborhoods suffered such casualties. And much of that intense urban fighting occurred in the summer and fall of 2004, after the UN survey. And yet we're supposed to think an estimate which excluded the one sample that suffered intense bombing was biased upward. Right now I'm using the word "bias" in its colloquial sense--a study with a small number of samples is going to miss neighborhoods that lost large fractions of their people unless such neighborhoods are relatively common. But I don't think that's a bias in the expected value sense. If Seicon wants to talk about bias in the colloquial sense, than anyone can play and I'll ride my own hobbyhorse, which is that we don't know what the intense urban warfare in the summer and fall of 2004 did to civilians--the UN survey ended too soon and the Lancet study had too few clusters.
I realize from earlier rounds of this that a small number of clusters in the Lancet study could also miss neighborhoods where Saddam had gone on some prewar rampage, but I don't know of any evidence he was going on such rampages just before the war. The UN survey could have picked this up, if they asked.
On the subject of bias, I just read a post in another comment section where my debate opponent used the IBC analysis of their own data to refute the Lancet study--in particular, since the IBC analysis shows that most of the deaths attributed to Americans occurred in the early months of the war, this meant there couldn't have been massive civilian casualties in Fallujah during the summer of 2004 because it doesn't show up in the IBC count. I blame IBC's analysis more than my opponent, because they do calculations of all sorts of percentages and give the results to two or three significant digits, even though their data is based on Western news reporters whose access to most of Iraq became increasingly restricted after the early months. The US and Iraqi governments aren't going to say how many civilians they've killed and if reporters can't go out in the field and talk to ordinary people, they're not going to be able to find out very much. They will be able to report on insurgent suicide bombings, however. The IBC analysts seem to have fallen in love with their data and they report on their various calculated percentages as though this came from an unbiased sample of the total number of Iraqi civilian dead. There's not a hint of any sort of bias problem in their analysis--compare that to the Lancet paper, which spends paragraphs discussing why its numbers might be wrong. Naturally it's the Lancet paper that attracts criticism.
Donald Johnson,
Perhaps that is because most people haven't heard of the IBC report, but everyone has heard of the Lancet report.
Donald Johnson, Perhaps that is because most people haven't heard of the IBC report, but everyone has heard of the Lancet report.
Wow, that has got to be the best darn reason to be critical of a paper that I have ever heard.
Why worry about accuracy when you can attack it because it is well known!
"The IBC analysts seem to have fallen in love with their data and they report on their various calculated percentages as though this came from an unbiased sample of the total number of Iraqi civilian dead."
Donald, correct me if I'm wrong, but IBC doesn't calculate its numbers based on surveys and extrapolations. Last time I checked, their figures were arrived at solely from media or morgue accounts of casualties.
Your comment# 81 seems to be suggesting that the Lancet violent excess death extrapolation of some 60,000 is more likely to be an underestimate of the actual death toll, rather than an overestimate. You base this on the exclusion of the Falluja data, and the absence of deaths from clusters in areas of significant violence.
I mentioned this several months ago here, but didn't get much feedback. The ILCS survey was a vastly broader one than the Johns Hopkins effort, and most seem to agree that the ILCS is much more likely to provide an accurate picture of the war-related mortality rate than the Lancet study. I think it's a mistake to claim that the ILCS corroborates the Lancet war related death extrapolation, because there are indications that the composition of the two war related totals are quite different.
The Lancet extrapolates approximately 12,000 children killed ex-Falluja, all by the coalition. On the other hand, the ILCS calculated slightly less than 3,000 war-related deaths of children. I realize there has to be an adjustment for the difference in time periods covered by the two studies, but I suspect this is largely offset by two factors; the ILCS cut-off for children is 18, while the Lancet cut-off is 15. The ILCS figure is made up of fatalities inflicted by all sides, which acknowledges the fact that insurgents certainly contributed to the child death toll, something the Lancet extrapolations completely exclude.
There's one other significant indication that the Lancet and ILCS may have very different compositions of their war-related death estimates. Tim disagreed with me when I raised this at the time, but I'm convinced that the ILCS war related death survey question was intended to capture the deaths of Iraqi military personnel during the initial invasion, and logically must have elicited reports of such deaths from respondents. The Lancet survey failed to capture any deaths of regular Iraqi military servicemen (which was understandable given the Lancet's survey question).
It would be revealing to see a more detailed breakdown of the ILCS data. What it seems to suggest, from the data relating to the deaths of children, is a much lower civilian death toll from coaltion bombing than the Lancet portrays. The ILCS covers the first year of the war, including the invasion phase. If it's numbers indicate a dramatically lower civilian death toll from coalition forces during the invasion, then that may be an indication that it's faulty to assume the fighting post-April 2004 killed tens of thousands of civilians in places like Falluja and Najaf, as you're implying, Donald.
Mike, the Lancet study found 4 children killed by the coalition ex Falluja. This really is too small a number to extrapolate from.
Mike, my point was that the IBC count is biased by the fact that the reporters are going to find it easier to count civilians killed by the insurgents. That's because they get much of their information from US and Iraqi government sources, which will tell them how many civilians are killed by the enemy, but not how many they've killed. Also, in Vietnam it was known that the bodycount of enemy soldiers included a fair number of civilians and I suspect the same occurs in Iraq. So yeah, the IBC numbers are an actual count (of sorts), but not only will it necessarily be an undercount, but it will also be a sample of the total civilian death toll that is biased by the limitations placed on the reporters. I used the analogy of the French/Algerian war in another comment section of another blog. If you tried to count the deaths in the French/Algerian war solely relying on press accounts, you'd probably have a fairly accurate count of the civilians killed by the Algerian rebels, but a huge undercount of the number of Algerian civilians killed by the French, especially in areas inaccessible to reporters.
Interesting you brought up the 3000 figure in the UN survey, Mike. That was 3000 dead for the first year or so (or maybe 15 months, someone else said somewhere). IBC counted 1300 dead in the first two years, so at a minimum they missed more than half of the
child deaths.
Though to be fair, I don't know the error bars on the 3000 dead children figure.
Tim:
The extrapolated war related deaths from children constitute much of the foundation for the Lancet's most controversial and high profile conclusion, that violence accounted for most of the excess 100,000 death estimate, and most of the violent deaths were suffered by women and children as a result of coalition air strikes.
I think you have to take the good with the bad in this regard, Tim. Much of the defense of the Lancet study has involved defending " small numbers " derived from the study. That sometimes cuts both ways.
Donald:
The statement from you that I referred to was a straightforward (at least from my perspective) assertion that the IBC was engaged in a statistical exercise based on survey samples. That's what I was taking issue with.
>>"That's because they get much of their information from US and Iraqi government sources, which will tell them how many civilians are killed by the enemy, but not how many they've killed."
I don't think that's the case Donald, although I'll have to do some chacking at IBC. If I recall correctly, the IBC has attributed many of the deaths it tabulated to coalition and Iraqi security forces.
>>" So yeah, the IBC numbers are an actual count (of sorts), but not only will it necessarily be an undercount, but it will also be a sample of the total civilian death toll that is biased by the limitations placed on the reporters."
Another significant variable that often gets overlooked is the fact that the IBC count most certainly contains insurgent deaths that have been counted as civilian casualties. An IBC estimate of how many fall into this category is very difficult, if not impossible to determine.
>>"Interesting you brought up the 3000 figure in the UN survey, Mike. That was 3000 dead for the first year or so (or maybe 15 months, someone else said somewhere). IBC counted 1300 dead in the first two years, so at a minimum they missed more than half of the child deaths."
Essentially, with the IBC figure for children, you've provided further evidence in favour of the accuracy of the ILCS study over the Lancet study, in terms of the discrepancy that exists between them in the context of war related child deaths.
Jet,
With regard to the infant mortality figures, did the 5/1,000 figure which came from Saddam's regime?
During the sanctions period, Saddam was conducting extensive propaganda abotu the impact of sanctions on Iraq's chidlen (which extended to such incredibly vile acts as deliberately withholding chemotherapy from children with cancer even though the drugs were available.)
I believe the recent survey from the IMF showed an even higher figure for infant mortality
" Under five years mortality is estimated at 115 (compared to 33 in Jordan and 107 in Yemen). Infant mortality is currently estimated at 102 per 1,000 live births, compared 105 in sub-Saharan Africa."
http://www.imf.org/external/pubs/ft/scr/2005/cr05294.pdf
Ian gould,
I believe they are UNICEF numbers I got from this very site. Search this site for "iraq child mortality" and Lambert has a pretty good thread covering it. I'm really just rehashing Lambert's own words about the infant mortality bit.
I still wish someone would explain on this thread if clustering of clustering is really a statistical method and the variance accounts for this method, or is Seixon right that CI does not account for clustering of clustering.
And on the off topic point, could someone clarify why the 100,000 number has so much backing even though the de facto final (blog) word on the matter, Lambert, has pointed out the problems with an extremely large chunk of the data (infant mortality numbers).
jet, clustering of clustering is called multistage clustering and yes it's a valid method as has been explained several times already.
"...as has been explained several times already." Always with the charm.
I don't know how I missed your explanation, perhaps it was explained here or here?
Since you are at least giving out teasers of information for the less "numerate", how did the Lancet team calculate the variance from their "belief" that governorates were alike? Or did they not and just use the exact same data for both provinces?
I should clarify. I'm referring to the situations where the teams found they could not access their randomly selected areas and would just find a new one they "believed" was similar. This isn't random and is accounted for somewhere, right?
Jet seeks enlightenment. This can hardly be the Jet who contributed this observation to a Crooked Timber thread:
No, of course not.
Howdy folks. Everyone have a nice weekend?
I've been away for the weekend, so I haven't had any time to look over all this stuff yet, just kind of skimming through it now.
BruceR, I skimmed over your numbers and I was just thinking one thing: do the pairs of governates have similar violence levels, even with your numbers? You say that all of Iraq had 28 deaths/M and the 10 provinces they sampled had 27 deaths/M. That's nice, but did those pairs of governates actually have similar levels of violence or not? That is the only relevant thing here, not only that, but the Lancet team did not support their "belief" with anything at all. It was an arbitrary pairing done for what reason?
Those still commenting that the sample is still random, etc. OK look. You could distribute all 33 clusters in one huge cluster and do it "randomly". You know, like picking a number out of a hat. Does that mean that the sample is still randomized? No. All the clusters go in the same damn spot. Now, cluster sampling is all supported and everything, but that depends also on how many clusters, and how many households each cluster contains. The UNDP did a cluster survey also, but they had 22,000 households total, the Lancet team had 990... Also, the UNDP did not cluster clusters arbitrarily like the Lancet team did.
For true randomness, I have been hearing people say that each household in Iraq has to have a chance of being chosen. No, no, no. They have to have an EQUAL chance of being chosen. The clustering of clusters in the 2nd phase screwed all that up. That was the whole point of me posting up those tables with probabilities, to show that the chances of being chosen were NOT EQUAL in the 2nd phase.
BruceR, I thank you for pointing out that I included non-combat deaths. That was a big mistake on my part, and I will redo the calculations over the Mar 19, 2003 - September 20, 2004 period. I think the casualty site has made it lots easier to extract data since I did that tabulation. Besides that though, I'm wondering if you have forgotten what the point was of doing it. Did the paired governates end up with similar levels of violence or not?
First, whoever was thinking that Mr. Lambert was spoofing himself as Kevin... Well, unless Mr. Lambert is currently residing in Ireland, I don't think Kevin is anything other than a fan of Lambert from Ireland. Mmmk? ;)
BruceR, I am troubled by your approach to my coalition death numbers, and mostly obfuscating from the real matter at hand here. The point I showed with those numbers was that, in fact, the governates that were paired up did not have similar violence levels. The JHU team had this "belief" that the governates they paired had similar violence levels, thus then it would be OK to oversample half of them and leave half of them out. Now, they provided no reason to believe this belief, so we are already venturing into dangerous territory as far as science and statistics goes.
With that said, let's take a look at the numbers you came up with, which I haven't had time to double-check as I am guessing that will take a while. These are your numbers, in coalition deaths per million Iraqi residents:
Pairs:
Ninawa (12) & Dehuk(0) <- similar?
Sulaymaniyah (0) & Arbil (0) <- similar, just as my analysis said they were
Tamim (16) & Sulah ad Dinh (72) <- SIMILAR???
Karbala (11) & Najaf (21) <- similar? (these were actually similar in mine...)
Qadisiyah (5) & Dhi Qar (33) <- similar??
Basrah (27) & Missan (13) <- similar??
So in fact, BruceR, you have proven quite useful. You have shown that in fact 5 out of the 6 pairings were not similar at all. In my analysis, 4 out of the 6 were not similar. Now you have corrected my numbers, and shown that in fact, 5 out of 6 were not similar.
You tried, for some reason, to circumvent this by saying that the Iraq total was 28 and the total for the provinces in the sample was 27. To do this, you had to take Anbar out of the provinces that were sampled. Hmmmmm, why? They sampled Anbar, so why did you take them out?
Anyways, the main point is that you have proven my point, even more so than I did: Lancet's pairings, even by rough estimates, did not have "similar" amounts of violence at all thus undermining the already arbitrary and suspect reasons for doing it.
Also, I am hearing some talk of the Lancet conducting "multistage cluster sampling". Really? One problem: they didn't use simple random sampling.
I can find absolutely no multistage clustering or multistage cluster sampling that allows for clustering of clusters. In fact, from cursory reading, I find no clustering of clusters at all. Multistage clustering entails distributing clusters randomly by large geographic area, and then into smaller areas within that area. With Iraq, there was no reason to do this. The geographic strata were not large enough to warrant multistage clustering. Even if they were, multistage clustering still entails simple random sampling. Simple random sampling doesn't mean you combine all the clusters into a super-cluster and then randomly place that one cluster.
In fact, the effect this clustering of clusters has is to actually reduce the number of clusters used in the study. If you changed the Lancet study to 11 clusters from 33, then you would get most likely the results you got with the Lancet "grouping process". 11 clusters are far too few for such a sample.
But please feel free, Mr. Lambert, to show me how the Lancet's methods are supported in theory or even a practical previous example.
Oh, and just to throw in another bone, cluster sampling is typically done with larger sample sizes to negate the effects of the clustering. The UNDP did this, with their 22,000 households. The Lancet team? 990.
Good night! Have a good time digging bigger holes for yourselves!
Kevin Donoghue,
You crack me up. That was obviously someone who didn't like what I had to say, so they were responding in the only way they could. Maybe I'll write something asinine and post it as you. Then I can refer to it whenever I want to make you look silly. Hell, it was probably you who posted that in the first place, so I'd just be giving fair turnarounds.
Seixon, if you're saying that the Lancet sample was nowhere near as complete as the UN study, nobody is going to disagree. You were saying the Lancet study was fraudulent, worthless, etc... This is silly. It was a less-than-perfect way to do the estimate, forced on them by the danger and limited resources and the expected value of their result wasn't biased. The figure they got for violent deaths is not out of line with the more detailed study.
Mike, I think the UN study is the one whose numbers we should believe for the period it covered, so around 20-30,000 extra deaths are indicated. The Lancet study gives 60,000 violent deaths for a somewhat longer period, of which maybe 40,000 were from either Americans or insurgents though that breakdown depends on taking the small numbers seriously. It overestimates (if we take those small numbers seriously) the number of children deaths. The two estimates are right on top of each other if criminal murders aren't part of the UN study statistic--if they are, then the Lancet 60,000 midrange figure is a little too high. Given the longer time frame maybe 40,000 or so would be a better guess. All this excludes Fallujah and any other place in Iraq where intense urban fighting may have taken place after the UN study was complete.
To me the two main points of the Lancet study are these--
A) Several tens of thousands of people died violently in the first 18 months, much higher than the IBC statistic.
B) Something really terrible may have happened in Fallujah as a result of the American air strikes there and this may also indicate high civilian casualties in other pockets of Iraq wherever Americans used air power. You can't get a sensible extrapolation from one neighborhood in Fallujah for the entire town (and certainly not for Anbar province), but reading what happened in that neighorhood, along with the judgment of the interviewers that large areas of the town appeared at least as devastated suggests thousands might have died there. So I value the Lancet paper more for its reporting than for its rough estimate of death toll (which was in the right ballpark).
Yes, IBC counts people killed by Americans. They compose around 39 percent of their dead, I think, for the first two years. My point is that there's likely to be a built-in bias in the way the information is collected which means there aren't going to be reliable body counts for places like Fallujah. The Lancet interview could be considered a media report--they investigated and found about 50 deaths. Presumably some other very very brave reporter could have snuck into Fallujah and interviewed people and tallied a few dozen deaths as well. But I don't think it would be recorded by IBC unless confirmed by another news source, and at any rate, it would likely be only a small portion of the total dead in Fallujah.
IBC also gets a huge portion of its data from morgues, but if people blown up in air attacks aren't brought to morgues, they won't be counted. I don't know if the media collects all morgue reports or if the Iraqi government sometimes controls what reporters can hear from the morgues.
I mentioned the 3000 child deaths in one year vs. 1300 child deaths in two years in another blog, which is why I was interested to see you bring it up. I agree it shows the naive small-number extrapolation of 4 Lancet deaths to mean 12,000 is probably wrong. But I think it probably also shows that the IBC methodology significantly undercounts some deaths. The strength of that argument depends on how big the error bar is for the 3000 number.
IBC claims to be careful in making sure they're only counting civilians. I don't know how they know. I bet a lot of "insurgent" bodies are civilian as well, which they'd have no way to check.
I'm frankly amazed at how tight their estimates are--it's always something like a range of 10-20 percent. I think something about their methodology is giving the illusion of much more precision than is actually present in the news reports and reading their report on the first two years of their data gives me the impression that they take the accuracy of their data much too seriously. There should be caveats sprinkled all through their report, every time they give a percentage, but they report it all to three significant figures as though it meant something.
"IBC also gets a huge portion of its data from morgues, but if people blown up in air attacks aren't brought to morgues,"
Additionally, muslim tradition requires that bodies be buried within 24 hours and forbids autopsies.
There are also credible press reports now of bodies being found in mass graves or dumped in rivers. Presumably not all bodies disposed of in this way are discovered and reported.
However the morgue figures for Baghdad for violent deaths would seem almost definitely to include some insurgents.
I don't think the Lancet study is perfect, I do think the actual number of excess fatalities is probably somewhere around 100,000 plus or minus 20,000. The continuing deaths since the end of the study would, of course, increase the total significantly.
What I am very disappointed by is the number of people claiming on spurious or extremely weak grounds that the people who conducted the Lancet study committed deliberate fraud.
Way to keep looking the facts in the face and keeping the faith guys. I will be doing a new post later today that makes it ultimately clear how the estimate was biased and that their method was anything but statistically/scientifically sound. You know, for those who continue to be in denial about it.
>I will be doing a new post later today that makes it ultimately clear how the estimate was biased
While you're at include your real name and your relevant qualifications - you know like the people who've you've been accusing of fraud did.
Ian,
I'm not sure what the point of giving my name would be. Do you? My qualifications? Again, are we seeking to argue from authority here instead of take the facts head on? I have no "qualifications" for statistics, and as far as I know, neither did the people who carried out the study in the Lancet. They were medical doctors. Tim Lambert I see has some kind of degree in math, which is kind of scary seeing as how he has resorted to almost nothing but condescending remarks instead of debate.
I think the problem here is that Lambert knows what I am saying has merit, but he doesn't want to acknowledge it, for obvious reasons.
If I understand correctly Seixon is attacking the very foundations of probability theory, so I don't think we need worry too much about Roberts et al. Nobody will think the less of them if it turns out that Seixon is right and Messrs Pascal and Bayes got it all assways.
Hmm, Sexion,
FYI, the four authors of the Lancet study, Les Roberts and Gilbert M. Burnham are at the Johns Hopkins Bloomberg School of Public Health, Richard Garfield is a Professor of Nursing at Columbia University and also an expert in public health; and Riyadh Lafta and Jamal Kudhairi of Baghdad's Al-Mustansiriya University College of Medicine
Seixon, do you understand the difference between a School of Medicine and a School of Public Health? And, by the way, do you know what people in a School of Public Health do?
Frankly, you just punched yourself in the face. Google is your friend.
Seixon, my counterargument took as a given that any limiting of the area surveyed, if based on subjective criteria, *could* queer a survey's results. The question was whether you had succeeded in providing evidence that it *did* in this case, by your using Coalition combat fatalities as a comparative baseline.
The net change you were asserting in your original argument amounted to one cluster out of 33 moving to a more violent province (three to more violent, two to less violent for a net of +1), so the effect of the pairing process would likely not have been huge in any case. I suggested a re-examination of the database you used actually made for a net of -1 (three moved to a more violent province, four to a less violent). Comparing anti-Coalition violence on a per capita basis in the 10 provinces surveyed produces a similar figure to Iraq as a whole in this period, which would seem to also suggest that any effect on the final 100K figure of the pairing process alone could not be large.
I would have thought it obvious that I excluded Anbar because the one cluster in Anbar (Fallujah) was ultimately excluded as an outlier, so Anbar as a province effectively did not contribute at all to the final 100K estimate.
Kevin,
I'm not going against anything that Pascal and Bayer have said. I have said that clustering of clusters is not what multistage clustering is about, and you will find that to be correct. I have also said that clustering these clusters has no precedent in statistics, something Lambert tried to steamroll by giving vague comments. The selection of some provinces for a 2nd phase, while leaving out the others, is the very definition of introducing a bias. Lastly, as BruceR can't get himself to admit, is that the pairings were completely fraudulent on their face, which already started with them simply doing so according to a "belief". Doing things according to "belief" should ring off chimes and bells all over the place, but hey, you guys want to believe the "belief" so then it's OK, right?
Eli, thanks for pointing that out to me. I said "as far as I know" leaving room for someone to correct me. That is really besides the point, since an attempt to argue from authority is really quite pathetic instead of taking on the arguments I have put forth. By that standard, we should just believe whatever any doctor says just because he is a doctor. Right? Give me a break. I mean, we should believe Dan Rather because he is a veteran reporter, right? Mhm, sure. When politics and opinions get involved, things aren't always what they seem.
BruceR,
I find it hilarious that you took Anbar out of the surveyed group. You are again trying to sidestep what the point of finding out those coalition death totals were from the outset. We were trying to establish the difference between the sampled provinces and the unsampled areas. Anbar was sampled and purposely left out of the "grouping process". So then you have to ask yourself, why was Anbar left out of the "grouping process"? Is it because they knew Fallujah was ripe with deaths to count?
All of this is you trying to get around the fact that 5 out of the 6 pairings were completely different in coalition deaths per million residents, thus undermining the JHU team's reasoning for pairing them up completely, as if their unsubstantiated one to begin with had any credibility.
So guys, when are you going to get around to showing me similar studies done where clusters were clumped together? Or any of the other issues I have brought up? Nope, instead you are coming up with irrelevant garbage and trying to bury me by arguing with authority.
I'm going to go write a new post about this now. I'm still waiting for Lambert to show me these magical multistage clustering approaches that had a precedent before the JHU team used them.
I've been wondering how the Lancet people handled the variance calculation, but don't know what bootstrapping and the rest of the jargon means. I'm guessing the variance they got is valid if you assume the left-out provinces weren't drastically different from the rest of Iraq, but you'd have to make some sort of assumption along those lines since the left-out provinces weren't sampled.
Anyway, the number they got for violent deaths is reasonably close to what the more detailed survey got, given the shorter time interval, so those bumbling Lancet authors apparently did a decent job in the end. The whole debate over the Lancet methodology has been educational for me, but in the end it's clear that several tens of thousands of Iraqis have died and we're not in a position to pin it down much more closely than that. "Several" is a useful word that way--it provides just the right level of fuzziness. Naturally I think my own obsession is much more important than all this quibbling over methodology--when you look at what the investigators actually saw in Fallujah and found in that one neighborhood, isn't it very likely that uncounted thousands of people died there?
"I'm not going against anything that Pascal and [Bayes] have said."
Seixon is too modest. The comments on his post "Slaying the Knight of the Lancet", as well as the post linked above, indicate that he entirely rejects conventional thinking about conditional probabilities. The clearest indication of this came when I made the following request:
Seixon replied:
Before I got back with my cumbersome response, Mike D presented the case for the conventional wisdom more succinctly than I could:
Seixon was not impressed:
He was equally unmoved by the arguments of Detached Observer (another slave to received dogma) in comments to the post linked above.
Donald,
I agree with you about what's important, but we can't do much about US policy. We can counter Seixon's effort to discredit serious researchers by making up his own version of probability theory and peddling it to anyone who will buy it. (Right now I would say his only taker is Jet.) Anyway, he's entertaining. I'm looking forward to his latest effort.
If you must another post Seixon, could you at least use the word "bias" correctly? It has a specific meaning when you are talking about sampling, you know.
Kevin Donoghue,
"Right now I would say his only taker is Jet." Perhaps because when I asked about multistage clustering and how the Lancet team dealt with the variance from the problems they encountered, I was given a delightfully uninformative snub from Lambert.
But I'm probably more in line with Donald Johnson's take of the matter. There are obviously problems with the study that none of the defenders will explain, but the study still confirms that a crap load of people have died. And given this coming month will probably see the start of overt civil war, any problems with the Lancet study being used as a bludgeon just don't matter anymore.
Jet, fair enough; evidently I read too much into your comment on Seixon's blog that Tim Lambert's remarks "weren't appropriate for the points [Seixon] made and hurt his argument."
Kevin Donoghue,
Concerning my comment on Seixon's blog, you might find this hard to believe, but sometimes I jump to conclusions about things I'm really not very well informed about ;)
But the Lancet's technique didn't sound like anything I could dig out of Wikipedia and no one would confront Seixon's argument head on, which led me to believe he was on to something.
>I'm not sure what the point of giving my name would be.
1. As I said, you keep tossing around accusations of incompetence and professional misconduct. If you made such claims anywhere other than on the net, you'd be opening yourself up to legal action for defamation. (Technically, the same laws apply here but in practice, it's open season for slander and abuse.)
If you are absolutely convinced of what you say (and you say your next post will absolutely convince US)you should be prepared to stand behind what you say.
2. At this point it's the original four authors of the Lancet study (including a man who appears to be regarded as a world authority on epidemological studies); the people who conducted the peer review; the thousands of PHDs who read the paper (including the ones who criticised it on other grounds)and Tim versus you.
"Argument from authority" is a frequently misunderstood concept. It refers to the use of qualifications in one area to impart credibility in another area.
Were Tim (for example) a marine biologist or a geologist with no additional training in statistics, his view on this matter might carry no more weight than that of a lay-man. He's not.
Ian,
The fact that I have a blog where I lay these things out means that I stand behind what I say. If someone really wants to take legal action against me, or some other reason that would require my name, then I would gladly divulge this information. With that said, any resourceful person would be able to find out my name anyways. What's in a name?
You talk about the peer reviewers, other people who have read the study, etc. Look, has anyone gotten to see what the peer reviewers had to say about this study? I sure as hell haven't seen anything, even though I have looked. I have also seen fairly general and vague quote-shopped statements from various scientists such as, "Les has used and uses the best methodology." Wow, that's great, but did he in THIS STUDY? In fact, if you guys want to bring forth a previous survey done by Les Roberts where he used the same methodology, e-mail it to me straight away. Unfortunately, it seems Mr. Roberts took a few liberties with his methodology this time. It seems the need to get a mortality study published the Friday before the presidential election was more important than anything else. Oh, and by the way, have the Lancet website lie about the conclusions of the study. That's just a queer coincidence though, right?
I'd say argument from authority could also mean to just fling around your weight as an "expert" to quell arguments that you yourself know are correct. In other words, "what do you know? you're not an expert. I am!"
But yes, it can also mean using an irrelevant title or authority to give your opinions of other matters. I see that all the time here in Norway. The only guy that gets up on TV and defends anthropogenic global warming in Norway is in fact a biologist. Go figure.
I think you can still make the distinction between an actual "argument from authority" and the "argument from authority fallacy," which is what you're referring to, Ian. For instance, in this reference from a website on logical fallacies:
"It should be noted that even a good Appeal to Authority is not an exceptionally strong argument..."
In this case, pointing out simply that Seixon knows less about probabilities than the Lancet paper's authors or reviewers is certainly a "good Appeal to Authority." It is also, as the logicians' text says, not as strong an argument as some might prefer to see.
I don't think anyone was saying Tim was committing a logical fallacy by pointing that fact out, just that he was being unnecessarily dismissive (not to Seixon, who I fear is beyond help, but to those of us who may suspect the guy's wrong on this, but still wanted to learn a little through the process of figuring out WHY). IMHO.
Bruce R,
For "those of us who may suspect the guy's wrong on this, but still wanted to learn a little through the process of figuring out WHY", both Mike D and Detached Observer have persisted in explaining things on Seixon's blog. Mike D's latest is particularly clear, while D.O. is, I fear, running out of patience. Seixon certainly isn't being fobbed off with appeals to authority. He is just being asked to do the math.
Seixon you continue hitting yourself in the face. Given your initial statement that:
>I have no "qualifications" for statistics, and as far as I know, neither did the people who carried out the study in the Lancet. They were medical doctors.
Your modified retreat to:
> I said "as far as I know" leaving room for someone to correct me. That is really besides the point, since an attempt to argue from authority is really quite pathetic instead of taking on the arguments I have put forth. By that standard, we should just believe whatever any doctor says just because he is a doctor. Right?
Ranks right up there in the Annals of Implausible Deniability. You appear to think that this is make-up-a-fact-Friday, and you can toss any disinformation you want out there, with the burden falling on others to correct you. Not
The US authors of the Lancet study are expert in the field of health surveys moreover their area of special experience is international emergencies. Any insinuation or claim by you to the contrary is just another fist in your face.
FWIW, Roberts and Garfield are not physicians, Burnham is.
And yes friend it pays to give more credance to what someone who has had experience and studied in an area has to say over your average blog frog. If you want examples, take a look at http://www.crank.net.
Remember Sexion, Google is your friend. A small hint: cluster sampling.
BTW Tim, take a look at http://www.malaria.org/ddtlancet.html
Eli,
For your information, it was actually medical doctors who carried out the survey. You know, the ones in the field? Roberts and all the rest didn't carry it out, they designed it. If you are going to keep flogging one simple and irrelevant thing such as that, and keep flogging the "they are experts, so shut up" rhetoric, then just save it.
Kevin,
I'm not disagreeing with you guys on what you are saying. Unfortunately, you continually act as if what you are saying relates to the Lancet study when in fact it does not.
I'm almost finished with my new blog post, and I expect you to take my points in that post head on instead of creating a bunch of irrelevant examples and BS that don't cut to the chase on what I am actually talking about.
I mean, Lambert's original post is the golden example of this. He completely mischaracterized what I meant in my blog post and made me sound like a complete moron. I know it must be Obfuscate-at-Seixon's-blog week, but seriously.
Sorry Seixon, but this post accurately described you. You don't understand sampling and you are determined not to learn. You've repeatedly ignored detailed and careful explanations of your errors.
I try to avoid argument from authority by itself because although it gives you the right answer it doesn't let you understand why it the right answer. But I think it is worthwhile alongside other arguments.
Yes, I particularly enjoyed the detailed explanation of multistage cluster sampling you gave me to show that the Lancet "grouping process" is a statistically sound practice.
Oh. Wait.
Anyways, for those of you who are interested in actually refuting my points, instead of beating up on strawmen, enjoy my latest post which I hope is very clear on what I mean.
Let's see if we can get the high and mighty Mr. Lambert to explain how the Lancet "grouping process" constitutes multistage cluster sampling, or confronting the akward truth that this process fundamentally changed the probabilities associated with these provinces to get any number of clusters.
I tried to lay it out as precisely and neat as I could in this post, so I hope I will get some precise and succinct feedback.
What I don't understand is that if Sexion has come up with something that invalidates the Lancet survey finding, why hasn't it been noticed by the many others who have put so much time and effort into trying to dismiss the work as flawed?
What's the statistical probability here?
Why waste time on some blog, Seixon, when you should be composing a letter to Lancet, as well as Nature etc?
Isn't it profound that some amateur with no experience in the subject can overturn the findings of those who have spent their lives in the field?
D
I think it has mostly to do with attention to detail. I think that most people who looked over the study did not see that the pairings' clusters were grouped together and then distributed with one random event.
In fact, many people on here acted as if the clusters between pairs were distributed via simple random sampling, unaware that they actually were not. They acted as if, "well, the clusters ended up that way by chance," and while that is true, it is unsupported statistically as far as I can fathom.
Combining clusters in a 2nd phase, in only arbitrarily picked provinces, to ensure that 6 provinces were to be excluded is completely without merit statistically.
It is as if they were using 11 clusters because then it would be expected that only Baghdad plus 8-9 others would receive clusters. In the end, that is exactly what happened with the Lancet survey. Using 11 clusters is an insanely small amount, which will give wide variances as the Lancet study has. However, that is not what they did. Instead, they chose to venture off into a completely unsupported "grouping process".
Dano,
What would be the point of writing in to the Lancet or to Nature? Whose side do you think their editors are on? You don't have delusions that the editors of these things are impartial, do you?
In any case, if no one here can provide evidence that this "grouping process" was sound statistical practice, then maybe I will have to submit it somewhere to get an answer on this.
Mr. Lambert pretending that it constitutes multistage clustering is hilarious.
What would be the point of writing in to the Lancet or to Nature? Whose side do you think their editors are on? You don't have delusions that the editors of these things are impartial, do you?
Whoopsie! Forgot - why, they're on the vast green-wing conspiracy, out to socialize all of us into being Canadian or something.
Now, where's that tinfoil...hmmm...seen it around here somewhere...
D
Dano,
If you think that most of journalists and editors being Democrats or liberals is a conspiracy, I have some property in Atlantis to sell you.
Seixon,
If you believe the editors of the world's leading science journal (Nature) and (arguably) the world's leading medcial journal (The Lancet) ignore deliberate errors in published papers (and presumably in the case of The Lancet participated in the original falsification) in order to advance some political agenda then I have some land in a militia compound in Montana to sell you.
Bring plenty of ammo, the war against Zog could start anytime now.
Ian,
do you think the metal detectors at the Montana camps allow tinfoil hats through, or would Seixon have to take his off prior to entry?
D
Guys, what's the difference between the editor at Nature and the editor at the New York Times? There are basically a few people who control what goes in and what doesn't. Does there have to exist a conspiracy for many of these editors to want to promote certain things? No.
I mean, is it a conspiracy that the Washington Times, FOX News, NY Post, among others, publish things that tend to be pro-conservative or anti-liberal? No, of course not.
I have argued that the Lancet most likely published the study for the following reasons:
Now, I find it very interesting that the Lancet website, in releasing the study to the public, said in their headline that 100,000 CIVILIANS had been killed. In fact, the exact heading on Lancet's index page was:
"New Early online publication: 100 000 excess civilian deaths after Iraq invasion"
Now, why was it written like this on the site? Why did it stay like that until it was displaced by a new title?
If the Lancet journal can't even get the conclusions of a study they peer-reviewed right, then... WTF?
Or perhaps, just perhaps, the false headline was intentional...
Nope, no agenda here, move along, nothing to see here...
Keep ignoring the evidence and facts guys, it's what you do best.
You forgot, Seixon,
4. Because cute puppies with big brown eyes may have been killed, and that would raise the ire of PETA and hasten the end of the war.
Widdle puppies, filled with shrapnel and DU. If I weren't so strong and resolute, I'd be crying right now, thinking of the cute puppies filled with shrapnel and DU.
D
where's my widdle puppy gravatar? *sob*.
D
Wow, looks like I struck a nerve Dano. Tell me, did it feel good to ridicule my facts? Did it help you deal with your denial of said facts?
Delightful.
Seixon, he's just trolling for a response and hasn't even tried to further the dialogue. His comments refute themselves, ignore him.
Comprendo, jet. I just got all weepy thinking about those puppies, poisoned by DU.
But I expend Tim's bandwidth for my amusement.
D
For me, Seixon's most interesting mistake relates to his idea that clusters should have been allocated to paired governorates one at a time. He rejects the Lancet method of using their populations to determine the probabilities in a once-off trial which gives the lot to one or the other. The question Seixon ought to have asked, but didn't, is how his "better" method affects the probability that any given household gets surveyed. Had he done so he would have discovered that the two methods give the same probability. Thus, if the Lancet's method is biased, so is his. But actually neither method is biased.
An interesting exercise is to take the binomial probabilities provided by Tim Lambert in comment number 60 (thereby avoiding the farting around with factorials and what-not). If the probability that Missan wins a single drawing is 1/3, then with Seixon's method:
Missan gets: 0 clusters (prob=8/27), 1 cluster (12/27), 2 clusters (6/27) and all 3 (1/27).
Using these, figure out the answer to the question Seixon ought to have asked, but didn't. Note that the probability that a household gets surveyed varies directly with the number of clusters. For practical purposes, the relationship is linear.
Before urging Seixon to write to the Lancet we should ensure that he is clear about all this. It would be embarassing if they granted him a public hearing and then floored him with the binomial theorem.
There is a major myth that Seixon is expounding here (with Jet as a hanger-on). This is the myth of western 'basic benevolence', whereby our top-down nominal democracies (in fact, plutocracies) only have good intentions and their sole wish is to spread democracy to impoverished masses around the world. This myth, endlessly promulgated by the corporate-state media, gives the ridiculous impression that the current US 'junta' is waging open-ended wars for democracy and that these wars have nothing to do with outright expansionism and economic expediency. The myth also suggests that Iraqi civilians 'matter' and that the Iraq war was waged in such a way as to minimize civilian casualties. If this was indeed the case, it was only because of the second great world power: public opinion, which has seen the results of US carnage and slaughter over the past century through direct military action or through the use of proxy forces (usually brutal regimes that support US foreign policies and its business agenda). Examples are endless - the massacre of as many as half a million Phillipinos when colonizing the country in 1901-02, when one American general famously stated that the US "Wished to create a howling wilderness" in the country; the Wilsonian agression against Haiti and Cuba, 1915-1920; Korea; Viet Nam; support for the abhorrent regimes of Suharto, Mubutu, Pinochet, Marcos, the Shah, Somosza, and others over much of the world (again, in support of a US corporate agenda) and the overthrow of many democratically elected nationalist governments (some 40 since WW II).
Another important tenet of this myth is that countries do not investigate their own crimes. We count our own dead but, because we view the others as "unpeople", or "collateral damage" that conveniently obstruct the true intent of our naked aggression, then our governments do everything they can to downplay the casualties of our actions. The bottom line is this: if we don't offically count the number of our victims, then the atrocity never happened. Seixon is using this vile tactic to its conclusion on this thread. In the immortal words of Tommy Franks, "We count every screwdriver but we just don't do body counts". The US/UK war party has, in fact, done just about everything conceivable to obstruct efforts by NGO's both inside and outside Iraq to assess the true death toll of civilians in Iraq. Iraq Body Count, in a press release early last year, strongly condemned the UK governmet which claimed in a completely disconnected statement to "regret the loss of civilian life" but to go on to state that they did not intend to census the number of civilians killed during the invasion. As IBC noted, "regret" over civilian casualties has nothing to do with the "intent" to count the victims of war. But the US/UK know that, so long as the number is unknown, then they can effectively ridicule any estimates that might cause public outrage. This is hardly a new strategy. How many victims were there of US aggression in Viet Nam and elsewhere in Indochina in the 1960's and 1970's? Estimates range from 2-4 million, but of course without a definitive total, the aggressors can downplay them. What about the Korean War? No attempt made to count victims there, either. What about the genocidal sanctions that are esitmated to have killed up to half a million Iraqi children between 1991-2003? I haven't seen any official attempt made by western governments to tally up this atrocity. Therefore, no 'official count' means it didn't happen. Thus, the number might be half a million, or ten thousand, or none.
Another one of the major parts of the myth, to quote Orwell's "Memory hole", is to enter a collective state of denial that a country which allegedly espouses principles of "democracy" and "freedom" is in reality a wholesale rogue state that is using a transparent 'war on terror' as a sematic camouflage for its real aims of global economic and miltary hegemony. Several historians have noted how, within a year of the US slaughter in the Philippines a century ago, the US government and media developed collective amnesia "almost overnight" and thus erased the episode from history. They could not perceive how a 'radically innocent' nation with traditions of 'freedom' and 'democracy' could engage in such a heinous crime. How many Americans, if polled today, knwo anything about this event? Before Seixon screams "Conspiracy! Conspiracy", I suggest he reads Paul Wolfowitz and Dick Cheney's 1992 piece, "Defense Planning Strategy" document, along with Zbignieuw Brezinski's 1998 "The Grand Chessboard" and, most famously , "Project for a New American Century", publsihed in 2000. In the original document, Wolfowitz, grand architect of the Iraq war, hardly veiled his intent that the US should wage open-ended wars around the globe in pursuit of its economic agenda. At the time, the media dismissed this as the rantings of fringe lunatics on the right, because the agenda was so clearly stated. However, since 9-11 the agenda has been repackaged under the 'war on terror' monicker, which the media have swallowed hook, line and sinker. Yet the agenda of hegemony remains. The fact is that this US regime is now polluted through the ranks by recycled Reagan neocons: Cheney, Rumsfeld, Fieth, Abrams, Libby, Bennett, Negroponte, Perle, Wolfowitz, Bolton and others who were routinely known as the 'crazies' in diplomatic circles during the 1990's because their views were so extreme. Their perpetual wars in support of blatantly obvious US economic and military goals should be clear to everyone, except perhaps those whose sole sources of information are CNN, Fox News, the Washington Post and New York Times.
The bottom line is this: human life, especially that of the poor in the developing world, means absolutely nothing to these people and the interests they represent. The Lancet study only highlighted waht we should have known all along: that our daisy cutters, cluster bombs, depleted uranium munitions and napalm do not discrimate but have slaughtered a hell of a lot of innocent civilians in Iraq, and that the only problem for the Bush-Cheney junta and their UK poodle is how to massage this fact so that the carnage is 'normalized' by our corporate media. The current regime, sadly, spits on international law and on the US constitution, and, as Gore Vidal suggested, makes the US in reality "The United States of Amnesia".
*[Deleted. See comment policy. Tim]*
Sexion, you say:
> For your information, it was actually medical doctors who carried out the survey. You know, the ones in the field? Roberts and all the rest didn't carry it out, they designed it. If you are going to keep flogging one simple and irrelevant thing such as that, and keep flogging the "they are experts, so shut up" rhetoric, then just save it.
where are you getting your information from? Are you as sure of this as you were that Roberts et al had no clue about statistical design because they were mere physicians?
Now and then Seixon gets something right. On page 2 of the study it says that 5 of the 6 Iraqi interviewers were medical doctors.
Darn Kevin, you spoil my fun. Oh well, did you get to page 8
>L Roberts was the lead investigator in the field and was principally responsible for the data analysis, interpretation, and preparation of this report. R Lafta was involved in study design, hired, trained, and oversaw the interview staff, led one of the two study teams, coordinated all logistical aspects of the study, and had a central role in data interpretation and preparation of this report. R Garfield advised on issues of study design, study execution, participated in the analysis and interpretation of data and preparation of this report, and initially organised the study team. J Khudhairi was involved in the study design, interviewer training, and oversaw one of the two survey teams in the field. GBurnham advised on issues of study design, study execution, participated in the analysis and interpretation of data and preparation of this report, and organised and facilitated the ethics review process at Johns Hopkins University.
Esp. the part
>L Roberts was the lead investigator in the FIELD.
Kind of leaves Seixon's diss of our Les in the dirt. Actually, although it is probably impossible on the net it would be interesting to know if the Iraqi's had public health and survey design experience. I would not be surprised.
Kevin,
You obviously didn't even read my latest post about the probability distributions. With 3 clusters to be distributed, Missan's households would have a 71.2% of being sampled with simple random sampling. With the Lancet method, this probability was only 34%.
Ooops. The only way the Lancet method and the standard method are the same is if there is only 1 cluster to be distributed, which is what Detached Observer has been clinging to like a drop from a leaf. Unfortunately, the number of clusters to be distributed in this case were 3 for 5 of the pairs, 4 in the last pair.
I have given all these probabilities in my latest post, which shows that the Lancet method made it much more improbable that the provinces' households would be sampled compared to the standard method.
Detached Observer, and Kevin, have completely ignored this and sought to keep being in denial about this simple fact.
Eli,
I got snagged in thinking that the interviewers were those 6 Iraqis that were spoken about, 5 of which were medical doctors. Thus why I said exactly "medical doctors" in my comment about that. I admitted my mistake, so shall we move on?
Jeff,
Your venomous post doesn't really deserve any rebuttal, but I took most offense to this from skimming through it:
"The bottom line is this: if we don't offically count the number of our victims, then the atrocity never happened. Seixon is using this vile tactic to its conclusion on this thread."
WTF? I have pointed to the UNDP study, and the IBC tally, and said that based on these much more credible surveys, taking into consideration the passage of time since the UNDP study, and numbers coming from the Iraqi government, I deduce that the correct death tally, for all Iraqis including combatants, to be somewhere around 35,000-50,000.
Am I thereby claiming the atrocity never happened? Of course not. However, dishonest smear-mongers like you will pretend that the USA killed every single one of those people, and you will negate the fact that the majority of them were in fact combatants and Saddam's goons. Insurgents have killed 12,000 civilians over a 17-18 month period in Iraq.
The real atrocity here is people like you who try to escape such facts and instead focus entirely on whatever the USA has done and does. You also take the PNAC, and mostly everything else, completely out of context.
You don't seem to understand the most fundamental pillar of the PNAC and similar movements: freedom and democracy promote free trade and economic well-being which only benefits the USA more and more in their scouting for new markets to sell to.
If you don't understand that, and that most African countries are asking for trade, not aid, then you don't understand a damn thing.
Seixon,
I read your post entitled "So I'm Not A Statistics Professor." Frankly, you're not even a statistics student, although I see from the comments that Detached Observer is still trying to educate you. It reminds me of something a lecturer once told me: nobody teaches anything; some people learn, some don't. My comment number 138 was not really meant for you, it was for anyone who is genuinely interested in figuring out where you went wrong. Clearly you are not remotely interested in doing that. But if ever you should develop an interest, the question I posed would make a good starting-point.
With 3 clusters to be distributed, Missan's households would have a 71.2% of being sampled with simple random sampling.
Detached Observer asked you to clarify this claim on your blog. He reckons you are (a) computing the probability that Missan gets at least one cluster if your one-trial-per-cluster method is used (with p=1/3) and (b) getting the answer wrong. Well at least you are consistent. This time you're only slightly out. (Hint: the easy way to do it is to calculate the probability that Missan gets zero clusters and subtract the answer from one.) Of course correcting your arithmetic won't advance the discussion. The probability that Missan gets at least one cluster is not what you need to think about.
>Am I thereby claiming the atrocity never happened? Of course not. However, dishonest smear-mongers like you will pretend that the USA killed every single one of those people, and you will negate the fact that the majority of them were in fact combatants and Saddam's goons. Insurgents have killed 12,000 civilians over a 17-18 month period in Iraq.
Seixon - and how many would they have killed if the US hadn't invaded?
For those of us still mired in the old-fashioned reality-based world-view, peopel are responsible for the reasonably foreseeable consequences of their actions.
Ever read George Bush's explanation of why he didn't order a full-scale occupation of Iraq back in 1991? something about a civil war if I recall correctly...
>freedom and democracy promote free trade and economic well-being which only benefits the USA more and more in their scouting for new markets to sell to.
Yeah that's why the PNAC's predecessors were such big supporters of Pinochet's Chile and Apartheid-era South Africa - and why they currently support dictatorships such as Kazakhstan.
Kevin,
Now you have fallen all over yourself. Correct my arithmetic? Go ahead. You would only be correcting the Oracle of Tim Lambert by doing so, with the slight difference that Lambert used p=1/3 when it should have been 0.34.
Your reference to (a) "your one-trial-per-cluster method is used" is quite ironic in that you say that I'm not even good enough to qualify as a statistics student. My method? I think they call it simple random sampling. But hey, what do I know? I'm just some dumbass who got an A in college statistics.
Let's go over this real slow, since you seem to think I, and by extension Tim Lambert, have fouled up our arithmetic.
Independent probability that Missan doesn't get a cluster: 0.66
Now, calculating the probability that Missan will end up with zero clusters, and thereby zero households sampled, in distributing the 3 clusters it shares with Basrah...
Lancet method: 0.66
SRS: 0.66*0.66*0.66 = 0.287
Due to Lancet's methodology, instead of using SRS for all 3 clusters which is supposed to be used throughout (multistage) cluster sampling, Missan now has a 37.25% higher chance that it will not be sampled at all.
Agree, or disagree?
Feel free to correct Mr. Lambert's... ehm, I mean my arithmetic. Funny how I got those confused, must be because they are so similar.
Ian,
How many Saddam's goons would have killed if we hadn't invaded? I don't know, how far into the future are you willing to compare the developing situation with what would have been if the US didn't invade?
So in other words, people of the Middle East do not deserve a shot at democracy because there are goon squads in each of those countries that would kill thousands of people if the US got rid of their masters to let these people have a shot at freedom? There's a word I am looking for, starts with C, ends in apitulation.
In fact I haven't read Bush Sr.'s musings or opinions of what might have been in Iraq. I really don't give a cute puppy's ass what he might have to say about the issue. Do you see Bush Sr. as some kind of foreign policy expert? A Middle East expert?
Now, what you might be forgetting is that Iraq had serious tonnage of chemical weapons in 1991. Taking that single fact into consideration, do you think it would have been wise to try and unseat Saddam Hussein at that point in time? Or would it be wiser to submit the bastard to inspections and a cease-fire agreement to remove all such weapons before eventually removing the bastard when, by anti-Bush experts' pre-war assessment, he would only have 10% left of them?
I hear you talking about civil war. It's been 2 years now, no civil war yet. The Sunnis are starting to see the light and going into politics, although there are still some hold-outs. Zarqawi is getting desperate now and has conspicously called for civil war against the Shiites. Unfortunately, not enough Sunnis want a civil war. Too bad, so sad.
True, PNAC's predecessors were screw-ups. Keyword: predecessors. Vietnam was the biggest boondoggle ever. So were all the dictatorships they played dominoes with in South America, Iran, Afghanistan, among others. Do you know why they were screw-ups? They screwed up because they didn't ensure that democracy and freedom took hold after they knocked the dominoes down. They didn't do what we are doing now in Afghanistan and Iraq.
What did we do in Iran so long ago? We knocked off one guy, and let some other "wonderful" character take his place. We didn't put troops on the ground, destroy the seeds of evil, and ensure democracy got a chance. It was a half-ass job, the easy way out. It FAILED. Miserably.
We now seek to amend those mistakes, and are well on our way in doing so in Iraq and Afghanistan.
True, we are "friends" with various forms of dictatorships all around the world. Saudi Arabia, Pakistan, you name it. What other alternative do we have though? Should we become fiends with Saudi Arabia? Pakistan? The rest? Would that be a brilliant idea?
Right now we are fighting al Qaeda alongside the dictatorship in Saudi Arabia, because we have a common enemy. Dismantling the alliance with Saudi Arabia would just put us at much more risk. We cannot pressure Saudi Arabia into much at all, seeing as how they have vast amounts of oil that we import from them daily, and they are the proprietors of the Holy Land for Muslims. Invade Saudi Arabia? Surely that's not what you would advise. "Crusade" would take on much more credibility if we invaded Saudi Arabia rather than being in Iraq.
We are stuck in between a rock and a hard place. Therefore, we seek to make Iraq free and democratic, boost its oil production, start buying more oil from them, become less dependent on Saudi Arabia, then we can finally start putting the heat on them to change their ways.
Mk?
I know chanting "no war" is fun and all, but the grown ups have business to attend to, and the world's future to think about.
Yet if you want us to drop the alliance with Saudi Arabia, Pakistan, among others, just say the word. Let's see how much deeper in the hole you can get us with your naive thought process.
>How many Saddam's goons would have killed if we hadn't invaded? I don't know, how far into the future are you willing to compare the developing situation with what would have been if the US didn't invade?
The Baathists would probably have continued to kill several thousand people per year.
Had Saddam and/or his sons remained in power for another 20 years, the likely death toll would be less than your own estimate of 50,000+ deaths resulting from the invasion.
Ever study the concepts of net present value and discounting (in the economic sense)? How about opportunity cost?
>I hear you talking about civil war. It's been 2 years now, no civil war yet.
Read the actual news from Iraq and track the daily reports of dozens of bodies turning up as a result of sectariuan revenge killings- the Sunni and the Shia are engaging in a low-level civil war already.
Or do you think the regular reports of Sunni clerics and academics being arrested by people in police uniforms and driving police vehicles and being found the next day dead in ditch are all the work of the insurgents?
Hey, maybe it's that damn left-wing journalist conspiracy at work again.
Maybe Iraq is actually an Earthly paradise.
Why don;t you go there and check.
Stop in on some of the people who've lost family members or whose kids had limbs blown off by allied munitions and tell them how grateful they should be.
>Do you see Bush Sr. as some kind of foreign policy expert? A Middle East expert?
Well let's see - former head of the CIA; President at the time of the collapse of the Soviet Union, put together the coalition that expelled Saddam from kuwait.
Yes, I'd say he qualifies as a foreign affairs expert.
Seixon,
If you are using p = 0.34 then your arithmetic is accurate in this instance. I assumed you had stuck with 1/3 to simplify the discussion. You ask me to agree that 71.25% is "37.25% higher" than 34.0%. The best I can say to that is that I know what you mean and your subtraction is correct. I refer to "your one-trial-per-cluster method" because nobody else I know would try to eliminate travel between Basrah and Missan using a method which is unlikely to do so. But since you do agree that such a method is unbiased it can be used a starting-point for the demonstration that the Lancet method is also. As several people have explained, your claim to the contrary is simply false. You have not withdrawn it as any honest blogger should.
If you got an A in college statistics then you must have forgotten a great deal, including the definition of bias, a term which you use in a wildly inappropriate way. If you can't bring yourself to retract your assertion you should at least make it clear that you are assigning a meaning of your very own to the word.
"Had Saddam and/or his sons remained in power for another 20 years, the likely death toll would be less than your own estimate of 50,000+ deaths resulting from the invasion."
It's easy to claim that Saddam and his sons wouldn't have killed that many in 20 years. I guess the freedom of speech, press, religion, all that good stuff, that doesn't matter. Right?
"Read the actual news from Iraq and track the daily reports of dozens of bodies turning up as a result of sectariuan revenge killings- the Sunni and the Shia are engaging in a low-level civil war already.
Or do you think the regular reports of Sunni clerics and academics being arrested by people in police uniforms and driving police vehicles and being found the next day dead in ditch are all the work of the insurgents?"
I do read the news. Yes, as I said, there are still hold-outs among the Sunni who don't want to let go of their old glory. I see a civil war as a war where two large factions in a country go against each other and fight a real war. Is that happening in Iraq? No. You weaseled your way out of it by saying low-level civil war.
In reality, it is just a few thousand Sunni thugs who are going around killing people, including Sunnis who are "infidels" for joining the political process. Sure, some Sunnis might be killed by Shia, but CIVIL WAR? Give me a break.
"Hey, maybe it's that damn left-wing journalist conspiracy at work again.
Maybe Iraq is actually an Earthly paradise.
Why don't you go there and check.
Stop in on some of the people who've lost family members or whose kids had limbs blown off by allied munitions and tell them how grateful they should be."
For the last time, it is not a conspiracy that the majority of journalists are liberal-leaning. It is an established fact that the majority of the US journalists vote Democratic. Is that a conspiracy? No, it is just the truth. Deal with it and join us in the real world.
As for your laughable ridicule, I have never claimed that Iraq was a paradise, so trying to attribute that to me is petty and cheap.
Yes, many people have died and hurt. I guess that means we should have stayed out of Korea, Germany, France, Japan, and the list goes on and on. Pacifism gets you no where when there are tyrants out there willing to kill you in the blink of an eye.
Ian,
I most specifically asked if he was a Middle East expert. I think you also didn't take the hint from what I said, that Bush may have said one thing back then to mask the real truth behind the plan to take out Saddam Hussein. Bush Sr. would have seemed a fool to advocate toppling Saddam while he was sitting on tons and tons of chemical weapons.
Kevin,
"I refer to "your one-trial-per-cluster method" because nobody else I know would try to eliminate travel between Basrah and Missan using a method which is unlikely to do so."
Aha. Why was travel to be eliminated between those two? Why not Anbar and Baghdad? Upon which basis was it legitimate to exclude one of them, and oversample the other with their combined clusters?
The answers are bias, and nothing.
I know full well what bias means, and picking out some provinces arbitrarily and then treating them to an unsupported sampling method that creates an inequality from the other provinces, that is what bias is all about.
Making convenient arbitrary decisions that alter the sample is what bias is all about.
I'm being honest here, you're not. Hello, you and Detached Observer claimed my numbers were wrong. LOL
Oh, and I have set Detached Observer to another task that will once again prove to be detrimental to the Lancet study:
Can you produce the normal distribution of probabilities for the value of the number of deaths found by the survey?
Assuming 98,000 is the mean, the range is 8,000-194,000, please come up with different values for the probability that the actual number is 98,000 using different standard deviations (since the Lancet study doesn't give us what it is).
Can ya do that? Thanks. I tried it over at my blog, but I don't think it came out right. Here's your big chance to correct me again. In fact, I am asking you to do so.
Seixon: "I know full well what bias means...."
Sorry, you don't. That's the problem. And it's not up to the rest of us to solve it.
Ian Gould,
The <50,000 is unfair and unproveable. In just 2 years when the Iraq-Iran war turned chemical, Saddam killed ~100,000 civilians and soldiers just with chemical weapons. Sanctions were on their way out and we don't know what Saddam or hid kids might have done.
While we have no way of knowing how many he would have killed. We do know how many gold plated palaces he built while his infant mortality rate went through the roof and malnorishment was widespread.
Don't take me wrong, when the Shi'ites revolted in 92, the US didn't back them because they were Iranian backed. And those same Iranian backed Shi'ites the US didn't like now control the Ministry of Interior, have the second largest militia (besides the Kurds), and are conducting a CIA/KGB style war with the Ministry of Defense. It is very conceivable that the US will be forced to back the Sunni's and Al-Sadr to counterbalance their bid for control. I'd have preferred the money spent on the war be spent of alternatives to oil making the middle-east not so geo-poltically important. Without all that oil money Saddam couldn't have fought his wars, nor could he have afforded the military and police to put down the majority population of Shi'ites.
In case anyone might think that there is any merit at all in Seixon's case, let's look at the question he resolutely refuses to address: if clusters are allocated to Missan in the manner he proposes, what is the probability that any particular Missan household will be surveyed?
Using the study's assumption that every household has seven individuals, Missan has 97,857 households. (The exact number doesn't affect the result of the calculation.) In Seixon's lottery Missan has a 34% chance of winning each one of 3 clusters. The number of clusters Missan wins may be 0, 1, 2, 3; with probabilities, given be the binomial distribution, of 0.2875, 0.4443, 0.2289 and 0.0393 respectively. Our chosen household can get a visit if any one of three events transpire:
Missan wins just one cluster in Seixen's lottery. One household in every 3,262 is surveyed and our household is one.
Probability of this event: 0.4443/3,262 = 0.01362%.
Missan wins two clusters in Seixen's lottery. One household in every 1,631 is surveyed and our household is one.
Probability of this event: 0.2289/1,631 = 0.01403%.
Missan wins three clusters in Seixen's lottery. One household in every 1,087 is surveyed and our household is one.
Probability of this event: 0.0393/1,087 = 0.00362%.
Since these events are mutually exclusive, the probability that any one of them transpires is the sum of the individual probabilities: 0.0313%.
Using the Lancet's method, there is a 0.34 probability that Missan wins three clusters so that one household in every 1,087 is surveyed. The probability that our household is one of them is: 0.34/1,087 = 0.0313%.
The two methods give us exactly the same answer. This is a consequence of a well-known property of the binomial distribution. An A-student like Seixon ought to know it. I certainly don't blame his teacher, who must have suffered enough. But whoever marked the exam should be shot.
Jet, it's true Saddam had access to billions which he spent on palaces. He's not a nice guy. But even if he'd spent his billions it wouldn't have made up for more than a portion of the damage done by sanctions, which according to a NYT editorial I remember reading, cut Iraq's GDP by a factor of between four and five. Though again I don't want to deny he could have reduced the damage somewhat if he'd wanted to.
I meant to say this before and hit the submit comment without thinking--What's happening in Iraq now is basically a role-reversal. Previously it was the US that deliberately destroyed the civilian infrastructure and tried to prevent its repair, and Saddam's corruption added to the problem. Now it's the insurgents (or some of them) who destroy the infrastructure (though some was probably hit during the invasion) and the corruption is at our end.
As I understand Seixon on one of his points (and if I didn't understand Seixon I think this is still a legitimate point), he isn't just saying that the probability of picking one of a chosen pair of clusters was biased, but, also, that the pairing was biased. There is nothing wrong, in a statistical sense, of pairing clusters (except that the more clusters are paired, the greater the uncertainty of the results) but it's important to pair them so that the resulting population is homogeneous. What method did the authors use to determine the homogeneous character of the pairs? They don't say. Reducing total miles traveled was the reason given for combining govenorates, and that goal is achieved, but the practical effect is that the pairs chosen ended up with inhomogenous populations. Why is that? Because distant regions were largely untouched by war and were matched with more proximate regions (deemed to have similar levels of violence). But, removing the most untouched governorates from the population automatically skews the results toward the remaining population affected by war. While the choice of which half of a chosen pair of governorates was random, the final result excludes half of the untouched governorates (with p=.5). The variance is naturally increased by this pairing, but the increased error introduced by a faulty choice of pairs is not measurable.
Further, I think the use of cluster sampling for a population for which the parameter of interest is most effected by outliers (bombing) is probably not ideal.
None of these problems put the study in such a quandry that it shouldn't have been reported. But the results should be taken with a reasonable amount of uncertainty. Likewise, a defense of the study should match that uncertainty. A vigorous defense should be reserved for a robust study.
A vigorous defense should be reserved for a robust study.
I would say rather that if a study is flawed, all the more reason to concentrate on the flaws. What Seixon has done is to push a critique which is plainly silly. If I had serious reservations about the study, whether because of some worry about using GPS to construct grids or some number-crunching problem, it would be even more annoying to have somebody butting in with an argument which can easily be refuted by referring to an introductory textbook.
Of course if the excluded areas had very different mortality rates then that's a serious problem. That kind of thing has been discussed at length in previous threads, with Heiko Gerhauser making the anti-Lancet case. But even if true it does not imply sampling bias, which is Seixon's claim. As he himself insists, his is an entirely original critique. It is also entirely wrong.
Kevin,
You, like Detached Observer, are operating with a given to make your argument sound correct. You were operating with the given that Missan did in fact win one or more clusters. Yes, in that case, the probability of a household being in the sample is the same whether you use SRS, or as you like to call it "Seixon's lottery", or the Lancet method.
Just one little fact that you disingenuously left out: the probability of Missan to receive zero clusters, and therefore not have any households sampled.
With "Seixon's lottery" it is 28.75%.
With Lancet's lottery it is 64%.
Ooops.
Also,
"Of course if the excluded areas had very different mortality rates then that's a serious problem."
Well, you just don't have any way of knowing this, now do you? The brilliant Lancet study authors have removed this from even being a factor, because you are supposed to believe their "belief" that the areas had similar violence.
My claim of bias is wrong? So the probability of NOT being sampled didn't increase substantially with the Lancet method? Can you give me an example of any statistical sampling literature, or any study done, that used clumping of clusters? Can you tell me on what basis they paired up provinces? If there was none, doesn't this represent a bias that would perhaps have oversampled more violent areas? In fact, didn't they introduce an unknown and ambiguous bias by doing things in this manner?
Start answering the questions buddy. I am also really getting tired of you lying with your statistics. You are cherry-picking like a dishonest hack.
Just one little fact that you disingenuously left out: the probability of Missan to receive zero clusters, and therefore not have any households sampled.
Seixon,
As any attentive reader will have noticed, I was calculating the probability that any particular Missan household will be surveyed. You wonder how an attentive reader would have noticed this? Well, because I used the words "the probability that any particular Missan household will be surveyed" - tricky, huh?
Stick at it, you will get the knack eventually. What you have to do is read the words, having regard to the order in which they are arranged into sentences. Of course it helps if you know their meanings, too.
Now, in that context - follow me closely here - one can make use of the fact that zero probabilities add nothing, absolutely nothing, not even the most insignificant quantity, to the sum being calculated. You can see why, can't you? As an A-student, you will surely have noticed that the equation:
x + 0 = x
holds for all x, whether x is an integer, or rational, or real, or even complex.
Identifying the other silly bits in this Seixon comment is left as an exercise for the reader. That goes for his future comments too, since there comes a time when expressing one's honest opinion of a commenter conflicts with the posting rules.
Kevin, you disingenuously left out the role of punctuation in language. Dishonest hacks leave out this important tidbit.
I was going to go to a movie tonite, but instead I think I'll save the ten-spot and read this thread instead.
D
Eudoxis, I'm not one of the experts here, but I've been wondering about the variance myself. I understand that pairing provinces and choosing one of them to sample doesn't bias the expected value of the result for the death rate if you do it the way the Lancet authors did, and I also understand that the variance is increased if the provinces have different death rates, but what I don't know is whether you can calculate the variance if you don't know what's happening in the unsampled provinces. In the simple-minded example I did much earlier in the thread, I could have calculated the variance (which I did later) but only by knowing what the death rates were in all the provinces. Maybe I'm not thinking about this the right way. Their discussion concerning the confidence interval contains jargon which went over my head.
Now if the Lancet team did pair distant peaceful provinces with nearby war-torn provinces, it causes an increase in variance, but it still doesn't bias the estimate, because those nearby provinces would have a chance of being the ones excluded. (Which I guess would also mean they'd have to drive all the way up to Kurdistan anyway.) I don't remember if the Lancet team did this and my copy of the paper is at home.
It'd be sensible to complain that by the luck of the dice, the Lancet team happened to end up sampling the bloodiest regions--whether that's true or not I don't know. Of course they actually threw out the bloodiest sample they had.
But it's not sensible for Seixon to keep making claims involving the statistical definition of bias which are false. I think it's become a matter of pride.
And for myself, I don't blame them, but in hindsight I wish the Lancet team had done things differently. In particular, if they could have done so without getting killed, it would have been very useful if they'd done even one more sample in Fallujah, even if they kept that extra one out of their 33 cluster analysis. (And of course as it happened they threw out the one they did do.) Les Roberts or one of the authors said something like this in an interview I read somewhere.
My impression is that there is a considerable difference in death rates between the sampled regions. That means the clusters from two neighboring regions are not readily interchangeable. The choice about which regions to combine was based on an assumption about levels of violence in those regions. This introduces a selection bias and there really isn't a way to calculate the error associated with that unless the study could be repeated (best with different clusters). Of course, time has passed and this study was dependent on memory at the time of the study and can't be repeated.
Bear in mind, that this critique is only a potential flaw. If this issue has been discussed on another thread or other blogs, I suspect it has already received more than enough focus. And, yes, Seixon's complaints are largely baseless.
The general underlying weakness of using this sampling method in a war torn area with localized increases in violence due to bombing, or of the dependence on memory to collect a detailed history are not included in the variance of the reported results. Only in the case of specific infant mortality recall bias, where the suspected overestimate was reduced by 2, is there a value placed on the possible bias or problem with the collected data. The authors of the study readily acknowledge these kinds of difficulties in their discussion.
Assuming a potential under reporting of deaths in this study, we can still rely on some intuition of war in general and a particular knowledge of a country devastated by sanctions, misgovernment and drought, destruction of infrastructure, followed by a violent invasion, and we can have little doubt about the large numbers of deaths in Iraq in the last few years.
Jet wrote: Ian Gould, The <50,000 is unfair and unproveable.
I think that should be a greater than symbol. The estimate of circa 50,000 deatsh since the invasion is Seixon's. If you think it's unfair - argue with him.
>I guess the freedom of speech, press, religion, all that good stuff, that doesn't matter. Right?
you go on to insist that you are too reading the news from Iraq so I'll assume that you are being wilfully disingenuous when you ignore the numerous rep rots that freedom of religion and women's rights are in an even worse state now than they were under Saddam.
As for freedom of the press - I assume you also choose to ignore the regular reports of journalists being tortured and murdered.
On the question of civil war: I'm not "weaselling" out of anything. Sunnis and Shi'ite are being killed for purely sectarian reasons on a daily basis. The death-toll from these killings is substantially higher than the 2-3,000 people per year Saddam was killing.
Spare me your selective outrage.
>In reality, it is just a few thousand Sunni thugs who are going around killing people, including Sunnis who are "infidels" for joining the political process.
Our realities must be different - in mine, as I specifically mentioned, there are numerous credible reports of random abduction, torture and murder of Sunnis by members of the shia-dominated military.
I have never claimed that Iraq was a paradise, so trying to attribute that to me is petty and cheap.
Yes and I'm sure you;d never sink to such depths.
>I most specifically asked if he was a Middle East expert.
After you asked if he was a "foreign affairs expert".
Do you really want ot reel off his qualifications to speech with authority on middle east affairs? (we can start with the fact that he was heavily involved in the Reagan administration's relations with Iraq in the 1980s.)
>Bush may have said one thing back then to mask the real truth behind the plan to take out Saddam Hussein. Bush Sr. would have seemed a fool to advocate toppling Saddam while he was sitting on tons and tons of chemical weapons.
Actually he said that in the mid-1990's in an essay for time magazine.
A correction ot my last post - it was the mid-90's not the mid-80s.
Ian Gould,
I'm pretty sure I was responding to you.
"Had Saddam and/or his sons remained in power for another 20 years, the likely death toll would be less than your own estimate of 50,000+ deaths resulting from the invasion."
Kevin,
So the probability that Missan doesn't get sampled doesn't matter at all? That this probability is completely different between the two methods doesn't matter?
Lancet: 64% chance that Missan will not be sampled
SRS: 28.75% chance that Missan will not be sampled
You are trying to get around this inconvenient fact by focusing on expected values. Yes, in the event that Missan gets around the fact that I just showed you, then the probability a household will be sampled is the same. However, the very fact that the chance they won't be sampled at all is much higher with Lancet's method blows over everything else you have to say. Which is why you continue to zoom right past it.
eudoxis,
First you say:
"My impression is that there is a considerable difference in death rates between the sampled regions. That means the clusters from two neighboring regions are not readily interchangeable. The choice about which regions to combine was based on an assumption about levels of violence in those regions. This introduces a selection bias and there really isn't a way to calculate the error associated with that unless the study could be repeated (best with different clusters)."
Then you say:
"And, yes, Seixon's complaints are largely baseless."
You attest to the validity of my main point, and then say that my complaints are largely baseless. How does that work?
My main complaint comprises of these main points:
They only chose 12 out of the 18 provinces to be paired. Is this or is this not a bias?
They paired up those 12 provinces based on nonexistant rationales. Is this or is this not a bias?
After pairing, they distributed the clusters not via SRS (as they should, and as is the default methodology in everything I have found), but with one trial that is biased towards the more populous of the two in the pair (since it gets all their clusters). Is this or is this not a bias?
Someone correct me if I'm wrong, but I think Seixon may be off on a few things, but I do think he's managed to put the Lancet study in extreme doubt. When the sample size is small, the CI is huge, and the CI almost includes zero, then the study is just barely significant.
This whole thing hinges upon, did the CI include calculations for the arbitrary grouping of provinces? If not then it is safe to say the real CI should include zero, making the study completely unreliable.
Seixon has done nothing to put the study in doubt. He's ululated so loudly your eardrums are ringing, hence your apprehension.
This issue in Seixon's mind, from what I've been able to unpack from the comedy, is that he doesn't like how the numbers were presented.
Other folk trying to quibble with the study don't like how the numbers were 'spun' either, but have been, in my mind, unable to unspin the numbers.
Thus Tim's characterization: flypaper for innumerates.
Innumerates don't understand that the numbers have a large range of uncertainty (well, they trumpet that phrase, but don't know what to do with it).
Numerates look at the study and say to themselves 'that's a wide range' and then maybe think 'I wonder what another study will show' in response to the human brain's need for one number to characterize (characterise) something.
HTH,
D
Ian,
Freedom of religion is worse off than with Saddam? So Saddam banning them from religious rituals and pilgrimages and all of that, which they are now allowed to do as they want, that doesn't mean anything? How has freedom of religion gotten worse?
As for women's rights, you seem to think that women had more rights under Saddam. Why? Those Iraqis who adhere to their religion on keeping women suppressed existed in Iraq before Saddam fell as well. Do you think that Saddam went in and forced them to do anything about that? No.
"As for freedom of the press - I assume you also choose to ignore the regular reports of journalists being tortured and murdered."
Heh. Yes, why don't we just always pull out the anecdotes of thugs trying to stop the progress that is in process. Right? I mean, here's a zinger for you: those journalists weren't being journalists under Saddam. There weren't hundreds of newspapers under Saddam. Yes, journalists are getting threatened by Saddam's thugs and murderers. What is your point? Next you're going to tell me that rain is water, and milk comes from cows.
"On the question of civil war: I'm not "weaselling" out of anything. Sunnis and Shi'ite are being killed for purely sectarian reasons on a daily basis. The death-toll from these killings is substantially higher than the 2-3,000 people per year Saddam was killing."
You couldn't claim there was a civil war, so you had to use "low-level" to get it in there. Yes, Sunni gangs and remnants of Saddam's clique are murdering hundreds of Shi'ites per month. Did you expect them not to? They had power over the rest of Iraq under Saddam, now they don't. It's only natural that they would fight to get back to their old glory, and they will keep doing so until they realize it is all for naught. al-Qaeda is trying, and failing, to start a civil war in Iraq.
"Our realities must be different - in mine, as I specifically mentioned, there are numerous credible reports of random abduction, torture and murder of Sunnis by members of the shia-dominated military."
Were those Sunnis also perhaps insurgents? Is it speculation on who killed them, or is there facts that show that the military did it? There's lots of questions and answers needed here, and you seem content to offer up speculation as an answer.
The fact remains that there is currently no civil war in Iraq, just a flailing insurgency trying as they might to stop progress in Iraq because they want their former glory back.
"True it's a conspiracy theory that anything and everything you read in the press that displeases you is a result of deliberate lies by the press."
Heh. Many times it actually is, as has been proven many times. So no, it is not a conspiracy or a conspiracy theory. That the majority of journalists are against the war is not a conspiracy any more than it is a conspiracy that 48% of the USA voted for John Kerry.
Journalists misuse their power of the megaphone to set their own agendas in through news. Through the power of omission, they can warp just about anything, just like Michael Moore. Does that mean all journalists do this? Of course not.
"After you asked if he was a "foreign affairs expert"."
Yes, it's called clarification.
"Actually he said that in the mid-1990's in an essay for time magazine."
It doesn't matter when he wrote it. He has to forever protect his reasoning for not going on to take out Saddam, and I think it is very odd that Saddam's possession of tons and tons of chemical weapons weren't a more prominent reason than what he said. Bush Sr, simply didn't want to go beyond the UN mandate, and he didn't want to risk sending American troops into the chemical weapons capital of the world at the time. Talking about civil war and a bunch of other stuff is simply concealing these very facts.
In fact, Bush Sr. advocated for the Iraqis to overthrow Saddam, effectively advocating a civil war. Saddam nailed them down, of course. So Bush Sr. claims he didn't take out Saddam because he didn't want a civil war to happen? So why did he advocate one then, albeit without a US presence?
Yes, I think you see how shallow that explanation really is.
Dano,
You haven't contributed to this discussion one bit as your only task has been to try and ridicule me. You are just a partisan cheerleader without any knowledge about anything being discussed here. So how about you let the adults sort things out, OK?
I think it is quite significant that Mr. Lambert claimed that clumping clusters constituted "multistage clustering" which is completely false on its face. His continued silence on the issue is telling.
Kevin's continued avoidance of the fact that the Lancet method made it much more likely for Missan not to be sampled is also quite alarming.
BruceR has disappeared after instead of disproving one of my findings, he strengthened it. Five out of the six pairings were fraudulent if we use coalition death rates as an indicator. Seeing as how the JHU team used absolutely no indicator what so ever for the pairings, that should also be ringing some alarm bells.
That the JHU team arbitrarily picked some provinces to be eligible for possible exclusion, while leaving others unscathed, should also be alarming.
That the JHU did not use SRS (obviously since they were trying to exclude provinces) in distributing clusters in the 2nd stage should also be alarming.
The fact that clumping clusters arbitrarily is an unsupported statistical method should also be alarming.
Yet none of this alarms some of you. Hmmm. I think I know why!
Someone asked me on my blog what the probability of 98,000 being the actual figure would be, and asked me if it really was the "most likely" figure. I told him that, yes, it is the "most likely" figure based on a normal distribution of probabilities. He asked me if it was around 50%. Well, no. Not even close. I have challenged Detached Observer to come up with the figure, and I am still waiting.
It might have something to do with the fact that with such a gigantic range, the standard deviation is huge, and the normal distribution of the probabilities tops out, according to my quick and rough calculation, to around 0.000013. In other words, 0.0013%.
So yes, 98,000 is the "most likely". Unfortunately, like many Lancet-apologists continually do, this only tells half the truth.
jet: This whole thing hinges upon, did the CI include calculations for the arbitrary grouping of provinces?
The short answer is, yes it did. This issue was thrashed out at great length in Tim Lambert's previous threads on this issue, some of which were very educational. It would be hard to find anything new to say about the study at this stage. Seixon has managed to come up with an original angle, but only by ignoring definitions and rules which are drummed into every statistics student in an introductory course.
This debate has been sidetracked because Seixen has used the wrong words to make his point. His detractors are concentrating on his incorrect wording while Seixen is concentrating on his point.
Seixen now admits that what he is worried about is some sort of biased picking of pairs -- which is not sampling bias. However, I think he somewhat undermines himself when he says...
They paired up those 12 provinces based on nonexistant rationales. Is this or is this not a bias?
If the rationales are truly non-existent, then the pairing would be totally random and so no -- there is no sampling bias.
However, more to the point I think what Seixen is trying to say is that (while there may not have been any sampling bias in the study) the outcome was "biased" by an unfortunate roll of the dice. I think he means to say that the results were not representative of reality.
An analogy. If you flip a fair coin 5 times and get five heads... then (1) there was no bias in the flip; but (2) the outcome certainly couldn't be taken as normal. You could see how somebody might easily refer to the outcome as biased. His language isn't quite right... but his point is obvious enough.
Good grief, Seixon. Yes, 98,000 is very unlikely to be the exact death toll. So is 98,001. So is 98,002. So is ...
(Several hours later)... So is 109,343. So is 109,344. I'm getting tired. Let's start counting down from the mean. So is 97,999...
Anyway, Seixon, if you want to make a case that by chance or maniacally evil liberal plotting, the Lancet authors sampled exceptionally bloody places, then that's not an intrinsically silly argument (whether it's correct is another story and I don't know). But this new attack is, well, remarkable.
On the confidence interval issue, I don't quite get how you get one not knowing anything about the unsampled provinces. One might be a place where everyone died. Another might be a place where people actually rose from the dead in order to shower the coalition forces with flowers and candy. I'm guessing there's an assumption present that the unsampled provinces have the same or similar range of death rates as the sampled ones, thereby ruling out genocide or mass resurrections, but that's a guess and beyond the elementary level my statistical knowledge is pretty meager. If I sample the Lancet paper based on what I do know, they seem like smart guys and as Jet or someone said earlier, presumably this pairing process played a role in making the CI so big.
Kevin,
Yes continue ignoring the fallacies of your obfuscation and keep drumming me up as a complete moron.
What's that? Lancet's method gave Missan a higher chance of not being sampled than SRS?
Can you find any statistical theory that allows for the clumping of clusters, any previous study that used such methodology? I challenged Lambert to show me some, he is uninterested it seems. So are you. The answers are too uncomfortable to merit revealing, I guess.
Keep being in denial buddy, it suits you.
John,
No, no, no. The fact that there was no rationale doesn't mean that the pairings were random. The study doesn't tell us how they came to pick the pairs. They arbitrarily picked them because they believed them to be similar. What is random about that? Exactly, nothing.
The outcome was not biased by an "unfortunate roll of the dice". They rolled no dice in picking 12 out of the 18 provinces. They rolled no dice in picking which provinces to be paired. They didn't roll the dice for each cluster to be distributed in the 2nd phase, violating the principle of the sample being random.
There was absolutely nothing random about the sample ending up the way it did. They purposely chose those exact 12 provinces, they purposely sought to exclude half of them from being sampled, and purposely sought to oversample the other half.
That is a whole lot of bias, unless you keep pretending that they randomly picked things when they in fact did not.
"An analogy. If you flip a fair coin 5 times and get five heads then (1) there was no bias in the flip; but (2) the outcome certainly couldn't be taken as normal. You could see how somebody might easily refer to the outcome as biased. His language isn't quite right but his point is obvious enough."
Eh, no. This analogy doesn't even mirror the Lancet study at all. The explanation is above. There was nothing random about the selection of the 12 provinces. There was nothing random about 6 of them being excluded - it was predetermined that 6 of them would be excluded. There was nothing random about ALL of each pairs clusters ending up with one of them.
All of this represents a bias introduced to the sample by the authors of this study.
Donald,
I was just pointing out the misleading, although accurate, point made by Lancet-apologists that 98,000 is the "most likely" from the range. The question no one wanders off to is "yes, but how likely?" With a smaller confidence interval, this would not really matter. The point was that with such a large interval, 98,000's probability is 0.000013 while 50,000 has a probability of 0.000004, about 1/3 as much.
If the range was 95,000-101,0000 with 98,000 as the mean, then the probability of 98,000 would be 0.0004 while 50,000 would have a probability of 0.
By saying "well, 98,000 is the most likely" and not telling how likely, or how likely other figures within the range are, you are misleading people. The fact that 50,000 being only 1/3 as likely as 98,000 should be alarming.
Yet again, it isn't. "IT'S THE MOST LIKELY!!"
All of that, of course, ignores all the bias that they introduced into this study by their arbitrary and non-random decisions.
You haven't contributed to this discussion one bit as your only task has been to try and ridicule me. You are just a partisan cheerleader without any knowledge about anything being discussed here. So how about you let the adults sort things out, OK?
I got an A in a Uni stats class too. Does that count?
I've contributed to a UHI paper where I analyzed the distribution of metro measuring stations for UHI bias, and my Pop Biol, ecology and botany classes regularly did this stuff. Anyway,
I've weighed in on this in a previous thread. There's nothing wrong with the paper per se, IMHO.
If you have substantive issues with the paper, the accepted and standard procedure is to write them up and send a letter to the publishing journal.
Anything else is ululating. No one cares about what you say on your website, because no statistician connected to the issue reads it.
HTH,
D
John H. said:
However, more to the point I think what Seixen is trying to say is that (while there may not have been any sampling bias in the study) the outcome was "biased" by an unfortunate roll of the dice. I think he means to say that the results were not representative of reality.
Yes, perhaps. I agree. We won't know until another study is done.
This has been gone over a million times. One paper doesn't make a tout.
Certain people are touting, but one can't conflate those certain people with the authors, the scientific community, or leftist green-weenie whack jobs.
But I don't want to seem, John H, that I don't value your post. I think you make an excellent point.
Best,
D
Dano et al,
The outcome was not the result of "an unfortunate roll of the dice". When are you guys going to come out of denial and admit it?
To prove that they introduced a bias to the sample, answer this one question:
Why was Anbar not paired up?
>Freedom of religion is worse off than with Saddam? So Saddam banning them from religious rituals and pilgrimages and all of that, which they are now allowed to do as they want, that doesn't mean anything? How has freedom of religion gotten worse?
Entire religious communities including the Catholic and Assyrian Christians and the Mandeans are fleeing the country to escape attacks by Shia extremeists.
Shia mosques are being destroyed and clerics murdered across the Sunni regions and the exact same process is happening in reverse in the Shia areas.
Allowing the majority relgious group to suppress all the others is not "freedom of religion".
>It doesn't matter when he wrote it. He has to forever protect his reasoning for not going on to take out Saddam
Ah so now not only are you an expert on statistics who has caught errors that eluded thousands of professional statisticians and an expert on climate science you also know George Bush senior's motives and reasoning better than he did.
Jet:
>I'm pretty sure I was responding to you.
>"Had Saddam and/or his sons remained in power for another 20 years, the likely death toll would be less than your own estimate of 50,000+ deaths resulting from the invasion."
Yes,and I was addressing Seixon hence my use of the words which you quote "your own estimate". There's an earlier post where Seixon concedes the Iraqi deaths may have been aroudn 50,000.
After a couple of my recent exchanges with Seixon I started to worry that I might be simply be provoking him and bringing out the worst in him.
So I went back to the first page of this exchange and he was arrogant and abusive before I ever posted on this thread.
Seixon, if your point about 98,000 was only that the confidence interval is very wide, everyone already knows it. If the 95 percent confidence interval goes from 8000 to 194,000 (2 standard deviations either way), then the 68 percent CI would run from roughly 50,000 to 140,000 or so. Or that's my understanding. So there'd be a 16 percent chance the excess death toll is under 50,000, using the Lancet calculations. That's my rough understanding, but anyway, allowing for whatever numerical mistake I might have made with my rough estimates, people have already discussed this and anyone who actually bothered to read the paper would probably know enough about statistics to realize this.
BTW, Seixon, though I'm one of your opponents here, on one topic completely unrelated to whether the Lancet article is right you have my complete sympathy--the philosophy underlying confidence intervals befuddles me. I read Detached Observer's latest comment to you about that and will head on over to wikipedia and see what they say, but if you can't make some sort of statement about which number is more likely to be right within a CI than what are confidence intervals for? If my numbers in my previous post were right, then the Lancet authors could say that there is a 68 percent confidence interval that the death toll is between 50 and 140,000. If one doesn't want to say the probability is 95 percent that it falls within the range 8-194 thousand and one doesn't want to say that the there is a 68 percent chance it falls in the narrower range, fine. Just say we are 95 percent confident or 68 percent confident and talk about confidence distributions instead of probability distributions.
Ian,
I guess the possibility that Bush Sr. wasn't being 100% candid about his reasoning has completely escaped you.
About the religion war inside Iraq, do you have any links to stories about the Shia running the Christians and Sunni out of the country? If that is happening, that is alarming, but still we have to know that this is only 2 years after the invasion. I don't think things in Germany, Japan, South Korea, and many other places got around to normalcy within such a short time frame. So I kind of frown upon this "must-happen-now-or-it's-a-failure" mentality.
Also, there was nothing about "conceding" that the death toll in Iraq is probably around 50,000 as of today. If the real toll was 100,000, that would be unfortunate. It would still not mean that the war was not worth it. My attacks on the Lancet study have nothing to do with the number they came up with.
Donald,
I have no idea where Detached Observer is going with what he is saying. Using 98,000 as the mean, and a confidence interval of 31,000 between 8,000 and 194,000... I calculate that the probability of the figure being between 50,000 and 140,000 at around 85%.
This was not a main contention of mine, nearly just another thing on top of everything else. It was mainly because someone commented on my blog asking if the probability for 98,000 was 50%. If that's what people are thinking, then that needs to be corrected at once, that is just completely wrong. I think that by saying that figure is the "most likely", many are being misled into believing the authority of the figure.
All of that is irrelevant because they introduced a bias to the sample which throws everything out of whack.
Apropos that, I see that no one has answered by question. I guess I will have to reiterate:
WHY WAS ANBAR PROVINCE NOT PAIRED UP?
What Detached Observer said on confidence intervals is what the frequentists say, according to the wikipedia article. I'm aware of the Bayesian vs. frequentist debate and to the extent that I understand it, I'm on the Bayesian side.
Anbar Province was thrown out of the final calculation of death toll, Seixon. Which makes it unlikely they were conspiring to slant the death toll upwards.
A paper was recently published which might be of interest to participants in this thread, because of its subject matter, and also because it was co-authored by Les Roberts, one of the authors of the Lancet article. The title and abstract are below.
This link is to a page from which the full text of the article (pdf) can be opened or downloaded.
INTERPRETING AND USING MORTALITY DATA IN HUMANITARIAN EMERGENCIES: A PRIMER FOR NON-EPIDEMIOLOGISTS
Mortality data, properly collected, interpreted and used, have much to contribute to the appropriateness and effectiveness of humanitarian action in emergencies, and to advocacy on behalf of populations in crises. Most actors involved in relief will one day be confronted by such data, but the different ways in which this information can be collected, and their potential pitfalls, are not yet common knowledge among non-epidemiologists.
This Network Paper describes the practice and purpose of that branch of epidemiology concerned with population mortality. It sets out the key indicators used to express mortality data, different options for how to measure mortality rates and suggestions for how to assess, interpret and use mortality reports. The paper also discusses the politics of mortality figures.
The paper's aim is to enable readers to critically interpret mortality study reports, and to understand how these are used (or misused) to formulate policy. The intended audience is therefore all humanitarian actors, policy-makers, the media and members of affected communities, who may be called upon to comment on or make use of mortality studies, regardless of their technical background.
Donald,
You didn't answer the question, chicken. It was only thrown out because their result was so crazy that they couldn't include it without looking like complete goons. Anbar was still in the sample, and was surveyed.
Why wasn't Anbar paired up Donald? Regardless of what happened after the survey was done (in which they did survey Anbar)...
WHY WAS ANBAR NOT PAIRED UP?
Or hell, since Donald is obfuscating...
WHY WAS BAGHDAD NOT PAIRED UP?
WHY WAS DIALA NOT PAIRED UP?
>To prove that they introduced a bias to the sample, answer this one question:
>Why was Anbar not paired up?
Let's see - because they were trying to reduce the time, cost and danger involved in travelling from Baghdad to the survey sites and Anbar is right next to Baghdad?
>I guess the possibility that Bush Sr. wasn't being 100% candid about his reasoning has completely escaped you.
No I just consider the probability of that being lower than that of some random schmo - who hasn't even read the comments in question- arbitrarily arriving at a more accurate assessment.
>About the religion war inside Iraq, do you have any links to stories about the Shia running the Christians and Sunni out of the country?
I can provide them if necessary but in the meantime try a little website called Google.com. While you're they're look at the ethnic cleansing of Arabs from the Kirkuk region.
>I don't think things in Germany, Japan, South Korea, and many other places got around to normalcy within such a short time frame.
There wasn't a single allied serviceman killed by hostile military action in either Japan or Germany during the post-WWII occupations.
Two years after the Korean civil war, the country was a repressive military dictatorship which retained power for 30+ years and tortured and murdered thousands of its own people. I'd hesitate before using it as a prime example of a "successful" occupation.
I need ot go back and look at your exact words regarding the 50,000 deatsh figure.
Message 144 (from Seixon):
>I deduce that the correct death tally, for all Iraqis including combatants, to be somewhere around 35,000-50,000.
Message 189 (from Seixon):
> there was nothing about "conceding" that the death toll in Iraq is probably around 50,000 as of today.
I had a few more minutes that I thought before my bus:
http://www.abc.net.au/correspondents/content/2005/s1467525.htm
...the worsening sectarian conflict between Iraq's Shia majority and Sunni minority has largely overshadowed the fate of another minority group, the Mandeans.
They're one of the most ancient, as well as one of the smallest communities in Iraq.
Their religion, Mandeanism, comes from the same general background as Judaism, Christianity and Islam. They share many of the same prophets, but especially honour John the Baptist.
Almost all Mandeans used to live in Iraq. But since the US-led invasion, many have fled, in order to escape persecution by Islamic extremists. One of the largest Mandean communities lives in Australia.
Kate Clark has been speaking to some Mandean refugees who are now based in Syria.
http://www.latimes.com/news/nationworld/world/la-fg-iraq1oct01,0,111787…
In Baghdad, as in the rest of Iraq, sectarian lines are hardening and residents are being forced out of their neighborhoods.
On Monday, insurgents dragged five Shiite teachers and their driver into a classroom in the village of Muelha, 30 miles south of Baghdad, and shot them to death.
Tuesday night, men in police uniforms came for seven Sunnis in the Hurriya neighborhood. Police discovered their bodies the next day, dumped near a railway line in Shula, a northwestern Baghdad district. The men had been blindfolded, handcuffed and shot execution-style.
In Dora, which stretches over 30 square miles on the southern rim of the capital, a systematic campaign of intimidation has changed the fabric of this once-diverse neighborhood, authorities say.
Jasim Hasan, a 63-year-old blacksmith, said three Shiite shopkeepers in his corner of Dora recently packed up and left. "We hear of two to three assassinations every week," said Hasan, who has noticed the constant rumble of trucks moving furniture out of the neighborhood.
The twin stacks of the oil refinery cast a shadow over the middle-class neighborhood, where a large community of Assyrian Christians came to work when the plant was built by the British in the 1940s. On these streets, family trees intertwined, and Christians, Sunnis and Shiites lived, and prayed, side by side.
But last year, things started to change.
First the Christians fled, their churches destroyed by insurgents.
Now Shiites are fleeing, leaving homes and businesses empty. At least 150 families have left; storefronts are boarded up, the shutters drawn on once-lively markets. Moving trucks rumble past paper signs proclaiming the exodus in hastily scrawled letters: "For Rent."
http://www.cascfen.org/news.php?nid=1357&cid=22
AINA, Baghdad, 29.08.2005 -- Assyrian protests against the proposed Iraqi constitution have resulted in deadly reprisal attacks against Assyrian Christian civilians by forces loyal to Masoud Barazani, the tribal chieftain of the Kurdistan Democratic Party (KDP). On August 24, several hundred Assyrian (also known as Chaldean and Syriac) protestors demonstrated in the two Northern Iraqi Christian towns of Qaraqosh (Bakhdeda) and Telsqof (Tel-sqeepa) against the constitution's splintering of the Assyrian Christian community into separate groups identified as either Assyrian or Chaldean. The Assyrian community in Iraq had previously been referred to singly as "ChaldoAssyrian" in the Transitional Administrative Law (TAL) to reflect the community's desire to be recognized as one.The demonstrations was also intended to send a strong signal of opposition to KDP's continued illegal expansion into Assyrian areas.
Two days later, the KDP struck back with deadly force. On August 26 37 year old Nabil Akram Ammona, a resident of Bartilla, was shot twice in the head by KDP paramilitaries while filling his automobile with gasoline. A group of armed KDP security wearing Iraqi National Guard (ING) emblems drove up to Mr. Amona and forcibly took away the gas nozzle he was using. A bewildered and irritated Mr. Amona protested. Instantly, one of the KDP thugs fired two shots execution style into Mr. Amona's head.Mr. Amona died instantly as he slumped into a pool of his own blood.
Further on the state of Christians in Iraq:
http://www.sfgate.com/cgi-bin/article.cgi?file=/chronicle/archive/2005/…
Basra, Iraq -- For the Christians in Basra, the downfall of Saddam Hussein has meant a terrible loss of religious freedom.
The social club where Yousef Lyon and his friends would gather in the evening to play dominoes, where families danced or listened to live music on holidays, is closed. Wedding celebrations are held quietly at home.
"Of course, during the Saddam regime, it was better," said Lyon, 40, a member of the city's small Armenian community. "Now we are afraid from the religious parties that maybe they will throw a bomb at us."
Not just the Christians, but many of the city's minorities -- from obscure sects like the ancient Sabeans to the sizable Sunni Muslim community - - live in fear of the hard-line Shiite religious parties and their militias that now rule Iraq's second-largest city.
Freedom has been curtailed for women, regardless of their religion. Several decades ago, almost no woman in Basra covered her head. Now, they all do, under fear of harassment or worse.
Women working for foreign companies or governments, and those considered to have loose morals, have been marked for death by the militants -- two Iraqi sisters who worked in the laundry at the U.S. compound in Basra were assassinated last year.
http://sfgate.com/cgi-bin/article.cgi?f=/c/a/2005/07/19/MNGSQDQ4C21.DTL
Damascus, Syria -- Seated in his parish office, Father Sarmad Yousef reflected on his hard choices: to disobey his archbishop by remaining in Syria or to return to Iraq, where his name has appeared on a death list.
"After the Americans came, I was one of the people telling the Iraqi Christians not to leave," he said. "After the violence started, I stopped telling them that."
Christians all over Iraq face a similar dilemma as relentless violence engulfs the country, some directly targeting them.
Staying in the midst of the threats is dangerous, yet leaving means abandoning communities, church property and a heritage with centuries-old roots.
Before the U.S.-led war, roughly 750,000 Christians lived in Iraq, out of a population of 25 million. Most were Chaldean and Assyrian, but there also were Armenian, Jacobite and Greek Orthodox Christians and a small number of Protestants. Most of them lived either in Baghdad or in northern Iraq around Mosul.
Since then, 15,000 to 20,000 Christians have fled to Syria, according to Christian groups, out of "about 700,000" Iraqis, most of them in flight from the war, according to the U.N. high commissioner for refugees.
http://www.telegraph.co.uk/news/main.jhtml;sessionid=KUBUPNSRFPGR5QFIQM…
ne of the most ancient monasteries in the world, St Matthew's, stands on a barren mountainside in northern Iraq, its last inhabitant a crusty old Syrian Orthodox priest. Nestled between sandstone crags with views of the hills around ancient Nineveh, now called Mosul, it looks like the final redoubt of the Christian world.
Seven thousand monks used to worship here; now there is just one, Father Ada Qadr al-Kars.
This thinning of the ranks has taken centuries, he said, but in the valleys Iraq's Christian community, targeted with especial ferocity by Islamic extremists for the past year, is disappearing rapidly.
Churches have been bombed, priests kidnapped and Christian neighbourhoods subjected to random shootings, the terrorists' revenge for the community's shared religion with the "Christian" invaders.
According to Church leaders, some 300,000 Christians - roughly a quarter of the population - have fled their homes since the US-led invasion.
"To prove that they introduced a bias to the sample, answer this one question:
Why was Anbar not paired up?"
Let us for a moment say that Anbar was not paired up because the authors were biased. Let us continue... and say that because of this bias the whole study is crap.
Let me be very clear about this ... there is still no sampling bias!
So while Seixen may have a point (let me avoid taking a stance on that issue at the moment) I think it should at least be clear that Seixen was wrong in his language. Lets have that admitted and then move on to the points of substance.
Or better still -- just move on...
I'm not sure that there are any points of substance left to discuss. D-squared pretty well summed up the final state of the debate in a WoC thread:
>Amac has been very fair on a number of issues where we disagreed, but the following facts do seem to be established, and I am surprised that nobody except me appears to regard them as important:
>1. It would be very odd to get a sample in which the death rate increased by 50% if the death rate had actually fallen.
>2. It would be quite odd to get a sample in which the death rate had increased by 50%, if the death rate had actually only risen by a small amount corresponding to the 8,000 excess deaths figure.
>3. It would be quite odd to get a sample in which violence had gone from an insignificant cause of death to a very significant one, if the increase in violence was not in fact the major cause of the increase in the death rate.
>I don't think anyone has a credible argument to suggest that the Lancet study does not provide fairly convincing evidence for all three of these propositions (the arguments at the Chicago Boyz site are now all about presentational matters rather than factual ones). And taken together, they do appear to substantially support the proposition "the invasion of Iraq has made things substantially worse for the Iraqi population (in so far as this can be measured by the death rate), not better".
>I'd also add a fourth comment:
>Everyone seems to agree that the vast majority of the deaths in Fallujah were, in fact, attributable to violence and most likely to violence by coalition forces. It seems unlikely to me that the death toll in Fallujah (plus that in Najaf, Samarra and Ramadi) will, in the final analysis, be so small compared to the total that it can safely be ignored.
>I don't believe that any of my 1-4 require any statistical training at all to understand, at least in qualitative terms. And taken together, they imply something really quite serious indeed.
None of that is controversial and there is little point in trying to squeeze more than that out of the study. Its main value now is that it shows what can be done by a small group of people with a small budget if they have the guts and the know-how. I suspect many teachers of statistics will also use it to bring the subject to life (yes I know that's an unfortunate phrase) for classes who are weary of exercises with t-statistics and such.
Ian noted that *"There wasn't a single allied serviceman killed by hostile military action in either Japan or Germany during the post-WWII occupations."*
It always strikes me as odd when people with little military knowledge or experience try to use military examples that just don't fit, and that's as good an example of one as any.
If Berlin had fallen to an allied invasion that had landed at Normandy on June 6th 1944 and then reached Hitler's bunker only *three weeks* later, then I think it would be fair to say that most of Europe would have ended up going through problems rather like Iraq's today. The upside is that Iraq wasn't comprehensively flattened like most of europe was between June 1944 and May 1945.
In fact, there were quite a few deaths of allied serviceman in europe after the fall of Berlin. Soviet troops were used to ongoing fighting with partisans in eastern europe for some time after May 1945, all though I suppose if we take that example to its logical conclusion we could argue that the soviets weren't really 'allied' serviceman as such. Nevertheless there was still pretty high levels of violence in the region long *after* the nazi's fell from power; it just happens to be outside of the western perspective and is largely forgotten today.
As for the far east; the occupation of Japan itself was fairly peaceful. The rest of the region was not. Insurgency and fighting took hold all over Burma, Vietnam, Korea and Malaysia almost as soon as japanese control had been removed. Certainly allied serviceman died in these places. To give but one example, Britain had fifty casualties in Vietnam (Yes- Vietnam; see http://www.britains-smallwars.com/Vietnam/Opening.htm ) in 1945/6 alone.
Allied serviceman seem to have been fairly safe within Germany and Japan themselves; they were often at risk anywhere Germany or Japan had *been*.
Ian also mentioned that South Korea wasn't a model 'occupation' which ignores the fact that much of its problems stem from it being too independent, not being too 'occupied'. Would he rather have lived in North Korea at the time?
Seixon, I don' t know the details of why the Lancet authors did what they did. And the name-calling just seems silly--it's not going to get me to speculate on why Anbar wasn't paired. BTW, if it had been paired then as I understand it there's a good chance Anbar would have received more clusters, possibly resulting in a significantly higher calculated death toll if Anbar is the bloodiest province. Or it might have not received any, but then, when Fallujah got tossed, I guess it didn't anyway.
Daniel's summary is exactly the one I'd make, except he did it better. What is a WoC thread, Kevin? I assume it's a blog name, but am not familiar with it. And if you read this, which thread was it? If I have time I'll read it.
Wilbur,
Seixon chose the comparison between post-war Japan and Germany and Iraq. If you don't like it, I suggest you take it up with him.
>Ian also mentioned that South Korea wasn't a model 'occupation' which ignores the fact that much of its problems stem from it being too independent, not being too 'occupied'. Would he rather have lived in North Korea at the time?
"Not north Korea" now there's a great slogan for the new democratic Iraq, something Iraqis and Americans alike can feel glad to give their lives for.
On "civil war":
Iraqis are already fighting Iraqis. Insurgents have killed far more Iraqis than Americans. That's civil war. We created the civil war when we invaded...
http://www.antiwar.com/orig/odom.php?articleid=7487
About the author:
>Lieutenant General William E. Odom, U.S. Army (ret.), is a senior fellow with the Hudson Institute and a professor at Yale University. He was director of the National Security Agency from 1985 to 1988. From 1981 to 1985, he served as assistant chief of staff for intelligence, the Army's senior intelligence officer. From 1977 to 1981, he was military assistant to the president's assistant for national security affairs, Zbigniew Brzezinski.
I look forward to hearing from Seixon what Odum's REALLY meant.
Anyone want to call the Reagan-era head of the NSA a wimp, a left-winger or unqualified in military or middle eastern affairs?
Donald,
"the name-calling just seems silly-it's not going to get me to speculate on why Anbar wasn't paired. BTW, if it had been paired then as I understand it there's a good chance Anbar would have received more clusters, possibly resulting in a significantly higher calculated death toll if Anbar is the bloodiest province."
And thus you prove my point about the pairings without even realizing it. Read through what you said right here again and see if you can see it...
Ian,
Alright, if what's happening in Iraq is civil war, then you have just water down the meaning of that phrase. Almost all of the people being killed are being killed by IEDs. Very few are being killed by regular military action. It's not like the Shias have their army, and the Sunnis have theirs, and they are fighting it out. We have a small insurgency that is fighting against the rest of the country, with little or no unprovoked fighting being launched the other way.
So fine, if you want to call it a civil war, whatever. I guess next step is calling it total war. Definitions seem meaningless.
What really gets me about all the people whining about the insurgency, the "civil war" and all the deaths, etc, etc...
The people who oppose the war that are not psycho, in other words, the ones that aren't actually on the other side... They include people like Greg Palast, who advocates that we should have supported a revolution against Saddam Hussein instead of what we did. There are very many in the anti-war group of people who agree with this, including my mother.
They don't want our men and women over there, they wanted the Iraqis to rise up and take Saddam out themselves, possibly with our assistance.
Now, since that's not what happened, they whine about all the deaths, all the damage, the insurgency, etc.
What in the hell do you think would have happened if we had supported a revolution against Saddam Hussein????????????????????
THAT would have been civil war in the full meaning of the phrase. That would have undoubtedly resulted in the deaths of hundreds of thousands of people. Saddam's army annihilating the rest of Iraq, just as almost happened in 1991-1992 until the resistance gave up because they were getting killed by the thousands.
Would that have been better? Would that have resulted in democracy, or would it have led to another strongman running the place? Would that have left less people dead?
Oh wait, I know why! Then 2,000 AMERICANS wouldn't have died! That's what it's all about!
If that's not it, Ian, then please do tell me what other possibility there was for getting Saddam Hussein out of there that would have ended up with less deaths, destruction, and a better situation in Iraq than exists right now.
I would really love to hear it.
Anyways, this is REALLY off-topic!
Check out Lambert's newest dishonest post on the Lancet study! Yeay!
I thought about it Seixon, right after posting, and if Anbar was thought to be the bloodiest province, then they wouldn't pair it because there wouldn't be another province quite like it.
Pairing with a province with a much different death rate wouldn't change the expectation value, but it would increase the variance, which is the technical-sounding rewording of what I said in my previous post. It's the point nearly everyone has been making--pairing doesn't change the expectation value, but to the extent that the paired provinces have different death rates, it increases the variance. If Anbar wins, the calculated death rate is likely to be too high. If Anbar loses, the calculated death rate is likely to be too low. It's best not to pair, in order to avoid that problem, but if you do then both the Lancet authors and presumably everyone here understands what the effect is going to be and it's best to try and pair similar provinces. If that's your point, we've had 200+ thread with people all trying to say the same thing, except you've not been using the technical language properly. But it's not quite your point, because you see the non-pairing of Anbar as evidence of bad faith, when it's the correct thing to do if the Lancet authors thought it was exceptionally bloody.
Ian, you are sidestepping, when you have made a misleading statement. You tried to imply that the aftermath of the downfall of Berlin & Tokyo in 1945 was all peaceful and nice, as a debating point. I only pointed out that your statement wasn't very accurate.
I'll see if I can follow the reasoning you have developed on foreign policy and military action in this thread.
1). That Saddam should have been left in power. That's because it would have meant *x* level of casualties instead of the *y* level resulting from the regime change. Thus you were pulling for regime *stability*. I guess this means that the touchy feely relationship western governments had with Saddam in the '70s & '80s was OK then? After all, they were only ensuring *x* number of casualties happened.
2). That regime stability in South Korea was wrong, because it created *x* number of casualties. The only other choice the south koreans had was rule by a lunatic protege of Mao in the North, and the only model of government they were used to was the Japanese one that lasted until 1945. A regime change in favour of the north would have meant more warfare and *y* number of casualties, and a lot more in the long run if north korean government practices are any example.
Somehow, you've managed to propose no regime change in one example, a change in the other, why not just say that you think US foreign policy is wrong no matter what it does? It would follow a more consistent pattern of logic.
You've also quoted Lieutenant General Odom as an authoritative source to say that 'Insurgents have killed far more Iraqis than Americans...'.
This is actually the opposite of what the research by Roberts et al found. Are you now quoting him as a resource *against* the study, then?
There is some bias at work here, and it isn't to do with sampling.
Donald Johnson,
By WoC I meant Winds of Change, where "AMac" had a (different) post which Tim Lambert also noted. AMac is one of the very few intelligent and civil Lancet critics, but most of his criticisms relate to the presentation of the report. That d-squared summary of the debate was in this Joe Katzman thread:
http://www.windsofchange.net/archives/006569.php#c13
Donald,
Again, who's to say that the pairings were similar? Yes, they didn't pair Anbar up because they assumed it was going to be higher than most others, but why not Baghdad?
This all comes back to them pairing provinces according to "belief" and nothing else. This, as someone else was asking, gives an unknown bias because you don't know if those provinces were similar or not, so you have no way of knowing what kind of bias you are introducing.
That still doesn't explain why Salah ah Dinh was paired, while Baghdad, Diala, and Wasit were not.
Regardless of that, matching up provinces that had unequal populations, and then giving all the clusters to one of them via a single trial biases it towards the more populous areas. That is just a fact.
Kevin tried to get around this by toying around with expected values, and showing that they would be the same regardless. Alas, someone else commented that expected values operate on the given that n elements are distributed randomly independently and blew the cover off that fallacy.
Face it, all the provinces that were paired up were artificially (and non-randomly I might add) introduced a higher probability of being excluded from the sample. That is a bias no matter how much you want to obfuscate. Lambert didn't even address this at all in his new post, because he can't without looking like a dunce. Lambert has now sought to redefine cluster sampling in order to get away with statistical murder.
Geez.
It seems to me that Wilbur is forcing the cards offering either or choices that he chooses out of a much broader menu. The US and its allies (remember them, we used to have a few?) had multiple other choices both in 2002-3 and earlier
For example wrt Iraq:
1. Shoot down Saddam's helicopters during the Shiite revolt after the first Gulf war,
2. Maintain the no-fly zones, establish a Shiite autonomous zone in the south
3. Expand Kurd autonomy while wiping out Zakarwie's encampment on the Iran/Iraq border in the Kurdish reqion
4. Kill alQueda in Afghanistan/Pakistan and then go after Sadaam.
5. Wait until enough troops were available to maintain order in Iraq after an invasion or even to be aware of the consequences of not doing so.
and so on
Similarly, the choice in South Korea was never reduced to Syngman Rhee or Kim Il Jung. That is a sour old red herring folk keep hauling out.
>Almost all of the people being killed are being killed by IEDs. Very few are being killed by regular military action.
1. Dead is dead.
2. The insurgents use IEDs as their principal weapon agaisnt the US forces. When fighting other Iraqis they use a mix of IEDs and other weapons (mortars and small arms for the msot part). The Shia death squads (whose existence you are doing a wonderful job of ignoring) seem mainly to shoot their victims.
>What in the hell do you think would have happened if we had supported a revolution against Saddam Hussein??
Well, for starters, there wouldn't have been a large foreign occupation force to attract hostility and foreign jihadis probably wouldn't have been motivated to take part...
If you actually look at the history of various dictatorships aroudn the wordl you'll notice that msot of them aren't overthrown from outside by military force - they either reform themselves (see Taiwan, Mexico, Brazil, Argentina, Chile, south Korea) or are overthrown from within.
The best (as in most credible model for what woudl have eventually happened in Iraq) is the military coup there in the 1950's which overthrew the monarchyg
Guess what - it wasn't followed by a civil war.
>Anyways, this is REALLY off-topic!
>Check out Lambert's newest dishonest post on the Lancet study! Yeay!
You really are desperate for attention, aren't you?
>Ian, you are sidestepping, when you have made a misleading statement.
Funny you didn't find it misleading when Seixon made it.
>I guess this means that the touchy feely relationship western governments had with Saddam in the '70s & '80s was OK then? After all, they were only ensuring x number of casualties happened.
No, the support for Saddam in the 80's was a mistake because it strengthened and encouraged him. You know, like the west is currently strengthening and encouraging Musharref.
Had the western powers remained neutral during the Iran-Iraq War, Saddam would have been severely weakened and might well have lost power.
Of course, at the time this was considered unacceptable becasue it coudl have led to a shia-dominated pro-Iranian regime in Iraq - you know like the one we now have.
>That regime stability in South Korea was wrong, because it created x number of casualties. The only other choice the south koreans had was rule by a lunatic protege of Mao in the North, and the only model of government they were used to was the Japanese one that lasted until 1945.
I never said that supporting South Korea was wrong. I merely pointed out that contrary to Seixon's Panglossian comments, it took several decades of brutal dictatorship to establiah s democracy there.
I hope it won't take as long in Iraq but I fear that it will - it took about 60 years of secular dictatorship for Turkey to become a more or less democratic state.
Ian,
"Funny you didn't find it misleading when Seixon made it."
Haha, I said that Berlin and Toyko were peaceful after liberation? No I didn't, *[Deleted -- see my comment policy. Tim]*
"No, the support for Saddam in the 80's was a mistake because it strengthened and encouraged him. You know, like the west is currently strengthening and encouraging Musharref."
Yes, I'm sure it would have been better not to support Musharraf, then I'm sure the whole topple the Taliban, root out al Qaeda enterprise would have gone so smoothly... Oh... wait a minute...
"Of course, at the time this was considered unacceptable becasue it coudl have led to a shia-dominated pro-Iranian regime in Iraq - you know like the one we now have."
Yeah, with a Kurdish president and all. Go figure.
"I never said that supporting South Korea was wrong. I merely pointed out that contrary to Seixon's Panglossian comments, it took several decades of brutal dictatorship to establiah s democracy there."
Eh, I made the case that things did not get better there right away after liberation, just like Iraq won't. I guess that point flew right by you.
"The best (as in most credible model for what woudl have eventually happened in Iraq) is the military coup there in the 1950's which overthrew the monarchyg
Guess what - it wasn't followed by a civil war."
No, but it was followed by Saddam Hussein. Brilliant! Back to square one!
Ian,
No, the support for Saddam in the 80's was a mistake because it strengthened and encouraged him. ...
Had the western powers remained neutral during the Iran-Iraq War, Saddam would have been severely weakened and might well have lost power.How many times must I refute this? The only "Western power" that influenced the outcome of the war was the USSR. Germany's, France's, and the US's involvement with Iraq was insignificant to the help supplied by the Gulf states (Kuwait 14 billion in loans and billions more in aid, Saudi billions in aid and use of air bases, loans, aid and help from other Gulf states), China (light weapons), the USSR (heavy weapons, advanced systems) and South Africa (artillery). Never mind that after the war had settled into a stalemate decidedly in Iran's favor with Iran having pushed deep into Iraq in the North (almost to Kirkuk oil fields) and the South (Basra oil fields) and holding key port facilities, it was the USSR that broke the stalemate by sending in military advisors in 87. Now magically in the Iraqi offensives in 87-88 Iraq's mechanized units performed as never before, almost as if Russians were crewing them. Needless to say, this help from the USSR was key to turning a certain protracted defeat into an almost instant victory.Iran only received what little help the Iran-Contra affair provided (infantry weapons), weapons from China, and ballistic missiles from Syria and N. Korea. Iran was mostly reduced to softening the Iraqis with unarmed/lightly armed human wave attacks followed by the Iranian regularsAnyone who claims "the West" enabled Saddam in the 80's is wrong as it was the USSR which single-handedly "won" the war for Iraq (sole supplier of advanced weapons and either crewed or supplied intense "military advising" for mechanized units which turned the course of the war). The US in particular gave much more help to Iran than Iraq.
*Jet* actually makes a good point- Saddam was a good friend of the USSR and France more than any one else in the '80s. There is a very silly scene in *Bowling for Columbine* where Moore claims that Iraq invaded Kuwait with american weapons- whilst he shows clear footage of Russian made tanks and armoured personnel carriers in Iraqi use.
Ian, you're the person who stated that *no* allied servicemen died in the occupation of Germany & Japan, as a rebuttal to Sexion's comment that they were reasonably succesful occupations after a long period of time. The fact that both occupations saw ongoing insurgencies develop nearby, where allied servicemen died, shows that your point doesn't pan out too well when considered (now that I think about it, the allied occupation of post-facist Italy saw US troops in occasional, and deadly, conflict with Yugoslav troops along the border, which is another example beyond the russian experience I mentioned). If the German and Japanese occupations were success stories inspite of the ensuing conflicts about them then it actually makes Sexion's example more relevant to Iraq in this instance.
Eli- you've argued some very good points; however, I don't think it would have mattered which option was chosen, Ian wouldn't have liked it, and neither would so many people for whom the west is always in the wrong.
jet,
I don't think George Galloway would appreciate your version of the Iran-Iraq war.
Heh, it appears that I was correct in relation to who carried out the survey. Les Roberts did not carry it out, as he says himself to the Socialist Worker:
And perhaps even more to the point:
Eat your heart out Eli.
Seixon, you are either being clueless or dishonest. You claimed that the people who carried out the survey were medical doctors with no more expertise in statistics than you did. Clearly you were not talking about the people who did the interviewing but the ones who did the design and analysis who did in fact have expertise and are in fact world-reknowned experts in their field.
I believe my statement was that the people who carried out the study in the field were medical doctors. This was correct as 5 out of the 6 Iraqis who were the ones who actually carried out the study in the field were medical doctors. This did not include Les Roberts, as far as I can remember, as he did not take part in carrying out the study in the field as I have just shown here.
Aren't you throwing stones in a crystal glass house regarding dishonesty Lambert?
"I have said that clustering of clusters is not what multistage clustering is about, and you will find that to be correct. I have also said that clustering these clusters has no precedent in statistics, something Lambert tried to steamroll by giving vague comments. The selection of some provinces for a 2nd phase, while leaving out the others, is the very definition of introducing a bias.
"Combining clusters in a 2nd phase, in only arbitrarily picked provinces, to ensure that 6 provinces were to be excluded is completely without merit statistically.
"Does your nifty textbook also talk about clumping clusters? Or is Tim Lambert the only one who has such a textbook? I'm seriously getting tired of waiting for verification from statistical literature that I am wrong about the cluster clumping being unsupported methodology.
"Now if you, or Lambert, or anyone wants to show me ANYTHING that indicates that clumping clusters together in a 2nd phase into a single cluster, and then breaking them off again once they are distributed as a block, is supported, scientific, and good statistical methodology, please, for the love of human intelligence, show it to me.
"Tim claimed this clumping action was cluster sampling - bzzzt, wrong.
"Tim claimed this clumping action was multistage clustering - bzzzt, wrong.
"The two-stage design introduces a bias because of the way it was done. The provinces chosen were not randomly chosen, and the pairings were also not randomly chosen. The pairings weren't chosen due to any rationale other than an arbitrary one. That no randomy a samply make.
"I mean damn, you can't even find a single paragraph or a page that says that clumping of clusters is an accepted methodology? That's all it would take. I've been waiting over a week for it, tick-tock.
"I find it hilarious that after over a week, Lambert still hasn't shown a single shred of evidence that cluster-clumping is valid methodology.
"He claimed that the clumping constituted "multistage clustering" which is ridiculously false.
"He claimed that the clumping constituted cluster sampling. Again, false.
"SO HOW ARE YOU GUYS DOING FINDING DOCUMENTATION FOR THE LANCET STUDY'S METHODOLOGY?
"Instead of being concerned with Lambert making a number of directly false statements, now you have created the most absurd argument I have read for a long time. That the great JHU researchers have indeed invented a new type of sampling, and that I must be a rube for not recognizing the immediate brilliance of it. Oh boy.
"Innovation? For what, creating imprecise mortality studies? Oh gee, sign me up for some of that!"
Multi-stage Probability Samples -1
* Large national probability samples involve several stages of stratified cluster sampling
* The whole country is divided into geographic clusters, metropolitan and rural
* Some large metropolitan areas are selected with certainty (certainty is a non-zero probability!)
* Other areas are formed into strata of areas (e.g. middle-sized cities, rural counties); clusters are selected randomly from these strata
http://www.ssc.wisc.edu/~oliver/SOC357/Lectures%20and%20Notes/SamplingB…
page 25