It never ceases to amaze me the way the Wall Street Journal combines superb news coverage with a completely clueless editorial page.
To balance an excellent news article by Carl Bialik on the first Lancet study, we have an innumerate article on the editorial page by Steven E. Moore. Moore claims that the sample size for the Lancet study is too small:
However, the key to the validity of cluster sampling is to use enough cluster points. In their 2006 report, "Mortality after the 2003 invasion of Iraq: a cross-sectional sample survey," the Johns Hopkins team says it used 47 cluster points for their sample of 1,849 interviews. This is astonishing: I wouldn't survey a junior high school, no less an entire country, using only 47 cluster points.
So how many clusters did Moore use to survey Iraq? In a 2004 LA Times article he wrote:
Polling in Iraq is done much as in any developing country. Interviews are conducted face to face by highly trained Iraqi interviewers. For a 1,500-person sample, for instance, 75 qada (the Iraqi equivalent of precincts) would be chosen at random, with interviews conducted in 20 randomly chosen households in each.
So 47 clusters is grossly inadequate but 75 clusters is plenty? Moore continues:
A 2005 survey conducted by ABC News, Time magazine, the BBC, NHK and Der Spiegel used 135 cluster points with a sample size of 1,711--almost three times that of the Johns Hopkins team for 93% of the sample size.
Funny how he didn't mention his own survey's 75 clusters here.
Moore then demonstrates that he really doesn't know what he's talking about:
Another study in Kosovo cites the use of 50 cluster points, but this was for a population of just 1.6 million, compared to Iraq's 27 million.
No, the size of the population can be mostly ignored when you work out what size sample you need. Steve Simon explains:
The best analogy I have heard about sampling goes something like: "Every cook knows that it only takes a single sip from a well-stirred soup to determine the taste." It's a nice analogy because you can visualize what happens when the soup is poorly stirred.
Margaret Smith has an activity you can try if you, like Moore, don't get this
Finally, it is easy to see whether or not the sample size was big enough -- look at the confidence intervals on the estimates. If the sample size is too small, then the intervals will be too broad for any useful conclusions. This is not true for this survey.
Moore offers a second criticism:
And so, while the gender and the age of the deceased were recorded in the 2006 Johns Hopkins study, nobody, according to Dr. Roberts, recorded demographic information for the living survey respondents. This would be the first survey I have looked at in my 15 years of looking that did not ask demographic questions of its respondents. But don't take my word for it--try using Google to find a survey that does not ask demographic questions.
Without demographic information to assure a representative sample, there is no way anyone can prove--or disprove--that the Johns Hopkins estimate of Iraqi civilian deaths is accurate.
This is an example of what Daniel Davies called the "devastating critique", where any departure from the pure Platonic Form of the Epidemiological Study is grounds for dismissing the findings. Would it have been better to have recorded ages for the living people (they did ask about gender)? Sure, but it would have taken more time, and it actually doesn't matter that much. For surveys where you have collected age and gender, weighting by age and gender usually gives you the same results as unweighted averages (which is what the Lancet study used).
So who is Steven E. Moore anyway? Turns out he is a political consultant at Gorton Moore International, a Republican-aligned political consultancy firm.
Update: Stats.org has more on Moore.
and it actually doesn't matter that much. For surveys where you have collected age and gender, weighting by age and gender usually gives you the same results as unweighted averages (which is what the Lancet study used).
I'm not sure weighting was Mr. Moore's point. I think he was suggesting that there wasn't enough reported or recorded data to be able to confirm that the households in the sample are representative demographically. Random selection decreases the probability of a nonrepresentative sample -- it doesn't eliminate it. You might still get a bad draw. I've always had to show that my samples were representative of target populations, and my treatment groups sufficiently similar to my control groups at baseline. Haven't the rest of you?
And 75 clusters is actually quite a few more than 47. I agree that the price paid by Burnham et al. is already on the page, but the Professor's remark about ignoring the cited-studies section because a grad student wrote it is unsettling -- just like the refusal to release even the SPSS/Stata code and log files is unsettling.
Finally, Tim, if you're going to engage in ad hominems regarding Mr. Moore's employers, does that make Mr. Roberts' political ambitions fair game?
You seem to be assuming that Moore is accurately describing their exchange. I'll bet his comment has been taken out of context.
My mention of Moore's employers is meant to help explain why he wrote what he did. And there is a difference between running for office, and having your job be to spin for Republicans.
Perhaps I'm going to just reveal my statistical ignorance here but...wouldn't the demographics of the respondants be totally useless? I'd be willing to bet that a very high percentage of the respondants were female, simply because men are more likely to be working and, from what I've heard, women in Iraq are often afraid to leave the house these days. I don't see how this says anything about the sample being unrepresentative though since the information being collected is not actually about the respondants in the first place.
Right. And if the Iraq population were a well-stirred soup we presumably would not be in this mess ;)
DrSteve:
This has been quintessentially Tim Lambert as long as I have read his blog for the last several years. Whoever disagrees with Tim's point of view must be one or more of:
The notion that the opponent might disagree with Tim, because, I don't know, he might sincerely think that he is right, appears foreign to Tim.
However, if someone agrees with Tim's point of view, he appears to ignore any evidence that they might have a point of view so strong that it might plausibly lead to bias. Consider for instance:
1) Counting the dead in Iraq: Interview with Les Roberts in Socialist Worker Online
2) Richard Horton (Lancet editor-in-chief) speech at Time to Go anti-war demonstration in Manchester on 23 September 2006 at YouTube
Not that there's anything wrong with their opinions, of course. But in this particular game, consistency is not Tim's name.
Tim still allows me to post these rude comments on his blog, a privilege I do appreciate.
Steven Moore once wrote:
One wonders whether Mr. Moore still stands by those words. The website that featured them, The Truth about Iraq, is no longer available.
You seem to be assuming that Moore is accurately describing their exchange. I'll bet his comment has been taken out of context.
So we'd both like to see some documentation. Well, that's progress, I suppose.
I mean, even John Lott eventually turned over his data and models, no?
Kevin P, your comments are untrue. If I think soemone is a liar, I will say so, and explain why. I do not accuse everyone who disagrees with me of lying. Here's a challenge for you: in well over a thousand posts, how many people have I accused of lying?
Like Victor, I'm going to display my ignorance of anything other than basic statistics as well -- I dunno, maybe all the criticism of the Lancet study has me feeling like innumeracy is the new in thing and I want to join in. But regarding the lack of demographic information and the possibility that the sample wasn't representative for age -- in order to discredit a study based on that, wouldn't there need to be a reason to believe that the results skewed with age in some way? Moore talks about "demographic information" -- well, the number of characteristics one could ask about respondents is pretty damn near infinite, isn't it? Left vs right hand, blue vs brown eyes, the last letter of their first name, etc. If one of these characteristics didn't match the population at large, would that cast doubt on the study? Seems to me the answer is no, only if you thought that the results correlated to that characteristic in some meaningful way.
Presumably, Moore is talking about age and gender, but why are those characteristics privileged? Is there any reason to think that persons of a given age or gender would systematically bias the results they reported in interviews?
DrSteve, you are being unfair to Lott. He's actually been very good about providing his data and models to other researchers. That's why his failure to do so in the case of his defensive gun use surveys stuck out like a sore thumb.
Tim, in my list above, "Liar" was the last element in a progression. I have not gathered statistics (hah!) but your other typecasts are fairly common. However, since you ask, in this one post, you manage to hit three people with the charge of liar (or repeating a lie).
Kevin P., I still don't understand what we are supposed to glean from waving the Socialist Worker/Time interviews that will lead us to believe the Lancet survey is suspect. It remains a transparent attempt to poison the well.
And oh, your "rebuttal" is nothing of sort, Tim Lambert linking to posts that contain, you know, substantive criticism.
Glenn:
I think Moore was not proposing bias, but rather referring to comparing the demographics (gender, age, etc.) of the surveyees to see how they compared to the population at large, as an independent verification of how representative the survey was. He says that this was not done, so we are left with no idea about it.
Spartikus, read the whole comment thread from the beginning, before leaping into a conversation already begun.
Well, then my apologies to Dr. Lott, and my awed admiration at your restraint. My reserves of goodwill for him were depleted a long time ago, and it had nothing to do with his work.
I hope that Burnham et al. will give researchers access to even a partially redacted data set some time soon -- I saw a UN estimate for Iraq at 1.5 million IDPs on CNN.com this morning, plus estimated outmigration of 1.6 million, so I'm very interested in figuring out how sensitive their results are to the population estimates. I guess I'm also wondering if the in-migration and out-migration data collected can be projected using the same measure as the excess mortality counts, and compared to other studies (like the raft cited on that CNN.com page) as another form of validation.
http://www.cnn.com/2006/WORLD/meast/10/18/iraq.main/index.html
Kevin P.
Is your concern decorum or accuracy? I assume you refer to the Hansen incident, which I would say was appropriate to call out as a lie. What do you have to offer to show that it was not a lie?
Neither of Moore's comments have salience.
You attempt in the case of Les Roberts, a fallacy of association; and in case of Richard Horton invert cause and effect--it is just as likely that his study led to his anti war stance. Neither applies to Moore, and make the likelihood of his bias more transparent.
Mike
Mike, read the whole comment thread from the beginning, before leaping into a conversation already begun. Tim challenged me to locate, in a thousand posts, one where he had called someone a liar. I found one for him in about 30 seconds. I have no opinion on whether the particular accusation of a lie is justified or not. Read the entire comment thread and you will figure out why I mentioned Les Roberts and Richard Horton.
Actually Kevin P., Tim challenged you to list how many times he's called someone a liar. The answer is...not many times. Not zero.
Maybe you should read the comment thread from the beginning, or something.
spartikus:
You must have read every word of his thousand posts then. What is the exact number?
How about this: let me drop the word liar, and leave the rest remaining:
Tim attacks the author of the WSJ op-ed as follows:
Note the above, where with little knowledge of Moore, Tim blithely implies selective quotation. The same is also below:
This is fairly characteristic of Tim Lambert. This is not inherently wrong, but you can't have it both ways, sorry. Someone's background, ideology and motivation is either fair game in trusting his work, or it isn't. Pick one.
That is really the point of my comments on this thread.
Who you gonna believe, two anti-war Democrat scientists who have said on the record they wished to influence an election or the government of Iraq?
For Lambert et al, it seems like a pretty easy choice. Same here, but for some reason I think we chose differently...
Thanks for your interest. Frankly, the WSJ op-ed page was to cramped to write everything wrong with the JH survey. I'll respond to some of your points, and give some additional thoughts.
First, it is "Gorton Moore" rather than "Gordon Moore," as you post. I guess that would make your posting "illiterate" rather than "innumerate." ;)
Second, the key point about the demographics is that they did not compare the results to the 1997 Iraqi census. Les Brown, one of the authors, told me in an email that he had not even SEEN the Iraqi census.
Third, we used 75 cluster points of 20 interviews each for our surveys of six cities in Iraq, with a sample of 1500. For a 33% larger sample, the JH team used 33% fewer cluster points, and for a nationwide survey. Considerable difference. That being said, it is almost always prefereable to use more cluster points. As Iraqi fieldwork capacity increases, more robust survey methods are used. Currently, most surveys in Iraq use hundreds of cluster points.
Which brings us to point four... The JH team has done two surveys of Iraq, 24 months (approximately) apart... it was months before any of the Iraqi teams I've worked with could do a nationwide survey. The six city surveys were the best they could do. There is considerable training and experience required to conduct a nationwide survey. The first survey ever conducted in Iraq, by an Iraqi PhD, was garbage. He got something on the order of 75% men and the survey skewed quite young. The JH team did not even go to Iraq to train the fieldwork team for the 2006 survey, and the conditions there are much tougher for survey work than they were in 2003-2004.
In terms of my qualifications... it doesn't take much in the way of qualifications to figure out that the JH team is conducting shoddy research. All you have to do is read their study carefully and compare it to its cited antecedents and it is obviously does not meet the authors' own standards.
That being said, I've done about 27 surveys in Russia, around 7 in Romania, half a dozen or so in Indonesia and one in East Timor, in addition to twenty or so in Iraq. I've spent almost two years in Iraq doing just this, and there are many talented and credible Iraqi organizations doing better survey research than the JH team. By a lot. I just got back from Iraq, and have been spending time with my family and reacquainting myself with my fiance, then the JH study came out. The JH numbers are just so outrageous, I had to write something about it.
I'd welcome comment.
I don't see how this study can be valid. From my mentor:
"Unless the cluster sampling was done extremely well (in the sense of random selection) it is pointless to project *any* number of deaths. The fact that they did so is no proof of anything except their desire to force a number.
I am presuming, I know, that you know very little about random sampling other than at the individual level. My students always have difficulty with the concept of random selection and how it relates to Bell curves and the Law of Large Numbers and other mathematical stuff, and I therefore tend to believe that others do, too.
Now, random sampling (for individuals) at base assumes that *every single possible individual chosen* has in identical chance of being chosen. Thus, exit polls, for instance, in which every fifth or whatever person leaving is chosen ARE NOT RANDOM, as those in between have no chance of being chosen.
Clusters are much, much harder and that much harder to understand (and thus easier to fudge and slip past those who should otherwise know better.) For instance, at first glance, the method of selection here seems to be 'snowball' selection (i.e., pick a random [purportedly] first knock then knock on all the doors around it) which has known and ENORMOUS problems. It is, for instance, NOT random at all, and thus, properly cannot be used to estimate to the general population with any level of confidence at all. Why? Because the 'sample' has been 'poisoned' by the inclusion of what appears to be 90% the sample non-randomly (i.e. once the first knock happened way over 98% of other potential doors were excluded).
This can be papered over when dealing with infectious diseases because once you find a source you can track it physically. For non-infectious deaths it is just silly to 'sample' this way.
Anyway, a proper cluster sample would use a stratified sample based at the very least on population (i.e. the number of original 'knocks' on doors should be proportional to the population of at least the province if not the city or whatever) AND by any other relevant demographic (i.e. make sure you sample appropriate numbers of Sunnis and Shi'a, Kurds and Arabs, rural and urban, young and old, male and female). If that was done, I haven't seen it reported.
For instance, while Baghdad accounts for about 20% of the total population, it has very few Kurds, so a stratified sample would have to reflect that.
And Fallujah had about 350,000 population in 2003 and has about 200,000 now. That is between .6% and 1.4% of the total population. Thus, no more than that percentage of the sample should have come from there. Notice any discrepancy here? If you want to project the death toll in Fallujah to the entire population I'd like to see how you arrived at the proper calculation.
I am not a professional statistician, I am merely a social scientist who uses and teaches statistics and survey and survey analysis.
This study reaches results that I can't understand without a great deal more information they are not giving out. Thus far, the onus of proof is on them that they did it right, and I don't see them as being very forthcoming."
This can be papered over when dealing with infectious diseases because once you find a source you can track it physically. For non-infectious deaths it is just silly to 'sample' this way.
This is totally wrong. Cluster sampling has been used numerous times in conflict situations such as Darfur, Rwanda etc.
Anyway, a proper cluster sample would use a stratified sample based at the very least on population
And that is what this one did.
This so-called statistics expert is clueless.
Who you gonna believe, two anti-war Democrat scientists who have said on the record they wished to influence an election or the government of Iraq?
What government again ? The same government that can barely control the Green Zone without the aid of American troops, let alone the rest of Iraq ? The government that has barely existed for 1/6th of the duration of this study ? The same government that includes people from a group (the Sadr faction) that is known to control the morgues in Baghdad and has been known to undercount deaths even in Baghdad ?
But I suppose if you still believe there were WMDs in Iraq, you can believe anything.
Well, the other point he brought up (and he seemed pretty worked up over) was:
If the WSJ piece is correct (and Moore cites specific conversations with a principal of the study) this is *DELIBERATE* malpractice of methodology and a total fraud!
Dr. Roberts flat-out states that they did NOT gather demographic data on their respondents and household inhabitants. Un-F'ing-believable!! My undergraduates take about 20 seconds for me to pound into them that you ALWAYS, ALWAYS, ALWAYS!! get standard demographic data, no question asked. Otherwise it's total crap.
Does this line of argument have any validity to it? Can anyone shed some light for me?
jon,
This is totally wrong. Cluster sampling has been used numerous times in conflict situations such as Darfur, Rwanda etc.
Does that in itself make it valid? You know, aside from the fact that the methodology in those was not the same as was used in the Iraq studies... Keep pretending that all cluster surveys are equal though!
And that is what this one did.
Ehm, not exactly, no. A simple stratified survey was not done here, for practical reasons.
What government again ? The same government that can barely control the Green Zone without the aid of American troops, let alone the rest of Iraq ? The government that has barely existed for 1/6th of the duration of this study ? The same government that includes people from a group (the Sadr faction) that is known to control the morgues in Baghdad and has been known to undercount deaths even in Baghdad ?
Alright, so you believe a conspiracy involving hundreds or thousands of Iraqis instead of a small group of anti-war Democrat scientists. Fine. That's your prerogative. I heard Bush was behind 9/11 too. It's a cover-up. Everyone was in on it, but no one is talking. Yeah, it's that kind of conspiracy.
But I suppose if you still believe there were WMDs in Iraq, you can believe anything.
Have I said that I believe this? Ironic that you bring that up, since you are believing in dead bodies that have been proven to exist precisely as much as the "missing" WMDs.
Can you hold the opinion that calculating SMRs wouldn't matter that much and would give you the same results as unweighted rates AND believe that the Lancet studies are true?
Taking Lancet1&2 at their word we've seen a dramatic increase in birth rate post-war, the death rate increase by roughly 150% and 1 in 15 adult males die violent deaths post-war. If this is true then there have been substantial demographic shifts in age and gender ratios in Iraq over the four and a half years. This would mean crude death rates would be misleading.
It does strike me as unsporting to argue the Lancet's results undermine their methodology like that, but there you go.
Does that in itself make it valid?
I was responding to a note that said that cluster sampling could not be used except for infectious diseases. The fact that has been used in other conflict situations indicate that is a valid tool to deal with such situations.
You know, aside from the fact that the methodology in those was not the same as was used in the Iraq studies...
Oh, please. More details on how there were substantial differences instead of mysterious generalizations.
A simple stratified survey was not done here, for practical reasons.
The point is that clusters were allocated based on population. Thus Baghdad Province got 12 clusters, Basrah province 3, and so on. This is the very thing the original "statistician" claimed had not happened. It need hardly be added that given the population displacements (including a large number of Iraqis who have left the country for Jordan etc.), especially of various ethnic groups, the idea of sampling appropriate numbers of Sunni or Shia etc. is impractical.
Alright, so you believe a conspiracy involving hundreds or thousands of Iraqis instead of a small group of anti-war Democrat scientists
I postulated no such extravagant hypothesis. That may be your hypothesis. In conflict situations, no vast conspiracy is needed, what is needed is systematic undereporting, displacement, governmental incompetence and occasionally malfeasance at various levels. But tell us how the vaunted Iraqi government is a model of efficiency, impartiality and honesty, why it can maybe control 2 blocks in the Green Zone !!
The bottom line of course is that we've helped cause the deaths and maimings of countless Iraqi civilians.
The WSJ can piss and moan all they want about the gathering of the data but the simple fact is that too many of the wrong people are being killed.
When the wrong people get killed it demoralizes everyone involved, I still think of a kid that I wacked in Nam in '69, and it's kinda hard to win hearts and minds at funerals.
The editorial board of the WSJ is interested in one thing and one thing only...propping up this corrupt administration and the folks who are making money off of it.
The editorial board never reads the news section of that rag and if they did would ignore any thing that countered their preplanned opinions.
Have over a half million Iraqis died since the invasion?
Who the hell knows!
All it takes is one to push a grieving family member or neighbor or co-worker to dedicate themselves to killing Americans or engaging in the civil war.
The Wall Street editorialist is a classic chicken hawk, sqwakking about 'how can they hate us for setting them free'
when he should just think about the how it feels to hold the lifeless body of a 5 year old son in his arms. He would rather quibbel about how data is gathered than face the reality that HIS president and the Neo cons have trashed our honour and squandered our blood.
Thus, exit polls, for instance, in which every fifth or whatever person leaving is chosen ARE NOT RANDOM, as those in between have no chance of being chosen.
Everyone going to the poll has the same chance of being the fifth one in line, that's plenty random.
FARTSINSLEEP - Probably the best and most accurate comment or post that I have read on the Lancet Study. You are speaking the truth here. The US and allies lost the hearts and minds campaign the minute they stepped over the border.
As I have said on more than one occasion 1 death is bad - we have just become desensitised so that we can now say that if the figure is 'only' 30 000 then this is OK. Clearly the death toll whatever the final number, which will never be known, is horrific and is breeding a whole new generation of even more desperate and ruthless terrorists.
Consider this - the 9/11 planes that hit New York flew near the Indian Point nuclear power plant where thousands of tons of waste is stored above ground quite apart from the 4 older generation nuclear power plants. As bad as 9/11 was it would not compare to the planes crashing into this target as most of New York would have been unihabitable for many years.
Let us hope that the next gen terrorists do not resort to even more destructive attempts to gain the world's attention.
[Link](http://www.ucsusa.org/global_security/nuclear_terrorism/impacts-of-a-te…)
You seem to be a bit shaky on the concept of ad hominem. Here's a primer:
If I say, "The results of the Lancet study cannot be trusted because the editor is an antiwar activist," that is ad hominem.
But if I say, "The results of the Lancet study cannot be trusted because the editor violated accepted scientific standards of editing and peer review in the following ways: X, Y, and Z, and I suspect that he did so because he is antiwar," that is not ad hominem.
What is the difference? In the former case, I cited only the editors' alleged political beliefs as grounds for rejecting the study. That makes it ad hominem. Any argument in which "consider the source" is the only evidence offered is by definition ad hominem. In the latter case, I cited specific evidence that the editing was improper (I had to denote them by "X, Y, and Z," because in all of the criticism of the Lancet editor, I have seen not a single example of anything that he did improperly.)
Now that we are intellectually equipped to understand what ad hominem means, let's look at Tim's comments. Is he solely telling us to consider the source, or is he offering actual rebuttals ("X, Y, and Z") to Moore's arguments?
As a matter of fact, he does:
X:
Y:
Z:
So Tim is not offering Moore's status as a professional Republican-aligned spin doctor as the sole reason for discarding Moore's arguments--he is offering it as a possible motive for the misleading claims that he has documented.
Yes, but no one's addressed the critical flaw in the study. The lack of demographic data. Isn't that the 800-lb gorilla in the room?
If the WSJ piece is correct (and Moore cites specific conversations with a principal of the study) this is DELIBERATE malpractice of methodology and a total fraud!
Dr. Roberts flat-out states that they did NOT gather demographic data on their respondents and household inhabitants. Un-F'ing-believable!! My undergraduates take about 20 seconds for me to pound into them that you ALWAYS, ALWAYS, ALWAYS!! get standard demographic data, no question asked. Otherwise it's total crap.
Mr. Moore,
Thanks for signing in and offering to take part in courteous debate.
Steven Moore [left a comment](http://scienceblogs.com/deltoid/2006/10/flypaper_for_innumerates_wsj_e…) but it went into moderation so its way up the page now.
Also [see stats.org on Moore](http://www.stats.org/stories/did_wsj_flaw_iraq_oct18_06.htm).
And therein lies the rub; Is Iraq a nicely stirred soup?
"Someone's background, ideology and motivation is either fair game in trusting his work, or it isn't."
There's a difference between being an honest fanatic (for any side) and a paid advocate. I'll gladly listen to someone who is burning to convince me (well, for a while at least, if he's being reasonable!). I'm not interested in listening to someone who has it as his job to manipulate me, to change my opinions by whatever means necessary.
That doesn't mean paid advocates are always wrong, just that I think that kind of behaviour is beneath contempt, whether it's coming from the left or the right. Once you accept money for promoting a certain view, without even having as part of your job description to critically evaluate that view, then I can no longer trust that your opinions are honestly your own, or that your claims are made in good faith.
So yes, I think it's interesting when Tim Lambert finds out that a poster is working at a political PR agency.
Steven Moore wrote:
1. Are you implying that 47 clusters would result in a biased estimate?
2. In the WSJ piece, you referred to "1200% error". Are you suggesting that the true number of excess deaths over the 40 months from March 2003 to July 2006 was around 50,000? If so, what is the basis for that estimate?
3. In the WSJ piece, you wrote: "It's important that voters and policy makers have accurate information. When the question matters this much, it is worth taking the time to get the answer right." If it was as important as you say, your teams were well-trained enough to do such a survey, and you were there for two years, why didn't you do the survey?
If you are a biassed researcher determined to inflate the numbers of dead in Iraq and get published like this, you have to go through a) raising the money, finding the researchers and dealing with the safety problems and then b) create the papers and then c) get them taken on by the most prestigious medical magazine in the UK and then d) run them through a rigorous peer review process in which you cannot at all guarantee that the anonymous critics share your bent world view.
If you are a Republican shill who wants to attack the study, you have to a) go to your computer and b) convince the editorial board of the Wall Street Journal that your position undermines the Left and supports the administration and c) collect your cheque.
Notice any difference here?
Kevin, if you have a problem with Tim's post, just say why you think he's wrong. Explaining in detail how he talks to people he disagrees with proves nothing except that he disagrees with some people.
Jon,
I was responding to a note that said that cluster sampling could not be used except for infectious diseases. The fact that has been used in other conflict situations indicate that is a valid tool to deal with such situations.
That's akin to saying that since a fork has been used by others to eat soup with before, that makes it a valid tool to eat soup with.
Oh, please. More details on how there were substantial differences instead of mysterious generalizations.
Why don't you just go read the methodologies before acting like a smart-ass? I promise, reading won't hurt. The only thing that is mysterious is your incuriosity for the truth.
The point is that clusters were allocated based on population. Thus Baghdad Province got 12 clusters, Basrah province 3, and so on. This is the very thing the original "statistician" claimed had not happened.
I'm not sure that's what he said, but I'm confused as to which statistician we are talking about. Stratified sampling is not the same as a cluster sampling, although the JHU team used a combination of both. Now, the problem with this is that when you have too few clusters, you are going to zero in on urban areas and will most likely not get any rural areas. If we just get a list from the JHU team over the sites they visited, I'm pretty certain this will be obvious.
It need hardly be added that given the population displacements (including a large number of Iraqis who have left the country for Jordan etc.), especially of various ethnic groups, the idea of sampling appropriate numbers of Sunni or Shia etc. is impractical.
Carrying out a cluster survey in Iraq to determine mortality is by itself "impractical" so I'm not sure why getting demographics for the sample would be any more impractical. Demographics are always used to ensure that the sample is representative of the population being sampled. The reason why they didn't do so is quite obviously because their sample is not representative of the population of Iraq. Saying that gathering demographic data is "impractical" is basically saying that, ah screw standard survey methods, we're doing this damn thing anyways.
That may be your hypothesis. In conflict situations, no vast conspiracy is needed, what is needed is systematic undereporting, displacement, governmental incompetence and occasionally malfeasance at various levels. But tell us how the vaunted Iraqi government is a model of efficiency, impartiality and honesty, why it can maybe control 2 blocks in the Green Zone !!
No doubt that there will be underreporting, lost records, incompetence, etc in such a situation. But for 500,000 death certificates to make like vapor is, shall we say, horseshit.
Controlling thugs with guns and bombs has no relevance on the ability to collect and maintain health records. That you would even bring one up to refute the other smacks of the desperation you face in the fact that you know the Lancet survey is full of it.
FARTSINSLEEP,
Great post. Its painful to see the war party supporters and genocide apologists here grasping for every straw they can find to feebly try and prove that the invasion of Iraq and subsequent disintigration of Iraqi society han't really killed that many Iraqi civilians.
When retired senior CIA official Ray McGovern said that those effectively running US foreign policy now - the likes of Perle, Feith, Rumsfeld, Cheney, Armitage, Abrams, Kagan, Bolton etc. - were routinely referred to as the 'crazies' in internal US diplomatic circles in the 1990's, he wasn't joking. Regardless of what the imperial supporters like Seixon, KevinP and their brethren here say, the 'crazies' DID NOT GIVE A DAMN ABOUT THE POTENTIAL COST OF THE IRAQ INVASION TO CIVILIAN LIFE IN THE COUNTRY (my emphasis added). All of the agenda is laid out clearly in the documents (1) Defense Planning Guidance (1992), (2) A Clean Break (1996), and (3) Project for a New American Century (2000). These docments authored by many in the 'crazies' ranks, spell out clearly the desired global economic and military agenda of the United States. The agenda was one of aggressive Wilsonianism - free market absolutism and predatory capitalism, the kind that has routinely devastated societies in Latin America. Before KevinP tels me to 'chill out' again, I suggest he learns a little about the economies in Latin America that William Casey refereed to as an 'example of the success of US interventon'. Poverty levels in the 'successes' he refers to - Guatemala, El Salvador and Nicaragua - are in excess of 50%, with Nicaragua and Guatemala being the second and third porest nations in the western hemisphere (after Haiti). Not surprisingly, these three countries are the most that have been at the receiving end of US military and economic intervention over the past 50 years. And Dick Cheney claims to want to model the Iraqi 'democracy' on El Salvador, another country with a devastated economy and close to 50% poverty.
The denialists here can pick away at what they perceive to be methodological flaws in the JHU study, but they are merely a distraction from the main two questions: has the US invasion of Iraq led to the unnecessary deaths of at least tens of thousands of Iraqi civilians, and DID THE CRAZIES EVEN CARE ABOUT THIS POSSIBILITY BEFORE THE INVASION? The answer to the first question is invariably YES, and to the second one is invarably NO.
Jeff,
Ray McGovern is the one floating conspiracy theories that the US is preparing to attack Iran. Ray McGovern floated the conspiracy theory that Bush would fire Patrick Fitzgerald for indicting Rove and Cheney (none of which happened). The only "crazies" are guys like McGovern who live in a fantasy world of their own making.
The denialists here can pick away at what they perceive to be methodological flaws in the JHU study, but they are merely a distraction from the main two questions: has the US invasion of Iraq led to the unnecessary deaths of at least tens of thousands of Iraqi civilians, and DID THE CRAZIES EVEN CARE ABOUT THIS POSSIBILITY BEFORE THE INVASION? The answer to the first question is invariably YES, and to the second one is invarably NO.
The distraction here is studies like the one in the Lancet which attempts to silence war supporters into submission by its sheer force of numbers.
Yes, the invasion of Iraq has led to the death of at least tens of thousands of Iraqis. So did the defeat of Hitler and the Axis Powers in WWII. So did the invasion of Kosovo. So did many other wars.
As for whether the "crazies" cared about this before invading: you'll have to ask them. You'd also have to ask every general in every war whether he cared that his mission would result in the deaths of innocent people. The answer is usually yes, BUT that the mission was justified nonetheless.
Iraqis in many polls have said that the invasion, even with all that has happened since, was worth it to remove Saddam Hussein and his regime.
I guess if you're not willing to listen to the opinion of the people affected by this war, you're not likely to listen to anyone about it.
Seixon,
Good post. However, Ray McGovern isn't the only one 'floating' theories about the aggressive intents of the Bush regime on attacking Iran. Many in the intelligence community are also doing so. The real prize for the 'crazies' is to set in motion regime changes acrss the Middle East, placing into power pro-Israel/US regimes that will do as they are told. There is nothing exceptional in this with respect to historical precedents (e.g. Latin America), except that the consequences for the region and for the world of what amounts to nakedly predatory expansionism could be profound.
The only reason I feel that the crazies have not attacked Iran already (and perhaps Venezuela as well) is that Iraq has turned into a complete disaster for the war planners. The country is dying. I know you and the other apologists for US unilateral aggression don't care about the validity of international law, the UN charter and the Nuremburg code, but most others in the world do, and that is why the US is now seen by quite a wide margin of populations in most countries as the greatest threat to global peace and security.
Bear in mind that Ray McGovern is a good friend of George H.W. Bush (the elder) and that even Bush Srs. National Security adviser, Brent Scowcroft, hardly an exteme voice, distanced himself from the ravings of Wolfowitz, Cheney etc. in their first document, "Defense Planning Guidance" (1992). This document more-or-less paves the way for future US aggressive wars that would be effectively fought to ensure that the US remained the global hegemon.
Most importantly, all of this discussion of the civilian death toll in Iraq - whether it is 100,000 or 655,000 - is a red herring. The current US regime - crazies included - DOES NOT CARE ABOUT THE LIFE OF IRAQIS or anyone who stands in the way of their military and economic agenda. It isn't that they actually set out to slaughter Iraqi civilians, they did't, but it was considered to be a price worth paying to attain the bigger objectives. Moreover, previous incumbents didn't care are about the so-called sanctity of human life either, but at least they tucked away their predatory intentions in their pocket and used it when necessary. The current bunch wear it on their sleeves. I have read enough planning documents (some of which are included in my last post) as well as the National Security Strategy to realize that 'democracy promotion' was never considered by the crazies when they were planning for regime change in Iraq. Check up on documents prior to the invasion (2002) and you will find that democracy promotion was not even indexed.
In summary, you and the other apologists can believe what they like about US global intentions, but the historical record as well as numerous documents should spell it out fairly clearly. These are dangerous times and the world's dominant superpower - by now attaining the status of empire - is being run by a bunch of truly nutty outlaws. The horrific consequences of their actions should not be underestmated.
Seixon is correct when he writes that Iraqis in many polls have said that it was worth it. However, those polls were conducted by Steven E. Moore. Maybe he had someone who'd passed first year stats to design the surveys for him. In email I suggested he consult with whoever helped him design the surveys. Maybe we'll get a correction.
Tim Lambert:
Oh good God. This is the kind of arrogance that comes from the academic sitting comfortably in his chair that reminds everyone of the ever true phrase "ivory tower". By his own admission, Steven Moore has conducted some 20 surveys in Iraq, about 20 more than anyone on this thread. But apparently, he needs someone with a first year stats class to design the surveys for him.
Jeez. In all my years of reading Tim Lambert, I have known him to be extraordinarily obstinate about his point of view, but this takes the cake.
Have fun, y'all. Continue to live in your alternate reality.
KevinP,
Who is living in their own jaded reality? In spite of bags of evidence to the contrary, it seems that you wish to believe that the aggressors invaded Iraq out of benign intent. From your posts, there's no evidence that you have any kind of historical knowledge of US foreign policy, other than the stuff you've seen on CNN, MSNBC or read from William Shawcross of Charles Krauthamer.
A poll in Iraq in the summer of 2004 (by the Brookings Institute I believe) asked the question: why did the United States invade the country? There were Iraqi's who believed that the US had invaded to bring democracy: 1%. Almost 70% said that they had invaded to control the country's resurces, and 62% sggested that even if they got a democracy, it would only be one that the US controlled. Sure, I am not surprised that most Iraqi's are glad to see Saddam gone. But this is no way justified an aggressive war in search of an alternate agenda. I am sure the majority of Iraqi people would have been happy to see him gone in the 1980's when he was fully supported by the same recycled bunch of crazes that are in White House now in full knowledge of his crimes. I am sure that most people in Indoensia would have been happy to rid themselves of Suharto, one the latter half of the 20th century's biggest mass murderers and torturers (worse than even Saddam if that is possble) who was fully backed by Washington right up until almost the very end. I am sure that the majority of Zaire's poplation wanted to see the back of Mobutu Sesu Seko at a time when he was being armed and supported by the US. George Schultz called Nicolai Ceaucescu a 'good communist' and his regime was backed almost until the end by the planners in Washington. Add to the list Habre, Pincohet, Armas, Montt, Marcos, the Shah, Somosza, the Duvaliers, and many other gangs of extraordinary mass murderers, and its clear that there are 'good killers' and 'bad killers' in the lexicon of US foreign policy.
"Jeez. In all my years of reading Tim Lambert, I have known him to be extraordinarily obstinate about his point of view, but this takes the cake.
Have fun, y'all. Continue to live in your alternate reality."
Well, Mr. Moore has commited heresy. He has questioned the Lancet study! He must be discredited and disgraced by all means necessary. The fact that his survey methods even for a little opinion poll were likely far more thorough than the Lancet method which is now supposed to rewrite the entire toll of the war (the whole world has been wrong, and everything you've every seen from Iraq has been wildly wrong, except for this study!), is irrelevant.
He is an enemy of the Party and must be dispensed with.
Mr Moore, in response to [your comments](http://scienceblogs.com/deltoid/2006/10/statsorg_on_the_lancet_study.ph…)
>First, it is "Gorton Moore" rather than "Gordon Moore," as you post. I guess that would make your posting "illiterate" rather than "innumerate." ;)
Thanks. I fixed it. Can we expect to see a correction to your WSJ piece?
>Second, the key point about the demographics is that they did not compare the results to the 1997 Iraqi census. Les Brown, one of the authors, told me in an email that he had not even SEEN the Iraqi census.
Les Brown? I think you mean Les Roberts. They used the 2004 UNDP/Iraqi Ministry of Planning population estimates which seem liekly to more up-to-date than the 1997 census.
>Third, we used 75 cluster points of 20 interviews each for our surveys of six cities in Iraq, with a sample of 1500. For a 33% larger sample, the JH team used 33% fewer cluster points, and for a nationwide survey. Considerable difference. That being said, it is almost always prefereable to use more cluster points. As Iraqi fieldwork capacity increases, more robust survey methods are used. Currently, most surveys in Iraq use hundreds of cluster points.
What? Your survey wasn't a national survey? Your sure made it sound like it was in your LA Times piece and the website you set up to campaign for Bush. And yes, more clusters are better, but not that much better. The standard error is inversely proportional to the square root of the sample size, so 33% less clusters means only a 15% increase in the standard error. This is not a big difference and does not come anywhere near to being a reason to dismiss the study.
>That being said, I've done about 27 surveys in Russia, around 7 in Romania, half a dozen or so in Indonesia and one in East Timor, in addition to twenty or so in Iraq. I've spent almost two years in Iraq doing just this, and there are many talented and credible Iraqi organizations doing better survey research than the JH team.
So how come you never thought to add a question about mortality to any of your surveys? How come these credible organisations have never surveyed mortality? It seems that JH folks are the only ones interested. How come?
Funny that, with surveys. I made the mistake of saying "OK" to a surveyor from AC Nielsen on the phone. Since that I've recieved huge catalogs of push-poll-like catalogs of checklists that they want me to fill out, about what brands i think are the most dynamic, whether I feel especially young when using this or that shampoo and so on. I suppose the selection effects of someone actually filling out all of that willingly (it takes an hour at least, plus 10 minutes each day for a week just for the paperwork) for immensely little reward... make the surveys pretty useless for anything but marketing.
There's a huge difference between just any survey and a peer-review epidemiological study. But then again, marketing is basically the field Moore is working in, isn't it?
I have another couple of questions for Steven Moore:
1. What was the topic of the surveys you have done in Russia, Romania, Indonesia, East Timor, and Iraq?
2. In Iraq, what was your non-response and refusal rate?
This is the kind of arrogance that comes from the academic sitting comfortably in his chair that reminds everyone of the ever true phrase "ivory tower".
Spoken like an angry man who wasn't good enough, and is now jealous of others who are.
Best,
D
He is an enemy of the Party and must be dispensed with.
Projection.
Best,
D
Tim, the fun part about engaging with your blog is each of your replies gets to be more arrogant and bizarre than the last.
You keep talking about "my employers" and "my campaigning..." I'm very up front about my political leanings. Can Dr. Roberts say the same? We've skirted around the edges of Dr. Roberts' Congressional bid, above, but just to be clear, Dr. Roberts was a Democratic candidate for Congress in New York during this election cycle, but dropped out of the race. http://www.thatsmycongress.com/lesroberts.html
Not being a member of the scientific community, I'm not sure of the origins of the "conflict of interest" statement on the JH study at the end of the piece just before the Acknowledgements. It simply says under the heading of "Conflict of Interest Statement" the sentence "We declare we have no conflict of interest."
Wouldn't publishing a highly controversial study such as this three weeks before an election be a conflict of interest worth mentioning if you were a candidate for office who was taking campaign donations?
Perhaps someone can enlighten me on the ethics behind this.
Stephen is there much chance of you finding time to respond to the quite serious, substantive criticisms of your WSJ attack on the scientists' work? Of any answers to Robert's and Tim's fair and balanced (I'd have thought) questions just above?
Good Lord!
Are you all that frikking far removed from reality?
The Lancet study numbers are little more than a foot note as to the REALITY OF THE MOMENT!
DO I NEED TO CONTINUE TO SCREAM AT ALL OF YOU?
BUSH AND THE NEO-CONS LIED AND LIED AND LIED!
THEY WASTED THIS COUNTRY'S HONOUR AND REFUSE TO TAKE RESPONSIBILITY FOR THEIR MISTAKES.
WORSE YET, THEY HAVE NO PLAN TO GET US OUTTA THIS DRUNKARD'S MESS!
FROM THE GREAT MASS OF UNWASHED CANNON FODDER, 'A POX ON BOTH YOUR HOUSES'!
Now go back to crunching your numbers and soon enough this despotic ass in the Whitehouse will be sending someone around to pick you up...'intellectuals' are always the first against the wall.
I'll answer some of the questions... It was not my intent to not be forthcoming. I just have been enjoying the craziness of the debate here. It has been a while since I have been on the blogosphere, and I don't recall any quite as... spirited as this one.
Steven Moore wrote:
That being said, it is almost always preferable to use more cluster points. As Iraqi fieldwork capacity increases, more robust survey methods are used. Currently, most surveys in Iraq use hundreds of cluster points.
1. Are you implying that 47 clusters would result in a biased estimate?
This study has so many flaws, that I simply didn't have the space to address them all in the WSJ op-ed. 47 cluster points is fewer than any survey of its kind I have ever done, or any that I could find, other than the 2004 Johns Hopkins survey, which had 33. I understand that sometimes you have to do "cheap and dirty" surveys, but I think it irresponsible to release such a survey nationally on the eve of an election. I probably wouldn't have spent the time researching and critiquing this study, based solely on the small number of cluster points, if my first look at the study had not revealed so many additional flaws.
A) The survey team had done exactly one survey prior to the 2006 survey, and that was some 24 months prior. Essentially, the survey team was a first time team. Having trained many teams in Iraq, it is difficult for a first time team to get a good sample.
B) According to Dr. Roberts, none of the study's Western authors felt accuracy in the survey results important enough to show up in Baghdad to actually train the survey team in 2006. While they might cite security as a concern, there are thousands of very brave journalists, NGO workers and other Western civilians who think their work is important enough to show up in Iraq to do it. If I have misunderstood this, and Dr. Roberts, Dr. Burnham or the other Western authors did go to Iraq in 2006, I offer my most sincere apologies in advance.
C) The interviewers were all from Baghdad. All of the Iraqi survey organizations of which I am aware have stopped using people from outside the communities they survey because it increases the non-response rate, and in some areas it endangers the surveyors.
D) The interviewers were presented as doctors. However, the medical community in Iraq, particularly at al Mustansiriyah, is perceived to be part of the Sadr organization. Here is an article talking about it
I'm not sure how well the Mehdi Army is received as a survey team.
E) The JH team, according to Dr. Roberts, did not collect demographic information, which is the only way anyone could prove conculsively that their sample is not representative. As I have mentioned frequently, this is a basic quality assurance method on surveys and its absence is highly suspect.
F) Roberts didn't look at the 1997 census or the 2003 update to the census. He dismisses these documents in one of his posts as not being useful due to migration, but his own survey shows a 7% rate of in migration, and an 8% rate of out migration. Doesn't seem like enough migration to render the documents unusable.
G) The 99.2% rate of cooperation is higher than any survey I've seen in Iraq. 80-85% is a rough average of the cooperation rate between 5 Iraqi survey organizations with which I have familiarity.
H) On the common sense front, it seems bizarre to me that such a large number of people could be dying in Iraq, yet the Iraqi bureaucracy is dutifully churning out death certificates for 92% of the deaths.
This list is not exhaustive, but my fiancé is a little tired of seeing me on my computer, and these are the ones that come to mind immediately.
I lack the academic language to describe the various places where error can creep in when methodology is implemented in such a way. Perhaps someone can fill in this description.
2. In the WSJ piece, you referred to "1200% error". Are you suggesting that the true number of excess deaths over the 40 months from March 2003 to July 2006 was around 50,000? If so, what is the basis for that estimate?
It was a rhetorical device, based loosely on the count at Iraq Body Count. If I make my own estimate, it will be apparent.
3. In the WSJ piece, you wrote: "It's important that voters and policy makers have accurate information. When the question matters this much, it is worth taking the time to get the answer right." If it was as important as you say, your teams were well-trained enough to do such a survey, and you were there for two years, why didn't you do the survey?
Because, despite what our kind moderator Tim might say, I spent the last year in Iraq doing surveys in order to get information that would be helpful in improving the situation of the Iraqi people and saving lives among the both the Coalition Forces and Iraqis. I was not conducting surveys to use the results in the US elections, as the JH team apparently was. In 2004, I used existing surveys to illustrate the situation in Iraq statistically, but that was a second hand use for the results of surveys conducted for use in helping the Iraqi people.
A better question is, if the Johns Hopkins team was interested in getting an accurate result, rather than having fodder to advance their cause in the US elections, why didn't they use one of several highly qualified Iraqi organizations for fieldwork, rather than an inexperienced team from an organization that is thought to be a biased Shia organization? I could extend that argument to any of the problems in fieldwork discussed above.
Moore said "75 qada (the Iraqi equivalent of precincts) would be chosen at random, with interviews conducted in 20 randomly chosen households in each."
75*20 = 1500 independent sample points.
The Hopkins study had only 47 independent points.
Les Roberts dropped out of his political race months ago, iirc. Anyone interested in this line of attack could google for the information. I'm not, so I haven't.
Third, we used 75 cluster points of 20 interviews each for our surveys of six cities in Iraq, with a sample of 1500. For a 33% larger sample, the JH team used 33% fewer cluster points, and for a nationwide survey. Considerable difference. That being said, it is almost always prefereable to use more cluster points. As Iraqi fieldwork capacity increases, more robust survey methods are used. Currently, most surveys in Iraq use hundreds of cluster points.
same question again Mr. Moore:
was your poll for this article taken in 6 cities only?
http://www.gortonmoore.com/truth_about_iraq.html
why didnt you answer this question?
[support for the lancet study](http://www.theage.com.au/news/opinion/the-iraq-deaths-study-was-valid-a…)
Mr Moore:
(1) Your own article referred to a 50 cluster sample in Kosovo. 30 cluster samples are recommended by WHO. You can find [hundreds of examples](http://www.google.com/search?num=100&hl=en&lr=&client=firefox-a&rls=org…) if you look.
(A) Burnham states "The survey team were experience in community surveys" -- it was not their first survey.
(B) Do you have a point here? Riyadh Lafta was in charge of the field work. Do you have any reason to think that he is incompetant?
(C) They got a very good response rate, so your point is not relevant.
(D) Nice bit of equivocation. The Ministry of Health is run by Sadr and he's but his own people in there. But no-one thinks that all doctors work for the MoH. And what effect is this supposed to have in any case. The survey teams were, in fact, well received.
(E) They did collect demographic information and Roberts told you this. He called your claim to the contrary a "fabrication".
(F) He used the 2004 population estimates. Do you have some complaint abbout these?
(G) Many other surveys in Iraq have obtained high reponse rates. [See here](http://crookedtimber.org/2006/10/18/floating-the-fraud-balloon/)
(H) Why is this bizarre? How long do you think it takes to produce a death certificate?
(3) You write "I was not conducting surveys to use the results in the US elections". But you did use the results of your surveys to campaign for Bush.
Steven Moore wrote:
Does that mean you are saying that 47 clusters would result in a biased estimate?
Could you describe your academic background? Are you a statistician, demographer, or epidemiologist by training? When you write that you have "done" these surveys, could you clarify what your role was?
What other rhetorical devices did you use in your article?
It is sad to read Mr. Moore confuse the usual practice of survey organizations he employs with statistical science. That is not to disparage his contractors. But usual and convenient practice is rarely universally optimal. And it is so evident that Mr. Moore's expertise does not include the comprehension of the statistical techniques, quite sophisticated these days, by which to estimate multiple ways from the data collected. These estimates of uncertainty are not just theoretical, they are done from the data themselves,and are now routinely estimated under different testable assumptions about the sample design. Would that Mr. Moore turned down client billing in proportion to the uncertainty (ie survey error) attendant to the random digit dialing telephone interviews most surveys employ. No clustering there, though. It should concern his clients more that he doesn't understand the statisical principles behind sampling, and can scarcely be relied on to be a careful and competant steward of their interests. In this, I'd give Mr. Moore credit for resembling President Bush. Business Card:
GOOD INTENTIONS PAVING COMPANY
We did the road to Hell.
The concern I'd have over the Iraq epidemiological work is the short time for quality control and error checking. I'll trust the integrity of the eight teams of interviewers, but I would rather have known there was ample time to search for and investigate anomolies. Errors DO occur in data collection and procesing. It is ironic that some of the less well educated commentators on this study have insisted on some terrible advice,and teach it to students. Fortunately, it will be ignored by professionals. For example, highly stratified surveys can generate wildly varying sampling weights,and post-stratification adjustments can and have blown up statistical uncertainty about estimates for the total population for the sake of less important additional precision about a few subpopulations. (Examples of this include a large annual survey on illicit drug use, in which one respondent received a sample weight of over 2 million. Whether she reported being a user changed the point estimate of the adult US population by over 1% in absolute terms. Public and political opinion about progress in that other never ending war, the one on drugs, depended on whether that single sample respondent granny sipped cough syrup.)
To the critics of the latest Iraqi mortality estimates: 1. Has the calamity been much less if true additional mortality was at the lower bound of the confidence interval? 2. Would you raise just $300,000 from your war-supporting friends, and wager the value of your homes that a larger, more expensive survey, with more clusters you insist on, would generate excess death estimates whose confidence interval did NOT include the lower bound of the JH School of Public Health estimate? No, I didn't think so.
Finally, It is unfortunate that the study findings are so irresponsibily attacked that some more careful questions have not been asked in the press. 1. Surely, the upper bound estimate of excess deaths is not credible. Have the epidemiologists had time, opportunity, and outside data with which to shrink the upper bound estimate? 2. What are alternative estimates of excess deaths when different amounts of confidence is given to the estimated baseline? And how much of the broad confidence inteval on the estimated is due to statistical uncertainty in the baseline?
"The notion that the opponent might disagree with Tim, because, I don't know, he might sincerely think that he is right, appears foreign to Tim."
Didn't you just say
"Whoever disagrees with Tim's point of view must be one or more of:
Clueless
Innumerate "
etc.?
TL:
" If I think soemone is a liar, I will say so, and explain why. I do not accuse everyone who disagrees with me of lying. Here's a challenge for you: in well over a thousand posts, how many people have I accused of lying?"
Kevin P:
"read the whole comment thread from the beginning, before leaping into a conversation already begun. Tim challenged me to locate, in a thousand posts, one where he had called someone a liar. "
I dunno, I'm finding somebody I'd call a liar pretty quickly....
"Clusters are much, much harder and that much harder to understand (and thus easier to fudge and slip past those who should otherwise know better.)"
Not really. The cluster takes the place of the individual sampled in the study; the number sampled per cluster is just a means of sharpening up the individual points a bit. I.e., the 47 clusters with 12000 individuals are closest in spirit to 47 individuals sampled, the 240 individuals per cluster is just a means of damping down the variability.
And did you miss the part where they DID weight by population? In both surveys?
Fun hypothetical: if the survey came up with 20,000 excess deaths instead, try and figure which of the debaters would switch sides regarding criticism and support.
Speaking of which, in all the debate thus far, I haven't seen any reference to the papers which were referenced in the debate over the original Hopkins study, which demonstrated that too few clusters are biased towards lower estimates?
Since I seem to be the only one putting any time or thought into postings, and my fiance is getting tired of it, I think I will bid this group farewell.
Robert, here is a community college guide go grammar so you can better recognize rhetorical devices.
http://grammar.ccc.commnet.edu/grammar/
Best of luck!
Steven Moore wrote:
Completely understandable. It must be really quite taxing keeping your story straight.
Since I seem to be the only one putting any time or thought into postings, and my fiance is getting tired of it, I think I will bid this group farewell.
hm.
i actually have some doubts that you put thoughts into this reply:
D) The interviewers were presented as doctors. However, the medical community in Iraq, particularly at al Mustansiriyah, is perceived to be part of the Sadr organization. Here is an article talking about it I'm not sure how well the Mehdi Army is received as a survey team.
the only other information that we have about deaths come from hospitals (and that means doctors) as well.
so your logic is flawed in several ways at once:
1. not all doctors belong to sadr
2. iraqi official numbers come from that very doctors.
-------------
anyway, as you decided not to answer my question, here s the final point on the subject:
you made a poll in 6 cities only, claiming that it was a nation wide poll. you didn t give any information on this pretty screwed up methodology in your article.
compared to such fraud, every attack that has been made against the lancet study looks insignificant.
Moore attempts a pouty zinger:
"Since I seem to be the only one putting any time or thought into postings, and my fiance is getting tired of it, I think I will bid this group farewell."
Um, Steve, Tim asked you a bunch of questions you decided not to follow up on. You're the author of a fairly weak attack on a peer-reviewed study. Are you accusing the Lancet of gaming the peer review process in order to publish an ideological study? If so, that sounds like grounds for a law suit.
Translation:
"Since my AEI-funded butt is getting handed to me, I must hie back to the Heritage Foundation to have a cocktail."
Steven Moore, I have to say it - oh, good frickin' god!!!!
Lets look at your answers:
"1. Are you implying that 47 clusters would result in a biased estimate?"
You spend about a dozen column inches avoiding answerign this question - you never address it. Were you hoping that no one would notice that your 'I wouldn't have raised this if not for other issues' sidestepping of the issue is not actually an answer to the question?
Your subsequent lettered obfuscation is absurd even on its own merits. Many of the points amount to "I don't beleive they did a good job." Some of your claims are false, and are repeating falsehoods that have been countered before. These include 'they didnt collect demographic data" (false), your implication (not even a real claim) that the fact they were from outside the community damaged the response rate (clearly false) when elsewhere you attack them for what you have argued is an unbelievable high response rate - when the respoonse rate is in line with many other samples in Iraq. You imply they didnt use relevant census data (false), when in fact they used more recent data that they felt was higher quality than the data you attack them for not using.
If I may parrot you "This list is not exhaustive, but... these are the ones that come to mind immediately."
You say, "Perhaps someone can fill in this description." Let me try - What you're doing is engaging in obfuscation, Steven. Stop it.
On point 2, are you claiming that a statistician (yourself) would make an unqualified quantitative claim about the magnitude of an alleged error, and NOT actually be referrring to either the actual quantitative value you stated or to the actual error you claimed? That wasn't a rhetorical device - it was a bald claim.
Point 3 is more obfuscation, but even worse, it is making a bald unsuported claim that the study is political, and it implies, without support, that the numbers are politically derived rather than derived from the survey, and that the survey must be wrong because they didnt use the survey teams you wanted them to use.
Bottom line, Steven - your refusal to actually address the actual criticisms of what you said,coupled with the fact that you get so many facts flat-out wrong, renders everything else you say suspect.
Finally, you announce that you are leaving because you are the only one putting thought into this. I will charitably assume that you are msitaken in your belief that you are putting thought into this - because the alternative is to conclude that you *were* putting thought into it, and are intentionally stating mistruths.