Burying the Lancet

MediaLens has a two part article (part 1 part 2) on the shoddy press coverage of the Lancet study. They describe how Mary Dejevsky, senior leader writer on foreign affairs for the Independent dismissed the study because:

personally, i think there was a problem with the extrapolation technique, because - while the sample may have been standard for that sort of thing - it seemed small from a lay perspective (i remember at the time) for the conclusions being drawn and there seemed too little account taken of the different levels of unrest in different regions. my main point, though, was less based on my impression than on the fact that this technique exposed the authors to the criticisms/dismissal that the govt duly made, and they had little to counter those criticisms with, bar the defence that their methods were standard for those sort of surveys.

Whether the sample size is adequate is a technical question. Your lay intuition is not going to be a good guide to whether it is or not. You have to do the calculations. Or if you don't know how to do that, get an expert to do them, or look at what experts have come up with in the past. And you can find out what they have come up with in the past by looking at what the standard practice is for conducting such surveys. So the author's defence was complete. Unless you were prepared to believe the government spokespeople had uncovered a hitherto unknown flaw in survey methodology just when it happened to be politically convenient.

Medialens also examines the very different coverage given to an earlier survey by Les Roberts that found that war in the Eastern Congo had killed about 1.7 million people. Even though this was many times the number of deaths their later survey in Iraq found, the media reported the results in a straightforward fashion, without all the stuff about how the sample was too small and the estimate could not be trusted that greeted the Lancet study.

This, by the way, is my 57th post on the Lancet study. Read them all here.

Hat tip: Antony Loewenstein.