This post is some more notes on a reply to the badly flawed “Main Street Bias” paper.
The authors claim that it is plausible that the Lancet paper’s sampling scheme could have missed 91% of the houses in Iraq. (That is, their parameter n, the number of households in the unsampled area divided by the the number in the sampled area could plausibly be 10 or more.) The only support they offer for this is a reference to this analysis of Iraqi maps.
To the right is a detail from their map. The red lines are main streets and the yellow are secondary streets. They assert that the blue areas are not samplable using the Lancet scheme, and yes, the blue area covers 90%+ of households. But there are two things wrong with their map.
Look at this larger scale version of their map:
First have a closer look at their yellow road that they say should not be counted as a main street and compare it with the one they count as a main street. (Scroll around on Google Maps if you want to see more of each street.) It’s just as wide and has a similar amount of traffic and a similar number of large buildings. Clearly it should have been classified as a main street and it is not even slightly plausible that someone who was trying to get every house in the sample frame would leave it out.
Second, the random house on the secondary street is only the start point for the cluster, which also includes 39 houses neighbouring the start point. This means that you can sample houses on tertiary streets that are a few houses from the secondary street. I added two obvious main streets to the ones they chose (the one across the bottom of the map above and one that it is off the map above), and redrew the blue areas to taken this factor into account.
What’s that? You don’t see any blue areas in my map? That’s because there aren’t any. Make just these two corrections to their map and the unsampled area is 0. In their model, that means n=0 and there is no main street bias.