If you measure the wrong thing, you get the wrong answer: Down's syndrome in Britain

One of the blogs I read regularly is Ben Goldacre's "Bad Science". I recommend
it highly. (Which reminds me that I really need to find some time to update my blogroll!) In saturday's entry, he discussed a BBC Radio documentary that described how Britain is becoming a much more welcoming place for Down's syndrome babies.

Ben did a good job of shredding it. But I also wanted to take a stab, focusing on
the mathematical problem that underlies it, because it's a great example of two very
common errors - first, the familiar confusing correlation and causation, and
second, using incorrect metrics.

There's one basic fact underlying this discussion: in England and Wales, the
number of live births of Down's babies has risen by about four percent between 1989 and 2006.

From this, the BBC's documentary concluded that Great Britain has becoming more welcoming to Down's babies. How do they conclude this? Well, since pre-natal testing
was first introduced, the number of Down's baby live births steadily decreased - until this most recent measurement, when the number started to trend upwards again. Based
on this, they assume that it's clear that more parents are deciding to
have Down's babies.

Right there, you can see the correlation/causation problem. The number of
Down's babies being born has increased; the main factor that reduces the number
of live Down's births is abortion; therefore, the increase must be caused by people
choosing not to have abortions.

What's wrong with that? To answer, we need to look at what other factors
are involved in this. First and foremost, what's the absolute number of Down's babies being conceived? If that rate has increased, then the number of live births could
increase without the rate of abortion changing at all. Unfortunately, we don't have
any real numbers for that; it's not (as far as I know) reported publicly.

But we can try to infer it, by considering what causes Down's syndrome. Down's
is a genetic disorder where the mother's egg cell (it's almost always the mother's
genetic contribution that causes Down's) ends up with an extra copy of chromosome 21,
so that when it's fertilized, the cell has three copies of that chromosome. That causes a large number of developement abnormalities; people with Down's have distinctive facial features, tend to be severely retarded, have very thick tongues, and a host of other things. But what causes the egg cell to screw up during
division and wind up with an extra chromosome 21? We don't know - but we know that
it correlates very strongly with the mother's age. A 25 year old mother has
a risk of about 1 in 1600 of conceiving a Down's baby. At 35 years old, the risk rises to around one in 350. (The reason why most doctor's only recommend amniocentesis for
mothers 35 or older is because that 1 in 350 is the crossover point where the likelihood of the baby having Down's corresponds to what most potiential parents
consider an acceptable level of risk to the fetus from the procedure.)
(Note: in the original post, I screwed up and said Down's was related to chromosome 23; it's not 23, it's 21. Thanks to commenter Alex for pointing that out!)

The population of women having children in Great Britain has changed somewhat since 1989 - it's decreased slightly; not dramatically, but slightly, because people on average are having fewer children. That should, arguably, decrease the number of live Down's births, since the overall pool of pregnant women has decreased. But the other factor that needs to be considered is the age of the women getting pregnant. And that has changed quite dramatically. Since 1989, the percentage of
women getting pregnant who are 35 or older have increased by two and a half times - from around 6% to around 15%!

The upshot of this is that the number of Down's pregnancies has almost certainly increased quite dramatically.

The underlying error here is that the BBC documentary confused the number of live births of Down's babies with the percentage of Down's pregnancies that ended with live birth. Their argument is that the increase in the number of
live births has increased, therefore more parents are deciding to have Down's babies. But they're using the wrong metric. The absolute number of people deciding to have Down's babies has increased - but that doesn't mean that "more people in society" are deciding to have Down's babies. When you look carefully at the numbers, it actually
appears that there's been an increase in the percentage of Down's babies
being aborted: the increase in number of live births is smaller than
the increase in number of Down's pregnancies.

So in fact, more parents are still choosing to abort Down's babies - continuing the existing trend; as testing becomes more available and less risky, more
parents are opting to test, and to abort if they find out they're carrying a Down's baby. The rate of abortion for Down's pregnancies has actually increased, not decreased.

This is what happens when you don't really understand what you're comparing.
To do a meaningful comparison of a trend, you need to understand what the actual
things being measured are, and make sure that you're using a metric that
meaningful measures the quantity you want to discuss. The BBC documentary fouled
that up horribly: they looked at some data, chose an invalid measure for it,
and then using that measure, drew a conclusion based on a faulty correlation,
and used that faulty correlation to infer an invalid cause. Bad math on multiple
levels.