Good Math, Bad Math

My good friend and blogfather, Orac, posted something yesterday about animal testing
in medical laboratories. I’ve been meaning to write something about that for a while; now
seems like a good time.

I’m not someone who thinks that being cruel to animals is no big deal. I have known
some people like that, but thankfully they’re very rare, and none of them
were scientists whose work involves doing animal testing in real laboratories.

But animal testing isn’t about pointless cruelty. It’s about understanding things that we
simply cannot learn about in any other way. It’s extremely important to minimize the
cruelty we inflict on the subjects of animal tests: there’s no benefit in torturing an animal;
there’s no good reason to inflict unnecessary pain on any living thing. When we
can study something without using animals, we should. When we must use animals
in scientific study, we should be as kind to them as we possibly can. But the fact remains that in many cases, animal studies are necessary.

I don’t want to get into a long discussion of the ethics of it here; that’s a discussion
which has been had hundreds of times in plenty of other places, and there’s really no sense
repeating it yet again. But there is one thing I can contribute to this discussion. One of the
constant refrains of animal-rights protesters arguing against animal testing is: “Animal
testing isn’t necessary. We can use computer simulations instead.”

As a computer scientist who’s spent some time studying simulation, I feel qualified
to comment on that aspect of this argument.

The simplest answer to that is the old programmers mantra: “Garbage in, Garbage out”.

To be a tad more precise, like any other computer program, a simulation can only do what
you tell it to. If you don’t already know how something works, you can’t simulate it. If you
think you know how something works but you made a tiny, miniscule error, then the
simulation can diverge dramatically from reality.

Simulations are an incredibly useful technique. They can be used in several different ways:

  1. If we know how something works, we can simulate it, and do experiments simply by
    changing parameters to the code. This can allow us to perform experiments that
    would be impossible in the real world, or to run multiple experiments much faster
    than we could, in a more controlled fashion, than we could in the
    real world. For example, we can run orbital simulations of our solar system which
    are astonishingly precise, and which allow us to try scenarios that would be impossible
    to test in the real world.
  2. If we don’t know how something works, but we have a theory, we can test it
    by implementing a simulation according to our theory, and comparing the results of
    the simulation to observed real-world results. If our simulation closely matches what
    we observe in the real world, it’s a great piece of supporting evidence. If it doesn’t,
    it means we got it wrong. For example, people have been running massive simulations of
    the experiments that hope to produce the Higgs boson, which allows them to make
    predictions about how they’ll be able to recognize it if their experiments manage to
    create one.
  3. If we know part of how something works, we can build simulations give us a way
    of experimenting specifically on that part without the added complexity of
    a complete system. For example, we can study specific parts of cellular metabolism:
    we know the basic chemistry of how a mitochondria produces energy for the cell. We
    can focus our attention on simulations of that specific process, without dealing
    with all of the details of cellular metabolism.

It might sound like I’m saying that simulations can’t surprise us – because they
can only produce things that we already know. That’s not the case – accurate simulations can be extremely surprising. The most common cause of that is a phenomenon called
emergence. Emergent phenomena are things where some thing behaves one way at one scale,
but changes dramatically when you put together huge numbers of those things and look
at them at a different scale.

The best example of emergent phenomena is our macro-scale universe. When we look at
the world, things seem concrete and predictable. When you watch a baseball game,
you can see the baseball fly from the hand of the pitcher to the bat, and it’s obvious
that you can precisely describe both the position and the velocity of the baseball when it’s
in flight. But the baseball is made up of a huge number of particles which do not behave in such well-mannered ways. They’re unpredictable, erratic. Their behavior can’t
be described precisely, only probabilistically. And yet, when we put together quadrillions of quadrillions of unpredictable, probabilistic particles, we get something concrete, comprehensible, and extremely predictable.

Simulations can (and frequently do) surprise us due to emergence. We
may understand the how some thing works, without understanding what’s going to happen when we
splice together a billion of them. (Simulations can produce surprises due to things
other than emergence; the reason that I stress that one is that it’s the one I’ve experienced.)

But getting back to the topic at hand: when it comes to animal testing, most of the
time, we can’t use simulations – because we don’t understand enough to be able to
do an accurate simulation. We can’t simulate what we don’t know. And when it comes to
medicine, there’s an awful lot that we don’t know.

A few examples: we know that a lot of non-coding DNA has function doing various things
like regulating coding DNA. We don’t know all of the functions of non-coding DNA. Of the non-coding regions we basically understand, we don’t understand how they all work. Getting
away from DNA, the basic day-to-day functioning of our cells includes tons of
processes that we simply don’t understand. There’s so much going on in a single cell
that we don’t understand yet, that we don’t have a chance of producing a dead-on accurate simulation of it.

And to try to simulate more than a single cell is even harder – because the cells of
our body have an extremely complicated set of interdependencies and interactions. To do a simulated drug test, we need to simulate both the intra- and inter-cellular processes that would be affected by that drug. And the fact is, we don’t.

To throw out another example: as I’ve mentioned before, I have clinical depression.
I manage it by taking a selective serotonin reuptake inhibitor called “zoloft”. For me,
and for many other people with depression, SSRIs are almost miracle drugs – they
completely eliminate the symptoms of depression. But: We don’t know why zoloft works.
It was designed under a theory – the “serotonin theory” – that thought that depression was
caused by an inadequate supply of serotonin in synapses of the brain; zoloft was
designed specifically to target the cellular mechanisms by which serotonin is
removed from those synapses. But that theory has, largely, been discredited. It appears
that the mechanism that zoloft was designed to target is not the mechanism
by which it works! Why? No one is sure, but a lot of people are doing a lot of work to try to understand it.

For the people trying to understand that, in order to develop better treatments
for depression, simulations are no good – because we don’t know the mechanism by which
the drugs work
. We know that they do, but we don’t know why or how.

When it comes to other things, we’re in a similar situation. Will a new antibiotic work? Maybe. Maybe not. How do we know? We need to test it. We start with tests in cell cultures. But lots of drugs that work in cell culture don’t work in a living creature. There are numerous chemicals that kill HIV in cell culture, but don’t work in an infected animal. There are tons of chemotheraputic agents that kill cancer in culture, but that don’t work in
an animal or a person. We can’t do tests of those using simulations, because we simply
don’t know enough about the underlying mechanisms. And if we don’t understand a process
or mechanism, we can’t simulate it.

Life is still very mysterious to us. There’s so much about living things and the
basic physical and chemical processes that occur inside of them that we don’t understand. Those mysteries are what we try to study with science, in order to develop more understanding. Simulations can be a useful tool in the process of exploring. But it can’t replace observations and experiments with real living things. There’s still no substitute
for reality.

Comments

  1. #1 khan
    April 2, 2009

    Reality is implacable and annoying.

  2. #2 Gex
    April 2, 2009

    A problem that has occurred to me that I can’t really think of a solution to, is how the for-profit, private enterprise use of animal research necessarily results in more animal testing than is needed. What one for-profit company learns in animal testing is trade secret and anyone else who wants to run those same tests will have no choice but to repeat tests.

    Like I said, I can think of no good solution to this. I generally agree that there’s no substitute for reality in these cases. But I wish that we could reduce redundancy by sharing results.

  3. #3 Deen
    April 3, 2009

    There is another issue with simulations that I’d like to add: as you said, you first need to understand a system before you can simulate it, so you will need a substantial amount of animal testing to even be able develop these biological simulations. But the assumption is often that after the simulation has been developed, the animal testing can stop.

    This is not true: you still will need animal tests to validate your simulation. This is the only way you can be sure that your simulations are accurate enough, and to verify that you’ve included all the relevant features in your model.

    Compare this to car manufacturers who need to make sure a car is safe. However, doing crash tests costs millions of dollars, as they will destroy a mostly hand-built prototype each time. So they use a lot of crash simulations, especially in the early design stages. This lets them weed out the worst of the designs. But the simulations have their inherent limitations, so they aren’t going to let a new model on the road before they ran several real crash tests. In the end, the biggest advantage of the simulations is not so much that they reduce the amount of crash tests, but that they allow the designers to evaluate their designs far earlier in the design process. This allows them to try many more different ideas before they have to commit to building an expensive real-life prototype.

    I expect the same thing to happen with biological simulations. They may eliminate the need for many animal tests, but they won’t make them obsolete. No simulation-tested drug will be sold without being verified in animal tests, you just can’t take the risk that the simulation missed something. The biggest advantage here is again that scientists can try out many more ideas than they could without simulations, even quite unlikely ideas that today would never get permission for real animal testing, even though some of them might turn out to be valuable. The simulations will mostly just be used as a filter to select the most promising ideas, before having to commit to the rigorous (and expensive) procedures for getting permission for real animal tests.

  4. #4 SAWells
    April 3, 2009

    I’ve written biophysical simulation software and some drug companies even bought it :) I can safely say there is zero chance of simulations replacing whole-organism tests for things like drug safety. We wouldn’t even know where to start; in order to write such a simulation we would have to already know everything about human biochemistry.

    My own modest suggestion is for all medical testing to be carried out on animal rights activists, but for some reason I never get a very positive response.

  5. #5 Paul Carpenter
    April 3, 2009

    Of course, if it weren’t for extremely good computer simulations then there’d be a lot more animal testing needed. Computer simulations can and subsequently do replace a lot animal testing – the difference is that they can’t replace all of it.

    As our understanding of biology furthers, there will be less and less need for animal testing – it the very long term animal testing could become practically redundant, but not any time soon.

  6. #6 William Wallace
    April 3, 2009

    So, as a person who probably believes in common descent (although I do note you are not an atheist) on what basis and at what point does testing on an animals become wrong?

    Humans are animals, according to biologists.

    Chinese were animals, according to the Japanese, who during WWII performed tests on them. (“Human weeds” may be a better translation of how they Japanese caricatured the Chinese).

    Jews were sub-human according to German scientists.

    Fetuses and embryos are not human, according to many liberal Americans.

    And, indeed, while almost all condemn the type of testing performed on Jews and Chinese, some scientific knowledge came about from this testing….scientific knowledge that was not discarded by science after the war, if my memory is correct.

    Consent is obviously a component when it comes to testing on humans, but, likely, you still wouldn’t accept it if a man consented to testing that would be fatal if a deal had been struck to give that man’s family enough money to be financially comfortable.

  7. #7 trrll
    April 3, 2009

    Nobody does studies on animals if they can possibly use any other methods. Animals are expensive to maintain and care for, prone to diseases and other sources of artifact, and in general highly variable. Animal use invariably entails jumping through multiple regulatory hoops that slows research down. Even adding an investigator to a project or making an insignificant change in a research protocol is often delayed pending approval by an animal welfare committee (which typically meets maybe once a month).

  8. #8 Alex Deam
    April 3, 2009

    So, as a person who probably believes in common descent (although I do note you are not an atheist) on what basis and at what point does testing on an animals become wrong?

    Humans are animals, according to biologists.

    Chinese were animals, according to the Japanese, who during WWII performed tests on them. (“Human weeds” may be a better translation of how they Japanese caricatured the Chinese).

    Jews were sub-human according to German scientists.

    Fetuses and embryos are not human, according to many liberal Americans.

    And, indeed, while almost all condemn the type of testing performed on Jews and Chinese, some scientific knowledge came about from this testing….scientific knowledge that was not discarded by science after the war, if my memory is correct.

    Consent is obviously a component when it comes to testing on humans, but, likely, you still wouldn’t accept it if a man consented to testing that would be fatal if a deal had been struck to give that man’s family enough money to be financially comfortable.

    As far as I know, no liberal Americans claim that “Fetuses and embryos are not human”, rather they say that they don’t have a right to life.

    Also, why are you lumping (your idea of) the attitudes of liberal Americans to abortion, to the attitudes of the Nazis and the Japanese to specific ethnic groups? How in any way are they the same thing? That’s a rhetorical question by the way.

  9. #9 William Wallace
    April 4, 2009

    Alex,

    How are they (Nazi’s killing Jews, or Japanese killing Chinese (and their unborn children–see Rape of Nanking not the same?

    Regarding liberals, I never hear anybody talk about the sanctity of personhood, but I do hear others of all political persuasions talk about the sanctity of human life.

    And you’ve honestly never heard a liberal refer to a fetus as parasitic tissue?

  10. #10 SAWells
    April 4, 2009

    Go away, little Willy, you’re not even vaguely on-topic.

    (Fetuses are parasites on the mother until birth and the mother _is a person_ and can make her own decisions. All this right-wing fetus-worship is nothing but misogyny thinly disguised.)

  11. #11 ttw
    April 4, 2009

    The same arguments apply equally to nuclear weapons testing (or ballistic missile testing.) The Clinton administration seemed to think that simulations were good enough that a comprehensive test ban treaty was possible. Of course, that’s a political decision, not a scientific on; they should have weighed the gain (or loss) from having such a treaty against the knowledge gained by more testing. (So far, it seems that having a treaty is the better option.) The succeeding administrations seem no more knowledgeable about such things.

    There are case where the political (or publicity) advantage of forgoing testing outweigh the ensuing loss of knowlege.

  12. #12 Adrienne
    April 4, 2009

    This article is intriguing. Simulation can be very useful in the preliminary testing stages of new pharmaceuticals; however, I do not believe that it will be able to replace animal testing. I agree that testing on animals may be wrong if done improperly and without care, but we have to face the reality that it is necessary in certain areas of research. One of these areas definitely includes medical research because we do not completely understand the dynamic biological processes involved with medicine. Simulation can be used to model many processes, but it is important that we understand its limitations because if they are ignored, the results may not only be false, but also may be catastrophic.

  13. #13 Alex Deam
    April 4, 2009

    How are they (Nazi’s killing Jews, or Japanese killing Chinese (and their unborn children–see Rape of Nanking not the same?

    If your moral compass can’t see any difference between the Holocaust and abortion, then maybe it’s not liberal Americans who are fucked up, but you.

  14. #14 William Wallace
    April 4, 2009

    Okay, let’s look at the moral differences.

    1. Jews were killed against without their consent.
    2. Unborn babies are killed without their consent.

    Opps, sorry, that’s not a difference. Let me try again.

    1. The NAZIs profited from labor camps that worked Jews until they died, or were too sick to work.
    2. Doctors profit from the deaths of unborn children.

    That is *almost* a difference, but it isn’t a moral difference.

    Hmn. Maybe you could help me understand the *moral* difference, Alex, and see if you can do it without cursing.

  15. #15 Mark C. Chu-Carroll
    April 4, 2009

    William:

    You’re coming extremely close to getting yourself banned.

    Members of my family, people that my father knew, were victoms of the holocaust. You are not going to use their deaths as part of your inane attempt to drag this blog entry into your idiotic abortion debate.

    There’s a remarkably easy moral difference – but people like you will refuse to see it, because you simple don’t want to. And there’s no point arguing about it, because the pro-choice folks have heard your arguments, and rejected them as bullshit; and you’ve heard the pro-choice arguments, and rejected them as bullshit. There’s no point in rehashing that debate.

    And you know that. And you know, full well, that this is totally off-topic. You’re just being an obnoxious little shit for the joy of it. But not here. Drop it now. If you drag the holocaust into this one more time, you’re gone.

  16. #16 William Wallace
    April 5, 2009

    You may ban me if you like, but members of my family were victims of abortion.

  17. #17 Gex
    April 5, 2009

    Well if it helps at all, call fetuses people with full rights. Then give the mothers the right to evict their tennants. No more abortions, just evictions.

    You don’t want to call fetuses people. You want to call them people and women somewhat less than people.

  18. #18 Enkidu
    April 5, 2009

    Are scientists egotistical bastards who think they know everything while they miss the true mystery and incomprehensible nature of life, the icing on God’s creation? Or do they really understand everything so well that they can simulate entire ecosystems in a computer, never needing to check their results against the real thing?

  19. #19 Robert Thille
    April 6, 2009

    Would the study of suffering/pain be sufficient reason to cause suffering/pain in animals? Why or why not? To be honest, I don’t think you can broadly generalize about it, but that thought popped into my head when I read, “there’s no good reason to inflict unnecessary pain on any living thing.”

    However, I think that a majority of people would support “torturing” ants or spiders or even snakes if there were a high likelyhood of transferable results to management of human pain.

  20. #20 wwwald
    April 7, 2009

    Agreed. Computer models will never provide be a complete alternative. Models will always be just models, and will never capture the full complexity of reality in all its details. So there will always be a certain need for testing on live biological systems, since they are the complexity.

    BUT: computer models are not the only models.

    In toxicology and related fields, there is an important trend towards in vitro testing of harmful effects of chemicals. This basically means that we take a physical model of the animal and do the testing on that.
    Example: if we have a good idea of the genes involved in a certain allergic reaction to a chemical, we can build cell culture-based systems to test for any harmful effects. Chemicals are applied to these cell cultures and effects can be monitored. No animals, inflicted pain or ethical issues involved, yet biologically very relevant.

    The European REACH[1] legislation includes strong incentives for the development of such alternative testing methods, based on physical and also computer models (look up “QSAR” modelling, for instance).

    Sure, even these alternative tests require validation based on animal testing. But they have the potential to strongly reduce the number of actual animal tests.

    Further reading:
    In vitro testing: http://www.cardam.eu/Cardam/Home.htm
    REACH: http://ec.europa.eu/environment/chemicals/reach/reach_intro.htm

    [1] REACH = Registration, Evaluation, Authorisation and Restriction of Chemical substances