In my post yesterday, I briefly mentioned the problem with simulations
as a replacement for animal testing. But I’ve gotten a couple of self-righteous
emails from people criticizing that: they’ve all argued that given the
quantity of computational resources available to us today, of course
we can do all of our research using simulations. I’ll quote a typical example
from the one person who actually posted a comment along these lines:
This doesn’t in any way reduce the importance of informing people about
the alternatives to animal testing. It strikes me as odd that the author of
the blogpost is a computer scientist, yet seems uninformed about the fact,
that the most interesting alternatives to animal testing are coming from that
field. Simulation of very complex systems is around the corner, especially
since computing power is becoming cheaper all the time.That said, I also do think it’s OK to voice opposition to animal testing,
because there *are* alternatives. People who ignore the alternatives seem to
have other issues going on, for example a sort of pleasure at the idea of
power over others – also nonhumans.
I’ll briefly comment on the obnoxious self-righteousness of this idiot.
They started off their comment with the suggestion that the people who are
harassing Dr. Ringach’s children aren’t really animal rights
protestors; they’re people paid by opponents of the AR movement in order to
discredit it. And then goes on to claim that anyone who doesn’t see the
obvious alternatives to animal testing really do it because they
get their rocks off torturing poor defenseless animals.
Dumbass.
Anyway: my actual argument is below the fold.
So – what’s the problem with simulation? It’s not that we don’t have
enough computational power. The amount of computational power available to us
is truly mind-boggling. Many things that would have been completely impossible
just a few years ago have become absolutely routine. Sitting on my lap, I’ve
got a computer with two CPUs, each running at 2.5 gigahertz. Under my desk,
I’ve got a computer with 8 CPUs. And I routinely run programs that use
several thousand CPUs in a datacenter. That’s not unusual
today: people who want to run simulations can easily and cheaply
get access to clusters of thousands of computers.
So computer power isn’t a problem. Using modern computers
and software, we’ve easily got enough power to run incredibly complex
simulations.
And that doesn’t help with using simulations to replace animal testing
at all. The problem with using simulations for testing has absolutely
nothing to do with computational power. No matter how much computational
capability you have available, there’s still a huge, fundamental problem with
simulations. It’s not particularly hard to understand: simulations only do
what you tell them to.
Let’s start at the beginning: just what is a simulation? It’s a
model of a real system, which attempts to reproduce the effects
and/or behavior of the system that it models. In the case of computer
simulations, which is really what we’re talking about, we produce a
mathematical model and algorithmically describe how the model evolves over
time. In other words, we write a computer program that runs a mathematical
model of a process.
And right there is the problem. We produce the model, and
we implement the model. It does exactly what we tell it to. We
programmers have a saying which applies particularly well to simulations:
garbage in, garbage out. Computers do exactly what you tell them to; if what
you tell them to do isn’t right, then no amount of computer power is going
to change the fact that you didn’t tell them to do the right thing.
If we really understand what we’re simulating, we can do simulations that
are astonishingly accurate. For example, we can do wind tunnel simulations of
aircraft designs, and the results of those simulated experiments are perfect
to within our ability to measure them. We’ve got it down well enough that we
can get more precise results with a computational model than we can
with a scale model in a wind tunnel. We understand how air flow works. We
know how to model it. It takes a whole lot of computation to do a decent job
of it, but as I said before, lack of computational resources isn’t generally
a problem.
But we can’t simulate something if we don’t know how it works. And that’s
the problem with biological simulations: we don’t know how the
biological systems work. If we don’t know how they work, we can’t build
an accurate model. And if we don’t have an accurate model, we can’t build
an accurate simulation.
When it comes to biology, we just don’t know enough to
be able to accurately or even meaningfully simulate many simple
processes.
Let me give you an example. In 1956, a scientist named Otto Warburg
discovered that most cancer cells have an abnormal metabolism. Normal cells
metabolize sugars to produce energy using their mitochondria. Cancer cells
don’t usually use their mitochondria – they use an entirely different
metabolic pathway. It’s called the Warburg effect. (My
friend and blog-father Orac sent me a note to say that the Warburg effect was
actually discovered in 1906 – fifty years earlier than the citation I found would suggest. So instead of having 50 years to study it as I write below, it’s over a hundred!)
We’ve known about the Warburg effect for over fifty years. And
yet, we still don’t know why cancer cells do it. We don’t
know what causes it. There’s no way that we can simulate it. We can’t
write a simulation of a cancer cell, because we don’t have a sufficient
understanding of its metabolism: we don’t know when it’s going to use
Warburg. And the metabolism is one of the simpler aspects of
cancer behavior: things like how the DNA in the cell changes in
cancer is vastly more complicated. We’re just no where
close to understanding it – which means that we’re
no where close to being able to simulate it.
So we can’t simulate it. Or, to be more precise, we can’t
produce a simulation that we know is valid. We can’t simulate a process
that we don’t understand. We can’t simulate a process that we don’t
know about. And biology and medicine are just chock full of processes
that we don’t understand, or that we aren’t ever aware are occurring. So
we can’t simulate those.
There’s another aspect of simulation that’s important: validation. Even in
the best simulations, you can’t be sure that the simulation is correct until
you test it. That testing is called validation. To validate a
simulation, what you need to do is to take some starting point in both the
real world and in the simulation, and observe both for some period of time,
and then check that the results of the simulation and the real-world match.
For example, for air-flow simulations, you put an object into a wind-tunnel and
measure everything; then you simulate putting an object into a wind-tunnel
with the program; and then you campare the results.
Without validation, you have no way of knowing if your model is
correct.
Validation is really key when you don’t understand what’s going on. I’ll
pull out another example. I come from a family with a history of clinical
depression. I suffer from it myself. For me, and many other people, there’s a
class of drugs called selective serotonin reuptake inhibitors (SSRIs). SSRIs
were developed based on a theory about what caused depression – specifically,
that depression was caused by a shortage of the neurotransmitter serotonin in
the brain. So they developed a set of highly targeted drugs that muck with the
serotonin chemistry of the brain. And lo and behold, it works. It’s
often cited as an example of the amazing success of modern targeted
pharmaceutical development.
Except that all of the most recent research about depression appears to
show that the serotonin theory of depression is wrong. And yet, for a
significant number of people, the drugs work – and they work dramatically
better than you’d expect from placebo effects. They do increase the
availability of serotonin in the brain. They also do a bunch of other things -
like stimulate cell growth in certain parts of the brain. How do we figure out
how SSRIs work? Among other methods, computer simulation with validation from
animal models. We start with a model: what would we expect to see if the
serotonin model were correct? We work out that model in detail. We convert
it to a program that predicts what kinds of biological effects we should
expect to see if it’s correct. Then we do the test on a group of animals,
and check to see if what we observe in the animals matches what we expect
from out model. Based on the results of that, we can judge how well the model
matches reality.
From all of this, it might sound like computer models are boring and not
terribly useful. After all, all that they can do is what we tell them to. So
don’t we always know the results before we start?
Fascinatingly, no. We have some amazingly precise models of physical
phenomena, but understanding what those models mean when they’re scaled up to
life size is incredibly difficult. And in fact, sometimes, we simply don’t
know how to see what a model says about how a system will evolve through time
without actually watching it progress. For example, I mentioned computational
fluid flow above. We don’t know if it’s possible to compute the
result of a fluid flow system without simulating it through discrete time
steps. Simulation is incredibly useful, because it lets us take dynamical
systems, and watch how they evolve over time.
So simulation absolutely has a role. And hopefully, it can reduce
the number of animals that are used in medical experimentation. But it can
never replace them. There really is no substitute for reality.