My Take on the Scientific Method (Basic Concepts)

This is my meager contribution to the "Basic Concepts" series that is going on around here. (I hope to do more later, but for right now I want to start with this one.) I've written about this before at my blog's old location, but I'm writing this now without looking back at that; we can compare later to see how consistent I am.

I also expect other scientists to have a slightly different take on this.

In three words, my view is that the scientific method is nothing more than Applied Common Sense. Now, "Common Sense" has at least three meanings. The first meaning is "the title of a tract written by Thomas Paine," and is not really relevant here. The second meaning is "what seems obvious to people in everyday life," and is very much not the scientific method, and indeed is often at odds with the scientific method. What I'm talking about here is just common sense in the sense of "apply logic, be careful, ask hard questions when something sounds odd." The most important point of this is that the scientific method is not some holy rite that is written, learned, and followed ritualistically by scientists. Indeed, it is something anybody can do in almost any situation.

The scientific method also isn't the clearly delineated set of steps you learned about in junior high school and high school science classes. Those were the steps that start, 1: formulate hypothesis. 2: design experiment. 3: take data. 4: compare data to predictions of hypothesis. Yes, in fact, we are always doing all of these steps, but it is very, very rarely that we do them in the clean, step-by-step method that you learn about in school. Often, we're stumbling about in the dark. We start looking at or exploring something to test one thing, but see odd behavior; we modify our hypotheses or form new ones, and slightly modify our experimental procedure. There's a constant feedback going on. We're playing, but we're doing it carefully, and we're doing it systematically.

The image below sums up my view of the scientific method in its full glory:

One thing to notice in here: the word "fact" is a small little brick down in the corner. When people say "Evolution is a theory, not a fact," they are trying to undermine evolution, but in fact the very statement makes no sense. Strictly speaking, yes, evolution is a theory... but in science, a fact is a small, low-level thing that doesn't really convey much deep understanding. A fact is a datum. "At 3:30PM with my experiment set up such-and-such, my voltmeter had this reading." That's a fact. Or, "At such-and-such a time on such-and-such a day, this instrument measured 25,000 photoelectrons when pointed at such-and-such a galaxy." Another fact. It doesn't tell you a lot, and requires thought to do anything with. But it's the data, the basic simple truths of science.

How do we turn that truth into understanding? First, there is the basic interpretation of the data. Do you understand how your instruments work? Are there things that could be giving you what looks like one result, when in fact something else is going on? How precise, how well understood, is the measurement we've made? Once we've done all of that carefully, we have our experimental result. Experiments can, in fact, be wrong! The facts aren't wrong, but it is possible to make mistakes in our interpretation of the fact. Sometimes, when we state the facts, we'll state them wrong; "this resistor had this voltage across it" may be what we say, but what we really mean is "this voltmeter gave this reading." Perhaps the voltmeter wasn't hooked up as we believed it was. Good experimentalists usually do things right, but we all make mistakes. Sometimes, even when we do everything right, there's something going on that we weren't or even couldn't be aware of. William Herschel wasn't aware of dust in our Galaxy, and so his star gauging placed us closer to the center of the Galaxy in his model than we really are. He didn't do anything wrong, he just didn't know everything that was out there. His star counts were right, but his final conclusions (i.e. his drawing of the Galaxy) were not.

Once we have our experimental results, now we set to the task of trying to digest them, trying to understand them, trying to put the min context and turn them into something that we might call an understanding of some part of nature. The ultimate goal is the big thing at the top: a good, working theory. A theory is just what it says there: a framework for understanding how nature works, and a means of predicting the results of experiments and observations. A good theory works and is supported by the data, but also gives us some sort of overview or picture of what's really going on.

There are wrong or discarded theories. However, in science, a theory is not simply speculation. In common parlance, it is, but that's not what it is here. Facts are the low-level, small things; theories are the big things, the goals.

When do we consider a theory to be "right"? If we are to be completely pedantic, never, really. However, when we have a theory that has explained a huge number of observations and has withstood myriad tests, it becomes conventional to view that theory as right. Certainly, applied common sense would suggest that any new hypothesis that contradicts the theory, or even an experimental result that seems to contradict the theory, should be approached with great caution. Biological evolution is one theory that scientists accept as "right," because it explains so much, and because it has stood up through an explosive increase in biological knowledge during the last century or so. Newton's theory of gravity is right where it works, but we recognize that it's not completely right, that it's only a special case of our deeper theory of gravity (General Relativity) that applies under certain circumstances.

Are we ever getting to fundamental "Truth"? That's perhaps a philosophical or metaphysical question rather than a scientific question. Science is more practical. When a theory works, and keeps working, we consider it right; for all practical purposes, it is. Truth is that which science hopes to asymptotically approach.

Somewhere below theory is the "model." A model can mean a few different things. A purely empirical model doesn't attempt to explain or provide deep understanding, it simply describes what is. Kepler's Laws are three laws that describe how planet orbits in the Solar System work. They are purely empirical laws; hey guys, Kepler says, planets are orbiting in ellipses! Check it out! It doesn't explain why, it merely describes what is. It's useful, because it can predict things. Newton's gravity comes along and says, well, if there is a universal force between every pair of masses that drops with the square of the distance, you get elliptical orbits. Newton's theory of gravity is deeper.

Sometimes a model is something born out of a theory. We have the theory of stellar evolution, but then individual theorists build their models for stellar spectra out of that theory; it is those models that get compared to data. Different theorists have somewhat different models. Partially, this is because the theory is not perfectly understood. Partially, this is because the calculations are monstrously difficult, and while the basic theory is rock-solid, it's tough to figure out what approximations are safe to make in order to render calculations tractable.

Sometimes a model is deeper than just a functional form that some data follows. Sometimes, a model is like a mini-theory. It may be because it's not what we think is really going on, but it's very useful. Or, it may be deliberate fiction that still has useful calculational power (e.g. the "celestial sphere" model used to describe motions in the sky). Sometimes, we call it a model because either it's not broad enough to get the full name theory, or because we're not sure enough of it yet to really want to call it a theory. There is this thing in astronomy called the Unified Model of Active Galactic Nuclei, that indicates that a wide variety of nuclear activity we see in galaxies are all really the same thing, just viewed from different angles. That's more of a model than a theory, because it's not quite broad enough, not quite fundamental enough to really be a theory. But it's more than a simple functional form to describe data, because it does describe to a greater or lesser degree what we really think is going on.

All of this verbiage should convince you that there is not a very clean, delineated line between what one calls a model and what one calls a theory. Models that grow up and start to sound like truth to those not concerned with fine philosophical points start to get called theories. Some people still call the Big Bang the "Big Bang Model," but I call it a theory nowadays.

Of course, the very term theory means two or three different related things, but that's a topic for another post.

What is the scientific method? In a nutshell, the scientific method is (a) carefully analyzing the facts to construct a coherent experimental result, trying to take into account anything that could make the facts appear different from what they are; (b) building models either ab initio to describe data, or from a theory; (c) testing those models against the data, and modifying them if necessary; (d) developing theories that either predict or naturally give rise to working models, or that produce new models to use in step c; (e) forming hypotheses to test in step a either by testing something that a theory or a model says should be there, or by testing something that a theory or a model says should not be there.

Importantly, though, it's not the clean set of steps you learned in high school. All of these steps are feeding back to each other and are going on in parallel. The process can start pretty much anywhere. Sometimes, we just take new data because we can. Hey! A new detector technology! Let's slap it on a telescope and see what we see! That kind of thing has led to all sorts of discoveries. Experiments don't always start with theoretically motivated hypotheses.

None of this is mystical, none of this is even terribly deep. It's not a complicated process that takes years of schooling and a PhD to learn. Anybody can can use and apply the scientific method, and anybody can do it well.