The Demarcation Problem... again

Janet has raised the undead argument from its grave again! I... must... respond...

The issue is (raised, as always, by the existence of woo science, antiscience, and pseudoscience) how do we know when something is, or isn't, science? This is often called the "demarcation problem" (DP hereafter). I'm going to offer a couple of ways to do this, based on the notion that science, like every other historical entity, evolves.

Janet discusses Popper's "solution" - something is science if it is potentially falsifiable. Leaving aside the problems with the notion of falsifiability, and indeed the problems with the whole Popperian view of science (which I happen to think is fundamentally incomplete and deeply flawed, in that it excludes around three quarters of what scientists actually do), let's consider if there might not be other ways to determine if something is science or not.

There are three classes, broadly speaking, of claims that vie for science status. One are those that are simply not at issue; they just are science by almost universal consent. These include the tenets of relativity theory, atomism, germ theory, immunology, genetics, the expanding universe, and yes, evolution. Anyone who denies that these are scientific theories or precepts is simply outside the pale of viable hypotheses.

The second class is that of obviously unscientific claims. Now these are not defined purely in terms of being the denial of established scientific views, as they are also inclusive of ethical claims, aesthetic claims, and other normative or fact-insensitive propositions, as well as those things that are just wrong. So, being nonscientific is not in itself a Bad Thing. Being nonscience that claims to be science is, and being nonscience that claims to be science but has been shown to be false is the Worst Thing.

But there is a third "class", if we can call it that: the Grey Area. Things that might be science or might not, we just don't know. And this, you may be surprised to know, is by far the largest class, because there are indefinitely many things we do not know if they are right or wrong, empirically decideable or not. How much of that very large class shall we allow to be included in the "might-be science" category? When is it woo and when is it reasonable?

Here there is, I think, no universally true rule or test, but I can offer, by analogy with biological evolution, some reasons for inclusion and exclusion based on history, and it might in fact offer some surprising implications about what "science" denotes.

David Hull, who I have often cited or referred to on this blog, wrote a (huge) text in 1988 entitled Science as a Process, in which he argued that science is an evolutionary process, analogous to biological ("Darwinian") evolution. Leaving aside the technical details of Hull's approach, what is important here is that there are two ways to classify things in evolution - by ancestry or by similarity. For example, one might say all birds are birds because they evolved from a single common ancestor species we can call Aves - this is the monophyly definition. Or we might say that all birds are birds because at some stage their ancestor evolved into the "flight grade using feathers". This is the adaptive novelty definition. Taxonomists refer to the first as cladism, and the second as gradism. A clade is a branch of the evolutionary tree, while the second might be thought of as a net cast over the tree, covering some branches entirely, others partially, and some not at all, for a given group of organisms.

If we take the gradist approach, and classify by adaptive similarity, an obvious problem arises - what to do when the descendants of the group evolve so much they are no longer recogniseable as members of that group? For example, mammals were once very reptilian. At what point do we draw the line and say, here are reptiles and over there are mammals? What if a reptilian mammal, say a naked mole rat, re-evolved? Is it still a mammal?

Translate this into the question of when something is a science. On the cladistics view, something is science if it is linearly descended from the "last common ancestor" of all things that are unquestionably science (just as Aves is sometimes defined as the last common ancestor of two extremely diverged birds, say, an emu and a penguin, and all that species' descendants). This is what philosophers call an "extensional" definition - take the borders and draw a line around all the items that fall within the extent of those borders.

But what we have traditionally done with demarcation disputes is to try to give an intensional definition (from the intent of the definition - what it means). Popper's of course was between things that could be falsified and those which could not. Then it was pointed out that many things in what we wanted to call science were not falsifiable, including crucial hypotheses. This is effectively a grade-based definition, and it has real trouble with things that do not exactly match whatever criteria were chosen for the definition. For example, Popper's view of science would make systematics, the description of taxonomic entities like species, nonscience.

In an evolutionary tree, the evolution of similar functional or morphological characters is called "convergent evolution", or more technically "homoplasy". There appears to be no term for things that once were functionally similar but have evolved to be different, as such - perhaps "alloplasy" would fit the bill here. When a science ceases to be a science, it becomes alloplasious. An example might be phrenology, which started out as a scientific enterprise, and nothing about it was necessarily unscientific to start with, but which rapidly developed into a popular form of fashionable nonsense. A bit like much of popular psychology today.

So, when does something that evolved from within the science grade cease to be science? And if something else achieves that grade independently (say, Chinese medicine), is it the same kind of science as the modern form?

On Hull's account, no matter how similar something might be to modern science, if it doesn't evolve from the "ancestral" science (which for argument's sake we'll put at the foundation of the Royal Society, although clearly there are prior things that we retrospectively ascribe scientificness to, such as Copernican astronomy and early natural history), then it isn't science. It might be something else that does a similar job, just as dolphins and ichthyosaurs independently do, but like them, they won't be in the same taxon. Why is this?

In part it's because the likelihood is that each lineage of practices, ideas and methodological concerns will have their own bundle of complex adaptations, and inheritances, so it's unlikely they would ever be the "same", no matter how similar they are, in the absence of "cross-lineage borrowing" (and probably not even then).

But more importantly, it's because similarity fails to access the actual shape of nature/society reliably. We determine what answers we will get by our choice of, and weighting of, similarity measures. Why, for instance, a "flight grade"? Surely it's because we find flight salient and significant. In the past, we called anything that lived in water a "fish", which included whales, crocodiles, cephalopods, some birds, and turtles. That's a bit extreme as an example, but it's not all that uncommon, and it shows why grade-based classifications are not "natural" - they can only be as natural as the property chosen is a representative or surrogate marker for what actually is a natural group. And once you have removed homoplasious markers, what you get just is a clade-based classification.

Science is not intensionally defined by its access to a grade of method or other characteristic. For any plausible candidate definiens, we can find examples that lack or contradict them, which we will want to call science. So it is clear, at least to me, that an intensional definition is inadequate, just as Popper's intensional definition of science is inadequate because it artificially imposes constraints on actual science, no matter how good his intentions.

Hence the desiderata for inclusion in science should be taken as genealogical. Something is science if it evolved, by a process of conceptual descent, from something that is science. But this can't be sufficient, because we know, as in the phrenology case, that nonsciences can develop out of science as well. It must also be somewhat gradistic in the sense that once it evolves too far from the mode of "being science" that the majority of the clade exhibit, it ceases to be included in science. So "science" is what taxonomists call "paraphyletic - a class of evolved historical entities that must share a common ancestor (the program developed with the Royal Society in 1665 or so, but which retains the ancestral characters of that process (called plesiomorphies for those who know the terminology).

Will this do? I do not think it will, because science was not handed down as a philosophy or methodology once and for all like Moses' tablets. Anyone who tries to do science now the way Newton did it would be regarded as both anachronistic, and somewhat naive (indeed, attempts to do this is one of the marks of pseudoscience today). Things have moved on since Newton or Harvey.

So what? Shall we say, with Feyerabend, that anything goes, and there is no "essence" to science at all? Or shall we say that science is a set of intensional properties after all? I think the solution is to say that science, being an evolving process, is in fact something that retains its genealogical links at a given time. Modern sciences are not independent of each other - statistics, which was formulated to deal with social phenomena and later genetic data, is now a major part of many sciences, including physics. Ideas like "simulated annealing" and "Markov chains" apply to physics and biology and probably other disciplines. Science is more like a species than like a clade, in other words. It has regular "interbreeding", forming networks of influence between the "populations". Science is a historical entity - a single historical entity, with its own particular makeup at a given time horizon, and what worked well at one point in history need not be a defining character of science at another moment.

Of course, there is structure within the science population, just as there can be within a single species. There is not unmitigated flow of influence between all disciplines at once. And a set of disciplines can easily evolve isolation from the rest of the disciplines and become nonscience, or something else altogether. They can even become theology.

What marks out something as science is that it has not yet done this - that it engages in constant interflow between itself and other scientific disciplines. We know that many disciplines are science - they act as our referents at t for what is science and we relate the other more problematic disciplines in terms of how they interact with those more certain ones. It doesn't mean that science is static, though - we can tell when something has left the building, as it were, by comparing it with the others that remain inside.

I think it was Samuel Johnson It was Edmund Burke who noted that day shades into night, but on the whole we can tolerably distinguish between the two. The potentially vast set of twilight disciplines are greatly trimmed down to those historical candidates that either develop out of, or in reaction to, science, such that we can nearly always tell that something just isn't even in the game. Creationism is one of these, and lately HIV/AIDS denial is another. Once they may have been candidates for scientific explanations, but things have moved on, and we can say they are now out of contention, by comparing their claims with the rest of the corpus of science.

We may never have a complete or universal science, and there may always be twilight cases, but mostly we are able to tell when something is, and isn't, science, by its parentage and its behaviour...

Late note: some interesting stuff at Mixing Memory on analogical reasoning, which is relevant to this topic.

More like this

Feyerabend did not say that anything goes.

I would hesitate to prima facie place aesthetic claims outside the realm of science. Certainly, the role that elegance has played in promoting scientific progress has to be acknowledged - and if the goal is to examine what scientists actually do and how theories are actually accepted, then it seems that aesthetics falls into the grey zone and not into the obviously non-scientific zone.

Perhaps there's a way around this by throwing context of discovery/context of justification at the problem, but I'm not sure that gets around historical dynamics that see evolution and certain economic theories becoming consensus before their dynamics have been rigorously worked out. Elegance seems at certain times to have become a justification rather than a suggestive recommendation for one theory over another.

I tend to regard demarcationist reasoning in science as being pretty useless. Not because there aren't things that can be regarded as science and things that can be regarded as non-science or pseudoscience, but because I have never seen a universal criterion that encompasses all of them in a satisfactory fashion.

As far as I can gather from being involved with the skeptical movement since my early teens, most debates around whether something is science, borderlands science and pseudoscience is best judged case by case. In fact, in most cases Popper's demarcation criterion is inapplicable, since things like homeopathy and ESP are not unscientific because they are unfalsifiable, but because they demonstrably false.

Since my primary interests are in computer science, it's interesting to note that similar discourse often goes on about whether computer science is really more properly classified as engineering or mathematics. All classifications overlook or downplay some aspects of the field, so it's a debate I don't find myself tempted to take part in.

Jason: I squat corrected.

Omri: Elegance in math is I think the implicit rules of mathematical reasoning cast in aesthetic terms. I don't think it is a claim that beauty is truth or anything.

Tyler: While I agree with you, I'm more interested inthe things that aren't demonstrably false and might be science, in this respect. Freudian psychology, for instance, circa 1910.

Interesting post. I wonder if you could extend your analogy further, and think about selection pressures: science seems unique in that its evolution is (mostly) guided by reference to empirical measurements, which is a stabilising influence against other pressures (fads, charismatic leaders, "common sense", conflict with dogma) which, if they become dominant, will push a science down the road to pseudoscience.

I think it was Samuel Johnson who noted that day shades into night, but on the whole we can tolerably distinguish between the two.

Gould, in his essay "A Quahog is a Quahog" in The Panda's Thumb attributed it to Edmund Burke:

Species are stable entities with very brief periods of fuzziness at their origin (although not at their demise because most species disappear cleanly without changing into anything else). As Edmund Burke said in another context: "Though no man can draw a stroke between the confines of day and night, yet light and darkness are upon the whole tolerably distinguishable."

But, hey ... it should have been Johnson (and maybe it was).

Really good essay. Thanks.

"statistics, which was formulated to deal with social phenomena and later genetic data"

What you mean to say is that statistics was formulated to deal with astronomical problems, and later adapted to problems in genetics and social phenomena. I suspect that your example of social statistics would refer to Quetelet, but his interest in statistics started when he wanted to make Belgium a great astronomical power.

Second, I'm curious why you'd say that "Popper's view of science would make systematics, the description of taxonomic entities like species, nonscience."

Systematics operates by proposing the hypothesis that a given group of organisms is a species (for some definition of "species"). One then tests that hypothesis by attempting to falsify the claim that the population is genetically distinct, or interbreeding, or on a unique evolutionary trajectory.

Indeed, the emergence of cladistic systematics in the '70s and '80s was a victory of Popperian falsifiability. An evolutionary hypothesis about groups can be falsified, while purely phenetic groupings cannot, at least not meaningfully. One cannot falsify the basis for selecting one set of phenetic traits rather than another.

By my thinking, this gets to a useful point about the nature of science. It isn't an encyclopedia, it is a way of testing knowledge. So phrenology was a science in the 1800s when it was presented as a testable claim. It was falsifiable and was falsified. It was scientific and wrong.

When people promote it now, they are behaving unscientifically because they are rejecting falsifying evidence, rather than a false claim. The problem Tyler is raising is misdirected. ESP isn't nonscience because it's wrong, it's nonscience because people advance it knowing that it's been falsified. Newton has been falsified also (in some sense), but didn't get demoted to non-science as a result. People who try to act as if Einstein didn't rewrite Newton's work are demoted to non/pseudo-science.

John Pieret:

I think it was Samuel Johnson who noted that day shades into night, but on the whole we can tolerably distinguish between the two.

Gould, in his essay "A Quahog is a Quahog" in The Panda's Thumb attributed it to Edmund Burke:

It seems that Gould liked the quote - he alludes to it in The Hedgehog, the Fox and the Magister's Pox as well - and he attributes it to Burke there too.

BTW John, there's a post of mine held up in moderation on your Google thread; is there a problem with it?

By Robin Levett (not verified) on 04 Dec 2006 #permalink

Popper was right about the centrality of critical rationality for the development of modern science. He was flat out wrong about falisification/falsibiability. Duhem, and later Quine, nailed it.

By bob koepp (not verified) on 04 Dec 2006 #permalink

Four thoughts wandered casually through my head as I read the post. First, the question "What is X" carries the necessary presupposition that there is some sort of demarcation method -- we can reliably discriminate X from not-X by applying some (external, intersubjectively replicable) criteria to the object to be classified and arrive at an unambiguous classification. Folks who take that route, implicitly or explicitly, could do with a dose of fuzzy set theory.

Second, I'd like in all such cases to see a parallel analysis of "swimming", with appropriate classifications of such specific activities as doing the dog paddle, treading water, being towed by an underwater scooter, and platform diving (an event in "swimming" meets).

Third, a key property (necessary but not sufficient) is John's throw-away reference to sensitivity to facts. (Leaving aside the sometimes tough question of what a "fact" is.)

And finally, when I took philosophy of science from Herbert Feigl, he referred (apparently affectionately) to Feyerabend as "that wild man" (hear it in a Viennese accent).

Really interesting post, John. It strikes me as strange, though, to draw from scientific rhetoric to make an argument about what is science (rhetoric about science, then, to make the case about scientific rhetoric). Can one ever see and analyze the thing (or process) being observed if still using the terms of the observed thing (or process) to do the seeing?

One Feyerabend: "To those who look at the rch material provided by history, and who are not intent upon impoverishing it in order to please their lower instincts, their craving for intellectual security in the form of clarity, precision, 'objectivity', 'truth', it will become clear that there is only one principle that can be defended under all circumstances, and in all stages of human development. It is the principle: anything goes. [Against Method, p27f]

I like Feyerabend because what he says is so often a necessary fillip to the standard views of conceptual analysis in philosophy of science. I am also his intellectual grandson. But I would revise him somewhat - "anything goes in the abstract. In historical cases, only some things go." Which is probably what he meant when he continued, "This abstract principle must now be examined and explained in concrete detail."

On the circularity of using a scientific model to model science: this is only circular in the vicious sense if it has no external support. Hull spends a lot of time arguing for that external support. And so long as what you say about science in general is not inconsistent with the principles of science, it is not self-defeating. Given that we have no God's-Eye View to address these issues (not even if we start from theological grounds), it is necessary that we use our knowledge to advance our understanding. Rebuilding the raft at sea, so to speak.

It appears Gould was right:

"No lines can be laid down for civil or political wisdom. They are a matter incapable of exact definition. But, though no man can draw a stroke between the confines of day and night, yet light and darkness are upon the whole tolerably distinguishable." p. 105 lines 11-13, Thoughts on the Cause of the Present Discontents (1770). From the Payne edition, available here as a 2.3Mb PDF.

In the 3rd edition of AM, Feyerabend also wrote (p. 230):

"Some reviewers have classified me as an idealist in the sense just describe with the proviso that I try to replace familiar rules and standards by more 'revolutionary' rules such as proliferation and counterinduction and almost everyone has ascribed to me a 'methodology' with 'anything goes' as its one 'basic principle'. But in Chapter 2 I say quite explicitly that 'my intention is not to replace one set of rules by another such set: my intention is, rather, to convince the reader that, all methodologies, even the most obvious ones, have their limits'".

I would say that, if there is one attribute which distinguishes good science from bad science or pseudo-science it is the same one that distinguishes agnosticism from atheism (sorry, I couldn't resist the dig) and that is doubt. In the end, that is all that is needed.

Pseudoscientists are characterised by the same certainty that they are right as religious fundamentalists and are utterly impervious to counter-argument and contradictory evidence. They believe they have found some Ultimate Truth which, whether they realise it or not, would mean an end to science.

Cromwell's famous entreaty, although this interpretation was not what he intended, puts it most pithily:

I beseech you, in the bowels of Christ, think it possible you may be mistaken.

Richard Feynman said much the same thing in a different way:

Another of the qualities of science is that it teaches the value of rational thought,as well as the importance of freedom of thought; the positive results that come from doubting that all the lessons are true. Learn from science that you must doubt the experts. As a matter of fact, I can also define science another
way: Science is the belief in the ignorance of experts.

The Pleasure of Finding Things Out by Richard Feynman, p. 186-187

That is the idea that we all hope you have learned in studying science in school. It's a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty a kind of leaning over backwards. For example, if you're doing an
experiment, you should report everything that you think might make it invalid, not only what you think is right about it. Details that could throw doubt on your interpretation must be given if you know them.

The Pleasure of Finding Things Out by Richard Feynman, p209-210

By Ian H Spedding FCD (not verified) on 05 Dec 2006 #permalink

"Feyerabend did not say that anything goes."

Yes, I believe it was Cole Porter who said that.

Sorry, couldn't resist, high school musical flashback, now back to your regularly scheduled discussion on demarcation

I suggest that one might be better off trying to define a scientific theory, rather then science in general. Thus, I claim that a scientific theory must have three components.

1. It must be explanatory, that is, it must explain observed phenomena.

2. It must be predictive, that is, it must predict that certain observations will be made of the theory is correct.

3. It must be, at least in principal, falsifiable, which is related to 2 above, that is, if it predicts a certain observation and that observation is not found, the theory is either not correct or incomplete.

As an example, consider the Special Theory of Relativity (SToR).

1. SToR explains the result of the Michaelson-Morley experiment.

2. SToR predicts the phenomena of time dialation, which was observed in the decay of muons.

3. If the mean lifetime of muons had not been observed to increase with increasing speeds, SToR would have been falsified.

Randy:

"Feyerabend did not say that anything goes."

Yes, I believe it was Cole Porter who said that.

But which Cole Porter? The one who wrote "Kiss Me Kate", or the one who drew the military map of Basingstoke in Westphalia?

Anything goes in
Anything goes out
Fish, bananas, old pyjamas
Mutton, beef, and trout