Testing 123

An emailer sends in a link to a comment on The White Coat Underground. It's in the context of naturopathy, the idea that medical problems ought to be treated with natural means. Some of it's a good idea - eating right, exercising, that sort of thing. Those sort of things aren't going to cure much, but they're good things to do. Other aspects of naturopathy pretty much go off the deep end, as ably discussed by any number of medical bloggers on ScienceBlogs and elsewhere. The point argued by this particular commenter went as follows:

Speaking of pseudoscience, what do you all think of Physics? Because, by all accounts and definitions of "pseudoscience", much of what Physics teaches us about the world is purely based on currently untestable theory. Dark matter? It's just a guess. String Theory? An obsession of many, yet unproven. Everything past the event horizon? Theories, but we can't test it yet. Haha...and PalMD...I'd like to point out that we don't entirely "know" how the universe works, as large chunks of our understanding are based on theory and "pseudoscience". :P

Point is, don't close your mind off to something just because we can't quite test it yet. The more open-minded we are (to all scientific fields), the better the progression of science and the human race.

I blame the book publishers. For some reason most of the press physics gets is related to string theory. People have this idea that we all sit around and chew the ends of our pencils and thing of ways that strings might do their stringy thing. Actual physics resembles this not in the slightest.

In the physical sciences and physics especially, science follows an infinite loop procedure. I'll label them 1 and 2, though it's really a chicken-and-egg thing. This is the way physics works, simplified:

1. The experimental results produce data which suggest ways old theories could be refined, replaced, or strengthened.
2. The new theoretical descriptions and predictions suggest new experiments to test the accuracy of those theories. Go back to step 1.

And recursively the theories become better and better approximations to reality. Some theories like QED and general relativity are literally perfect as far as we can tell, though most physicists expect that (say) general relativity will eventually require changes at very high energy. With this as groundwork, let's go through the comment.

Because, by all accounts and definitions of "pseudoscience", much of what Physics teaches us about the world is purely based on currently untestable theory.

Bzzt. The majority of work in physics is is disciplines like solid state physics, atomic/molecular/optical physics, nuclear physics, and related disciplines. In these the 1,2 loop is very fast and the theories are exceptionally accurate and well-confirmed. In short, almost all physics that happens from the nuclear to the solar-system scales is very well understood and experimentally tested. There's lots of work left to be done for more and more complicated and interlocking ways of manipulating the processes we've discovered. But with a few gaps, the basic principles are tested unbelievably well and their applications and connections are constantly being further tested in tens of thousands of physics labs all over the world.

Dark matter? It's just a guess.

Not at all. We know how mass creates gravity, and how gravity manipulates mass. So when see galaxies being manipulated by gravity without there being sufficient mass around, we can pretty well expect that in fact there is sufficient mass, which happens to be hard to see. But because you can't make a galaxy in the lab, the experiments are necessarily quite difficult and the 1,2 loop is slow. Nonetheless we've already been able to eliminate many possibilities and numerous new experiments are in the works to further test the possibilities. And of course there's other theories that are generally considered long shots (like MOND) which are being worked on and tested in the loop as well. Maybe it will turn out to be right, and dark matter won't exist after all. The point is that the loop turns and has been turning just like in the more earthbound branches of physics, and experimental tests are ongoing as we speak.

String Theory? An obsession of many, yet unproven.

I'll certainly agree that string theory doesn't have a lot going for it experimentally. The construction of the LHC may change this to an extent, or it may not. However, is it entirely wrong to think that string theory in any way fits the description of "large chunks of our understanding are based on theory and "pseudoscience"." Quite the opposite. It's entirely speculative and effectively none of our understanding of physics relies on string theory being true. It could be shown to be entirely false tomorrow and pretty much nothing would change except the research focus of string theorists. The theory is simply a good idea that's being explored because good ideas in "beyond the Standard Model" physics are in short supply. It may turn out to be an accurate description of reality or it may not, but at the moment it's simply an idea in the very first stage of loop iteration. It's not anything that would damage our current understanding if it turned out to be wrong.

Everything past the event horizon? Theories, but we can't test it yet.

This isn't right. It's not even wrong. It's like asking "What is the exact position and momentum of this electron?" The question even make sense. Presuming general relativity is an accurate description of the universe, we can't observe what's inside the event horizon. At least not if we wish to report our findings to the outside world. For our own curiosity we can use GR to predict what a person would see falling into a black hole, but that's just a purely intellectual exercise. The question as such is simply not a scientific one because its answer is not observable, testable, and repeatable.

I'd like to point out that we don't entirely "know" how the universe works, as large chunks of our understanding are based on theory and "pseudoscience". :P

We don't know anything about the universe without testing. String theory and many cosmological speculations are just that - speculations. None of what we understand and use is predicated on those speculations being true, and it won't (and can't) be until those speculations can be fitted into an experimental framework.

Point is, don't close your mind off to something just because we can't quite test it yet. The more open-minded we are (to all scientific fields), the better the progression of science and the human race.

In fact I agree with this. But it's true in the other direction as well. An open-minded physicist is willing to believe any wild theory - once it's been shown to work. Until then he's going to focus on that "we can't quite test it yet" and will not start ignoring prior tested knowledge in favor of the untested idea.

More like this

Thanks for setting this straight. I find it simply awful what kind of picture people must get of physics and physicists from what they read if that is the outcome. Most of all, I wish they would realize that the vast majority of physicists work on down-to-earth topics that the press regards just to dull to ever report on. What they report on is, more often than not, extremely speculative, not to mention wrapped into a sensational story that has little if anything to do with how research is done in daily life.

effectively none of our understanding of physics relies on string theory

The Higgs, string theory, SUSY (proton decay, axions), SUGRA... if falsifiable must be corrected or discarded. A polynomial interpolates a sine wave arbitrarily well, but it won't extrapolate at all - curve-fitted hubris like economics and psychology. OTOH, NMR spectra from a continuous wave 60 MHz electromagnet Varian A60 are duplicated to sub-parts-per-billion (allowing for frequency) by a multinuclear pulsed FT supercon Brucker 300 MHz Avance NMR or 900 MHz WB US^2 50 years later. Real physics works.

Do right and left shoes violate the Equivalence Principle? Do single crystal solid spheres of enantiomorphic space groups P3(1)21 and P3(2)21 cultured quartz violate the EP? That a fundamental postulate like vacuum isotropy could fall to a parity footnote is ridiculous - just ask Yang and Lee.

"An open-minded physicist is willing to believe any wild theory - once it's been shown to work" An open minded scientist is also willing to accept when a theory is shown to be flawed. I find it interesting when areas commonly defined as pseudoscience try to justify ignoring consistently negative results by misinterpreting the scientific method. Nice round up, but I fear you are howling at the moon.

Maybe we're all stuck in The Matrix, and everything we see is an illusion. Unbridled nihilism has something philosophical going for it, but not much practical. Everything we know about the world is inferential, and at some point we have to expect what is practically certain and what has no real prospective value to it whatsoever.

Science journalism in general is interested in the cutting edge, and the cutting edge is much less certain than the well-established. I think the average nonscientist who reads the paper comes across new ideas that contradict each other every few years, and they figure that means quantum mechanics and special relativity are probably pretty kooky as well. It would be nice if there were more room for conveying which things have scientific consensus behind them and which are more speculative. Scientific consensus isn't guaranteed to be true, but if the basics of special relativity weren't a much better description of the universe than Newtonian mechanics, nothing like the tevatron (for example) would act in nearly the way it does.

Finally, it's worth noting that the criticisms of string theory from outside physics are largely those that started inside physics. String theorists are a lot more humble than they were in the mid eighties, and may even be closer to some calculations that bear on measurable reality; back then Glashow suggested that, until they made a testable prediction, they be confined to "departments of mathematics and schools of divinity", and even string theory might better be considered a branch of mathematics than of physics. String theory, as you say, is not all of physics. When parapsychology can provide the basis for a faster microprocessor we'll treat it on par with physics.

"We know how mass creates gravity".

Do we? Really? Just how, exactly, does mass create gravity? Description does not imply understanding.

I think the key point for me, though, would be that no one is treating patients with string theory.

More importantly, Mary, no one is making any quantitative predictions from string theory. "We will see something new at higher energy" is not a theoretical prediction, it is speculation. Similar speculation drove the construction of the AGS at Brookhaven in the late 1950s, when particle physics was still in stamp collecting mode. Those speculations, based on little more than unitarity, were correct. Similar "can't miss" arguments have been applied to the searches at the LHC and, originally, the SSC.

"Strings will predict the mass of the electron". Or not. Only those who heed Glashow's insult will make the transition from mathematics to physics.

Nothing in physics today depends on whether string theory is correct or not. The standard model could be based on something completely different. In contrast, relativistic corrections from QED predict that gold will be yellow. No accelerator is needed to test that.

#5, we understand how mass generates gravity because we have a set of equations which describe gravitational phenomena with (thus far) perfect accuracy. At that point the physics is done. Asking "why" about fundamental theories is philosophy.

"#8
#5, we understand how mass generates gravity because we have a set of equations which describe gravitational phenomena with (thus far) perfect accuracy. At that point the physics is done. Asking "why" about fundamental theories is philosophy.

Posted by: Matt Springer | April 19, 2009 3:00 PM"

Call it philosophy if you will Matt, but if you don't know the why, you don't understand.
-Frank

By Anonymous (not verified) on 19 Apr 2009 #permalink

Call it philosophy if you will Matt, but if you don't know the why, you don't understand.

See, this is the kind of crap i have to deal with. In "operational" fields, such as medicine, what is most important is knowing what works. My problem with all these pseudoscientists isn't that they aren't creative, or that they are wrong---it's that they don't fundamentally understand the difference between proving something is safe and effective, and simply wishing it were so. When their imagination isn't congruent with reality, rather than alter their ideas, they fuss about how unfair reality and its proponents are.

Do we ever get the feeling that scienceblogs.com is preaching to the choir?

Who exactly is reading this post who might be swayed out of a thorough-going belief in science (the "scientific method", the results of recent science, physics, chemistry, biology, the mathematization of the universe, etc)?

By Anonymous (not verified) on 20 Apr 2009 #permalink

#8, we understand how mass generates gravity because we have a set of equations which describe gravitational phenomena with (thus far) perfect accuracy. At that point the physics is done.

If this is the definition of physics (and you seem to imply that physics is the paradigmatic science, therefore this is the ideal definition of science), how is medicine or biology ever scientific?

At the risk of being labeled a ID-believing bigot who is just stupid, I ask the following questions:
How do you distinguish between science and pseudo-science?And rather than speak in generalities about the thick-skulled nature of people who believe in pseudo-sciences, can you give some examples of specific practices in a pseudo-science that make it problematic?

By Anonymous (not verified) on 20 Apr 2009 #permalink

Anon#2, if you accept methodologic naturalism as the best way to go about understanding the universe, then pseudo- and real science differentiate fairly quickly.

Oh, and biology has certainly established itself as a science. Ernst Mayr describes a few things that helped that along. First, the recognition that cosmic teleology has no place in biology (or other science for that matter). Next that vitalism is invalid. Finally, the discovery of molecular bio and genetics.

Finally, as to pseudo-science examples, it's really everywhere. In my own field for example, we have people who make claims for homeopathy. These claims if true would require us to overturn much of physics and chemistry as we currently understand it. Also, studies have shown it to be ineffective. Despite these two facts, adherents continue to believe. This failure to dump a crappy hypothesis is at the heart of pseudoscience.

Doesn't "methodological naturalism" assume causal explanations, rather than merely mathematical descriptions? Regardless of any "perfect accuracy," mathematical formulae that describe phenomena without offering natural causes would seem to fail to be science. In this strict reading, then, "a set of equations which describe gravitational phenomena with (thus far) perfect accuracy" fails to be science (unless we are willing to fetishize "gravity"--maybe that's what we have to do).

I'm not saying physics or the set of equations isn't science, I just would like to know how to distinguish between science from non-science (or pseudo-science). I'm happy to accept methodological natural (a.k.a., scientific naturalism), but I fear that such a basis excludes too much or too little.

And let me add, I'm not trying to be a pain in the a$$. I am just interested in hearing from practicing scientists their logical and coherent arguments for demarcating laudable science from laughable pseudo-science.

By Anonymous (not verified) on 20 Apr 2009 #permalink

@13:

sometimes it can be hard to distinquish real science from pseudo science. especially if you have limited experience in the field in question. often you have to look at the credentials of the institution or person promoting the science. still, there are guidelines to follow.

here is a quick answer wikianswers.com:

"Pseudoscience is usually identified by the following criteria:

1.Use of vague, exaggerated or untestable claims
2.Over-reliance on confirmation rather than refutation
3.Lack of openness to testing by other experts
4.Absence of progress
5.Personalization of issues
6.Use of misleading language

If you take astrology as an example; it qualifies (in spades) under criteria 1, 2 and 4: astrological predictions are classically vague, "negative" results are often ignored, essentially astrology is the same as it was thousands of years ago."

i will add homeopathy qualifies under criteria 1,2,4,6.

and intelligent qualifies under criteria 2,3,4,5.

keep in mind that this is a list of common features that distinguish science from pseudoscience. it is not exhaustive or definitely indicative of pseudoscience.

ultimately a scientific theory has to stand up to falsification by others and be supported by empirical evidence.

@16:
Thanks for the list of criteria. It is certainly a nice heuristic starting point. As I struggle with these and the difficulty in grounding a distinction between science and non-science, I have two questions:
First, about labeling pseudo-sciences.
Second, about identifying sciences.

First,
Yes, astrology is the typical pseudo-science whipping boy (homeopathy and ID are lucky even to be labeled pseudo-science). But if we move beyond the horoscopes in newspapers and other popular press, how does astrology meet criteria 1, 2 and 4 (though it's not clear why 4 should be a hallmark of science)?

For the record, I don't believe in astrology and am vaguely bemused by those who do. But the practicing astrologers I have encountered (I have a strange life that brings me into contact with all sorts of people, including practicing alchemists (don't ask)) make rather specific claims (usually explanatory rather than predictive) that are as testable as many claims in other historical sciences (e.g., geology). They seem rather keen to adjust and modify their explanatory/predictive algorithms when new "data" becomes available (e.g., they readily incorporate new astronomical discoveries such as Pluto and other planetary type objects) and even to reject certain parts of the approach when adherence leads to inconsistent or contradicting conclusions.

Second,
How far can we press falsification and support by empirical evidence? Certainly Karl Popper tried to argue that science ought to progress by attempting to falsify theories (not simply individual statements or propositions), but it's unclear how successful he was. Both as a description of how scientists in fact work and whether or not scientists ought to work that way (both historians of science and philosophers of science as well as scientists turned philosophers/historians of science seem to question both aspects: see people like Kuhn, Lakatos, Putnam, Hacking, Pickering, Sokal). It seems that much science does not try to falsify a theory (or even a statement), but tries to exploit the productive nature of a theory or statement (productive here points to the predictive or explanatory fecundity of a theory). And, how do we agree on an interpretation of empirical? As the Poincaré-Duhem debates of the 19th-century illustrated and the failures of the logical positivists demonstrate, what constitutes empirical is problematic. Moreover, there seems to be lots of science currently taking place that is confined to the theoretical spheres. Can we not call string theory scientific until an empirical test confirms it? How do we save room for these a-experimental sciences? Or do we claim that they are not science until they produce a prediction that can be verified (ignoring for the moment the problems with verification) through empirical testing?

Thanks in advance for any thoughts and comments.

Best,
Frank

Frank,

Since I'm not a Platonist, I don't think there are airtight definitions which sort "science" and "non-science" into two (and only two) mutually exclusive boxes.

As far as I'm concerned, science is what scientists do, but some science is more believable than others.

And some, at the moment, is just playing:
http://xkcd.com/171/

By Roger Sweeny (not verified) on 24 Apr 2009 #permalink

To anonymous at number 13: "How do you distinguish between science and pseudo-science?"

Pseudo-science is where scientific terms and 'sciency-talk' are used to justify something in a way thats not scientific. And nowadays it is used a *lot*. For example I recently saw an advert for bottled mineral water that claimed that studies had been done that showed that drinking 1.5 litres of their water a day helped to make you feel healthier, because 'the water is filtered through four layers of vital mineral-filled rock'

That is pseudo-science. it's rubbish. Water is good for you because it's water; it doesn't matter how many layers of limestone it's gone through. They are making sciency-sounding claims with no evidence, hoping that the sciency tone of voice will lul people into believing that they have some higher authority somewhere.

"specific practices in a pseudo-science that make it problematic? "
The main problems with pseudoscience is that they are used to generate huge amounts of money for no adequate reason and, most importantly, they undermine people's faith in science and scientists. When something that looks and feels like science is shown to be rubbish, people tend to distrust proper science as well. And this is dangerous; in an extreme case:

"Well obviously tap water is just as good as bottle water, therefore the advert is rubbish, therefore so called 'scientists' don't know what their talking about, so abviously all this stuff about antibiotic resistance is rubbich so I can take those pills from my doctor anyway"

"how is medicine or biology ever scientific? "
Because it looks at things critically, questions things all the time, and comes up with working models that are useful. That is science.