Posted by 
“We’ve learned from experience that the truth will come out. Other experimenters will repeat your experiment and find out whether you were wrong or right. Nature’s phenomena will agree or they’ll disagree with your theory. And, although you may gain some temporary fame and excitement, you will not gain a good reputation as a scientist if you haven’t tried to be very careful in this kind of work.” -Richard Feynman
Did you hear the news? A game-changing story about the Universe has just come out! Something is vastly, spectacularly different from the way we thought, and it will revolutionize the way we think about the most basic, fundamental properties of our very existence.
Image credit: U&I Software, http://www.uisoftware.com/.
Blah, blah, blah. Or, you know, not.
Extraordinary claims like this come out all the time: cosmological inflation is unnecessary, neutrinos can travel faster-than-light, our experiment has detected dark matter, the fundamental constants aren’t really constant, and so on. If you keep your ear to the ground (or listen to the more speculative sources), you’re bound to hear at least one or two of these a month.
And they’re also almost always wrong. (And I hesitated there to put almost in that sentence.) Here’s why.
Image credit: D. Horn et al., Phys. Rev. Lett. 36, 1236-1239 (1976); the Oops-Leon particle.
Let’s say you’ve got a new piece of data, one that doesn’t line up with the presently best-accepted theory. Maybe you made an observation that didn’t line up with what that theory predicted. There’s a first-line-of-defense that this data must pass: is this observation robust?
In other words, when you perform this experiment or make this observation over and over again, will you get the same results? Will you see the same signal?
Image credit: Null Result from the Center For Inquiry, re: Power Balance bracelets.
Assuming the answer to this is yes, and the experiment / observation is repeatable, there’s another test you must do: do you understand where your errors and noise come from?
This one is hugely important, and one where practically all of the claims fail. Let me explain.
Image credit: J.K. Webb et al., Phys. Rev. Lett. 107, 191101 (2011).
This is the graph that shows how the fundamental constant, α, varies throughout space. The problem with this data, unfortunately, is that the variation is a signal superimposed on a giant amount of noise, and we don’t understand what causes that noise.
In other words, we can’t say “observation X shows that Y is happening,” because Y doesn’t fully explain observation X; nothing that we know does! Until you understand your background and your noise, you have no business drawing fundamental, scientific conclusions about what you’ve seen.
Image credit: R. Bernabei et al., (2008), from http://xxx.lanl.gov/abs/0804.2741.
This is the same story with DAMA or CoGeNT‘s claims that they’ve detected dark matter: they see a modulation in their signal as their detector moves about the Sun. But what the graph, above, doesn’t show you is that this “signal” is pulled out of a background that’s 50 times as large.
What is this background that we’re looking at? They have no idea.
Is it irresponsible to conclude “therefore, dark matter?” You bet. Because if you don’t understand the forest, you can’t draw conclusions about the trees.
Image credit: CERN / LHC, via research.gov.
There’s always a lot of hype floating around, because people get excited about new, revolutionary ideas. I can’t blame them for that; I do, too. But this is science, and we have high standards of evidence here! For instance, if you’re going to claim that the Universe is going to end soon, because the quantum vacuum is unstable, you’d better check that the quantum vacuum is actually unstable. (It’s actually incredibly stable, as people have been saying for years.)
The Higgs Boson is exactly the mass it needed to be to stabilize the Standard Model all the way up to very high energies, well within the error bars of all measured particle masses. So you can read all the hype stories you want, but they’re not right.
Image credit: ©2001 CERN.
The next thing coming down the hype-machine pipeline is surely going to be the Alpha Magnetic Spectrometer‘s results, due for release in a couple of weeks. Flying in orbit around the Earth, it’s detecting cosmic rays, or high-energy particles originating from the Sun, the galaxy, and even extragalactic sources in the Universe!
It’s conceivable that annihilating dark matter can cause an excess in cosmic rays. After all, one can easily design a scenario where that’s exactly the case.
Image credit: MAGIC collaboration / U.A. de Barcelona.
Does this mean that we can look at the data, and know right away whether we’ve seen dark matter or not?
Almost definitely not! There are a ton of sources for cosmic rays, and they’re very poorly understood. It would be shocking if there weren’t an excess of cosmic rays in some locations at some energies; we don’t fully understand where they come from or how they’re produced.
But you can bet that won’t be what the news outlets (and Sam Ting) will tell you. But remember.
Image credit: Center for Inquiry.
Always remember, extraordinary claims require extraordinary evidence.
For a scientific theory, that means:
- The new theory must be consistent with everything that came before,
- The new theory must explain this new observation, and
- It must lead to a new prediction of an observable phenomena which can go out and be tested.
Don’t be fooled by these claims; they’re a dime-a-dozen. But one that holds up to scrutiny?
Now, that’s science. Not hype.
February 20, 2013
February 20, 2013
February 20, 2013
February 21, 2013
February 21, 2013
February 21, 2013
February 21, 2013
February 21, 2013
February 21, 2013
February 22, 2013
February 22, 2013
February 22, 2013
February 22, 2013
February 22, 2013
February 23, 2013
February 23, 2013
February 24, 2013
February 24, 2013
February 24, 2013
