Over at Scientific American, John Horgan has a blog post titled In Physics, Telling Cranks from Experts Ain't Easy, which opens with an anecdote any scientist will recognize:
A couple of decades ago, I made the mistake of faxing an ironic response to what I thought was an ironic faxed letter. The writer--let's call him Tachyon Tad--had "discovered" a new physics, one that allowed for faster-than-light travel. In my reply, I told Tad that if he built a warp-drive spaceship, I'd love to hitch a ride. Dumb joke. For months, my fax machine churned out sheets covered with Tad's dense elaborations of his theory and plans for a superluminal machine.
After that, it kind of goes downhill:
After that, I simply chucked cranky letters. What else was I supposed to do? I had neither the time nor wherewithal to find the flaws in their logic, any more than I could double-check the math that yields, say, quantum electro-dynamics or some spiffy new variant thereof. As a mere journalist, I relied on experts to do that for me, especially ones at fancy institutions like Caltech and Cambridge, who presumably had been thoroughly vetted. My job isn't to uncover scientific truth, I told myself, but to report on what professional scientists think the truth is.
As Tom has already noted, it's not actually that hard. I can give you a big hint on how to distinguish cranks from experts: Experts don't mail/fax/email unsolicted theories of everything to random science writers. If the latest theory of everything crossing your desk comes direct from the "expert" who claims to have developed it, rather than, say, from the press office of a major university or research institute, odds are that it's from a crank.
The framing is doubly unfortunate, as the post is mostly a plug for a new book by Margaret Wertheim about fringe physics, which actually sounds a good deal more interesting than the way Horgan uses it to set up a cheap shot at modern theoretical physics (I haven't read it, so I can't say). But really, as much as I enjoy funny stories about famous physicists who believed silly things, the fact that Pauli was nuts for Jung doesn't make it any harder to tell a whack job from a serious scientist. And as much as I have my doubts about the current state of theory, I don't think Brian Greene or Leonard Susskind overselling multiverse theories lends any extra credibility to nutty "quantum" explanations of ESP or whatever.
I mean, sure, you can write out a summary of string theory that makes it sound utterly preposterous. But you can do the same thing with ordinary quantum theory, making it sound about as plausible as "Ancient Alien" theories. And the thing is, we know absolutely and unequivocally that quantum mechanics works. To fourteen decimal places.
Now, there's a big difference between the readily testable predictions of nonrelativistic quantum mechanics and some of the wilder speculations of modern high-energy theory, which are beyond the ability of modern technology to test, and may never be fully testable. But this does not mean that the line between cranks and serious scientists has been significantly blurred.
It's really not that difficult to figure out. Serious scientists work as part of a broad community of other scientists working on similar topics. They engage with that community through a variety of fairly standard means: published papers and preprints on the arxiv, talks at conferences and workshops, etc. They're not loners working out of their own basements, blast-mailing their work directly to the media.
Do individual scientists sometimes believe goofy things? Sure. But the scientific community can compartmentalize that goofiness. Schrödinger got all fired up about Indian philosophy for a while, but other than an entertainingly incoherent ad that used to run in the back of Physics Today, this left essentially no trace in the wider physics community. Pauli was way into Jung, and obsessed with the number 137, but this was fodder for jokes, not anything that anybody else took seriously.
Some cranks attempt to mimic the structures of serious science, setting up PO box Institutes, or holding their own conferences with other like-minded folks. But these are generally a thin pretense, that should pose no real obstacle to even a journalist. If there's anything to an idea, there will be people working on it at universities that you've heard of. If all the speakers at a workshop are from mysterious institutions that even Google struggles to locate, chances are you're dealing with cranks.
Are there any places where the line does get blurry? Sure, a few. There are some topics in, say, quantum foundations that attract a small but highly passionate following. These still engage with the broader community, though, and follow accepted norms of the profession.
Does this mean that there's nothing wrong with the state of theoretical high-energy physics? No. There are a lot of problems stemming from the lack of solid experimental data to provide direction. But these are problems that, if pushed a bit, most of the people involved will admit are problems, and there are significant numbers of people actively trying to find something, anything, that can be used to test their predictions. But making a blanket argument that the entire enterprise is indistinguishable from crankery is the sort of unsupported claim that, well, you'd have to be a bit of a crank to make.
Pauli's 137 preceded the 1/137 for the fine structure constant, I take it?
I think the fine structure constant was known before Pauli's obsession with 137, but I'm not sure of the exact history. This is, however, the basis for a funny Pauli joke:
Pauli dies, and arrives in Heaven. When he's brought before God, God offers to answer any one question. Without hesitation, Pauli asks "Why is the fine structure constant almost but not exactly equal to 1/137?"
God whips out a blackboard, and begins explaining. After a few minutes, Pauli interrupts, saying "Ah! There's Your mistake!"
Eddington claimed to have a derivation that the fine structure constant was exactly 1/136. When the measurements were refined and it turned out to be closer to 1/137, he found what he said was an off-by-one error in his derivation.
I'd say the closest thing to actual blurring of the lines is in the occasional episodes of "pathological science" that Langmuir warned about (Pons and Fleischmann's cold fusion was a classic example). In the workings of normal science, they tend to burn out pretty quickly and leave behind nothing but a core of increasingly cranky true believers.
"If there's anything to an idea, there will be people working on it at universities that you've heard of."
That's somewhat wishful thinking! Ideally yes, but this is too often quite dependent on the expected financial returns of the idea.
Ideally yes, but this is too often quite dependent on the expected financial returns of the idea.
Chad was referring to theoretical physics, for which expected financial returns are slim to none. There, it's true that not everybody working in the field is at a big-name institute or university, but the ones who aren't but are nonetheless doing legitimate work will be in close contact with people who are at such places. Some big-science experimental fields with large collaborations (high-energy particle physics comes to mind; there are other examples) will have a similar profile, with some people in obscure places but most of the team at one of a handful of major institutions. Yes, it can be different in other areas such as astronomy (amateurs still regularly make contributions in this field) or engineering (there are many ideas, more than can be pursued by R1 university professors and much of which is done by private companies who are trying to develop salable products).