An Enthusiastic Amateur is Worse Than Any Pro

Kate points me to a real head-scratcher from Slate, about Harry Collins posing as a physicist. Collins is a sociologist who studies expertise, and also has a very strong interest in gravitational wave detection experiments. Collins and co-workers collected a bunch of qualitative questions about gravitational waves and detectors, and got an expert in the field to write answers to them. He then wrote his own answers, and sent both sets of answers to a bunch of people in the field, and asked them to guess which set of responses was from the expert in the field.

The surprising result from this is that the majority of the experts (seven of nine) picked Collins's answers as the one written by the real expert. Slate ties this in to Alan Sokal's famous stunt of a few years back, and gets some quotes from Sokal himself, but I really don't see what the point is.

I mean, is it really surprising that a non-scientist can master a scientific subject well enough to write a good, coherent qualitative explanation of it? After all, some of the very best science writers in the business are people who don't have science degrees. If you don't need a Ph.D. to write a book explaining physics to the masses, why on earth would you need one to answer a half-dozen qualitative questions?

The key qualifier here is "qualitative." Even Slate admits that Collins couldn't fake the mathematical details of the work he talks about-- he can present himself in the manner of an expert, as long as he doesn't need to actually demonstrate any real expertise. He's no more able to calculate the expected signal for LIGO from some odd configuration of colliding neutron stars than I am, but he can talk intelligently about the basic features of the field. But, then, so could just about anyone who spent enough time reading about the projects, and talking to people in the field.

The one really interesting thing in the actual article (PDF linked from Collins's page above), other than the fact that sociology journals apparently like to publish cartoons, is the reason why the experts picked Collins's answers: he didn't use any math. The real expert threw in a lot more technical jargon, and the expert judges thought that it sounded like somebody trying too hard to impress people. This probably indicates that most experts are deluding themselves about the degree to which they avoid jargon in explaining things to each other and to the public. Again, this isn't really news to anyone who reads science blogs, but it's sort of cool to see it confirmed.

But, really, I just don't see what the big deal is.

Tags

More like this

Your "very strong interest" makes it sound like he's an enthusiastic neophyte. In the Slate article, it says "Collins ... has made an amateur study of this field for more than 30 years but has never actually practiced it." I may not know that much about the science behind evolutionary theory, but after devoting the past six years of my life researching its origins, I'd wager I could sound like I knew what I was talking about. Multiply that 5x and that's a bet I'd win every time. It'd be more surprising if Collins wasn't able to convince the panel.

And something I haven't seen anyone mention yet - there's absolutely NO comparison to Sokal's prank. Collins wasn't trying to bullshit the panel and get them to support some cockamamie hogwash - he knew the subject, and elaborated on it well, that's all. Sokal babbled with big words and jargon, while saying nothing that was either accurate or of substance. No relation, no contest.

Hi Chad, I saw the story too and was amused by it, wondering what the big deal was. As in this post at The World's Fair. Ben

I heard this story sometime ago, and I have to admit, I do not understand the fuzz made about it. After all, Collins not only "has made an amateur study of this field for more than 30 years", he has also written an 800+ page book about the history of the experimental search for gravitational waves that has received very favourable critics, also by physicists involved. So, you really would expect that he knows quite a lot about this subject. Maybe if the questions would go deep down into technical details, they could tell "real experts" from him, but for the more "general" questions that were used in the example, I am not so surprised by the outcome.

One does not requite a license to think. A professional is responsible for his actions - authority contrained by responsibility and liability. Rogue professionals are reigned in by their peers.

Deep aquifer sequestration of CO2 proceeds at tremendous expense (PV = 101.325 J/liter-atm). Enviro-whiners bear no responsiblity or liablity. Somebody else suffering permanent loss bothers Enviro-whiners not a whit. CO2 and water together are tremendously corrosive to rock ("weathering") and exponentially more so when hot and concentrated. The catechism is to feel good about it with somebody else paying the tab. (Then, the blow-out re Sidoarjo, East Java.)

Rachel Carson's 1962 Silent Spring had a politically correct long list of endangered species. 45 years and a wasted $trillion later, has anything changed? Do you feel good about it? Would you prefer that science professionals rather than professional bunko artists were in charge?

A lot of the commentary I've seen on this has picked on the math, but though I would agree it's important to know the math, we can't underestimate how important understanding the concepts are, either. After all, there was a debate 'round this place some time back (can't remember on which blog) about whether mathematicians are qualified to speak about any damn thing that enters their head just because they can write an equation about it, and the prevailing opinion seems to have been that the answer is "no" (especially as it relates to evolutionary theory). If you don't have both - understanding and the math to back it up - you've got nothing.

I can't wait for the equivalent study regarding an 'educated layman' impersonating a priest.

*snark*

By melior (in Austin) (not verified) on 06 Oct 2006 #permalink

Yet another terrible Slate article that tries to simply create heat while shedding no light. The comments below the article are considerably better that the article itself. Slate is becoming ever more like Entertainment Tonight.