I'm a little late on this, but Avery Comarow, the reporter who wrote the big story three weeks ago in U.S. News & World Report about the infiltration of woo into academic medicine, has responded to criticisms of his column in his blog.
His response, I'm afraid, is underwhelming.
First, he starts out with the claim that he is an "evidence" guy:
A few words about that story and how it evolved. Regular readers (scratch "regular"--anybody who's read a single post) can tell I'm an evidence kind of guy. Claims without backup data give me the urge to turn the page, click on another link, get off the phone. Call it a bias if you want, but every medical reporter knows it's a crucial filter, considering some of the stuff we get from researchers (from their PR agencies much of the time) and publishers.I suggested an article on alternative medicine because academic medical centers all over the country--venerable altars of clinical research and practice like Mayo and Duke, top-ranked cancer centers, and even children's hospitals--are scrambling to roll out therapies that five or 10 years ago most regarded as dubious at best, crackpot at worst. Acupuncture, homeopathy, herbs, traditional Chinese medicine. It's a fascinating development. And I vowed to report it with an open mind.
The problem is, Mr. Comarow's mind appears to have been so open that his brains fell out for this story. He freely admits that the one thing he discovered at every academic medical center that he contacted or visited was "that not a single researcher or clinician bothered arguing that the evidence for any of the alternative therapies they were testing and using on patients was persuasive. To the contrary, all agreed that almost none of the studies that show positive results have been designed or run very well."
Sadly, none of that stopped Comarow from writing something that is deeply disturbing to see coming from a reporter:
If I wanted an evidence base, I was out of luck. But absence of evidence, as the late astronomer Carl Sagan said, is not evidence of absence. And if we lack an understanding of or explanation for how something works (as was the case for decades for how an airplane could stay airborne), that doesn't give us the ammunition to state that it doesn't work. In philosophy, that kind of reasoning is called argument by ignorance. Heaven forbid I should be guilty of a sin with "ignorance" in its name.
Yet, arguing from ignorance is exactly what Camarow has just done! He has in essence stated that, just because we don't know how these therapies work (never mind that we don't even "know" that they do anything) doesn't mean that they aren't necessarily effective. In other words, just because we don't know how ghosts pass through walls mean that they don't. He's basically arguing that, because science can never prove a negative 100% we should take dubious, physically and scientifically highly implausible CAM therapies seriously as scientists. This sort of argument totally belies Camarow's claim of being an "evidence guy," as does his utterly risible analogy to the airplane. We know the airplane worked because it flew! There was repeatable, unequivocal evidence that airplanes could fly. All that remained was to figure out the physical laws that allowed that to happen, some of the physics of which was understood years before the Wright brothers hit Kitty Hawk with their contraption. There is no such unequivocal, repeatable evidence of efficacy greater than that of a placebo for the vast majority of CAM therapies. Moreover, just because he "wrestled" with the story for weeks before publishing it does not make it any more valid to have written a mostly credulous and approving account of the infiltration of unscientific medicine into the bastions of academia than if he had had no doubts and published it immediately upon completing his research. Worse, Camarow lacks any understanding of how anecdotes are not scientific evidence and how they are inherently misleading:
Some of these therapies, maybe most, do indeed work. The patients I spoke with told me how acupuncture had made their allergies go away, how they were able to avoid painkillers after major surgery because of hypnosis or visualization or other mind-body techniques, how a homeopathic remedy that science would regard simply as water reduced swelling and pain within hours after an injury. I heard many such anecdotes, along with candid appraisals of treatments that seemed to be effective only for a short time or not at all. These people were not all true believers.
Yes, well we also used to think that bloodletting "worked" based on anecdotes. We used to think that purging with toxic metals "worked' based on anecdotes and tradition. Based on anecdotes and the all-too-human tendency to confuse correlation with causation, there are large segments of the population that believe that vaccines cause autism, despite several large, well-designed studies that have found no correlation and no evidence of causation, and that quackery such as chelation therapy "cures" autism, despite no good evidence that it does. Finally, he should be aware that, because of the vagaries of scientific studies of hypotheses with a low prior probability of being true, there is a high rate of false positive results, producing apparently positive studies when there really is no effect. That's why the totality of the literature has to be considered. Surely a reporter such as Camarow should know that, but apparently he doesn't. Worse, he demonstrates it even more pointedly in the comments, where some of his responses to skeptics, including the Amazing Randi himself, are weak at best and pathetic at worst. In a followup post, he backtracks a bit, but still seems unrepentant
Camarow calls his blog Camarow on Quality. Sadly, as far as his story and beliefs about alternative medicine and its infiltration into academia, his blog title appears incongruous in the extreme.
The aeroplane analogy annoys me immensely. It is certainly an argument from ignorance.
People like George Cayley had a pretty good idea of the lift/drag properties of wings and understood enough about stability to design workable gliders almost ten decades before the Wright brothers.
By the 1890s Lanchester in the UK and Prandtl in Germany were investigating fluid flow on aerofoils and coming up with sensible theories to complement the mechanical forces which were known.
Some people knew how aeroplanes flew even before there were aeroplanes!
(at least if you define aeroplanes as powered, controlled, heavier than air flying machines)
Then go and tell Mr. Camarow! ;-)
Yeah, that analogy was horrible. I suppose we should be grateful that he spared us the even more wrong bumblebee analogy.
anybody who's read a single post) can tell I'm an evidence kind of guy
Now, wait, do we interpret that sentence as him saying that he's an "evidence" kind-of-guy, or an "evidence, kind of" guy?
I don't know why all these medical centers _do_ adopt CAM, but I suspect that for many it's simply a way to harness "the power of placebo". I wouldn't hesitate to tell a patient that "many people find that guided visualization helps reduce post op pain" - it's easier than saying "here, take these sugar pills" without lying.
Gilad wrote "I wouldn't hesitate to tell a patient that "many people find that guided visualization helps reduce post op pain"
Do you have such information? It is the lack of such information that leads us to object.
I am concerned that woo is bleeding out of medical education and into reporting by MD journalists. My local NPR station (WAMC, Albany, NY) has a nationally syndicated "Health Show" which is nominally headed by an MD. Two months ago they broadcast tips from a naturopath (!?) on cancer prevention. I was preparing a delicately-worded letter to the MD, explaining that the ND was factually wrong, and is a quack even if she were right about that. Then (weeks later), they broadcast the comments of another ND. I realized that the doctor (Nina Sax) in charge of the show would be refractory to anything I might say. (Actually, she was probably the last person at the station who did not have me listed as a crackpot because of my complaints about their coverage of sCAM).
Predictions are always hard, especially about the future; but I can see naturopaths, homeopaths and their ilk being licensed in all 50 states (just like chiropractors).
"Do you have such information? It is the lack of such information that leads us to object."
I don't think so. I think it's understandable reluctance to support things that sometimes claim to be more than they are. I would never claim it does better than a placebo in clinical trials (I doubt it does).
And yes, I do have such information. Many people telling you guided imagination made them feel better is only an anecdote when it relates to the claim "guided imagination makes one feel better". It is not only evidence of a fact, but it is almost the fact itself, with regard to the claim "many people find that guided visualization helps reduce post op pain".
Requiring drugs to have more than the placebo effect is simple (and necessary) science we need in order to understand what effect a drug has, and what effect the mere administering of treatment had (regardless of its nature). This does not mean the placebo effect can't help control pain.
I don't know why all these medical centers _do_ adopt CAM, but I suspect that for many it's simply a way to harness "the power of placebo". I wouldn't hesitate to tell a patient that "many people find that guided visualization helps reduce post op pain" - it's easier than saying "here, take these sugar pills" without lying.
The first problem that comes to mind with this logic is "exactly how much more is the customer paying for guided visualization than they would have for sugar pills?"
I dunno, I just find myself thinking a lot lately that the western society of today would be very different if people just assumed that every time they find themselves constructing a thought of the form "well, it's not technically lying", that means they're doing something wrong...
This guy's a science reporter? That's gives me a sad.
People spend years studying their branch of woo. If it's just placebo, it's an enormous waste of people's resources. Imagine if all the NP and homeopaths and accupuncturists became doctors, nurses, and PA's.
Oh, wow. I just had a really scary flash there. But imagine they weren't stupid and guillible, too... it would be a good thing.
Gilad,
So, I take that as a "no" you don't have any good evidence in favor of guided imagery relieving post-op pain.
Joe - I'm not trying to sell you guided imagery (or any other similar technique). I'm trying to understand why institutes that usually make fairly rational decisions are including these treatments in curriculum's and health care plans. One answer would be "because CAM is popular, and it gets us students" (or "gets us patients"). However, I doubt the majority of the better medical students (those sought by schools) base their school choice on the existence of such courses. The "patients" hypothesis could be correct.
I would be (more than) very surprised to learn Homeopathy works better than a placebo drug - this would contradict what I know about chemistry (and, in fact, physics). I would not be surprised to learn that, e.g., Shiatsu works as well as a placebo. Say you had evidence that 90% of post-op patients said Shiatsu made them feel better (I don't think you'd get that high a percentage, but this is a hypothetical). Obviously this wouldn't mean it has any effect beyond the psychological, but would you recommend it?
P.S.
What if you were told 90% of patients prefer the person operating on them call them by their first name, because it reduces pre-op tension - would you recommend that?
Gilad,
Perhaps I was not clear. If you can provide reliable evidence for the efficacy of "guided imagery" or "shiatsu," it will be called "medicine" (not sCAM). The problem we are addressing is the lack of evidence for those, and other, methods for which people are paying money.
Joe,
No need to treat me like a simpleton. Efficacy is, indeed, measured against a placebo by default. But a placebo doesn't do "nothing" - quoting Wikipedia (I'm a bit too lazy to go further): Perhaps Graves (1920) was the first to speak of the placebo effect, when he spoke of "the placebo effects of drugs" being manifested in those cases where "a real psychotherapeutic effect appears to have been produced"<\i>.
In pain control, in particular, the placebo effect can be significant. If your only objection is a preference to guided imagery compared to sugar pills, that's fine. If you think sugar pills can't help a patient manage pain better than no treatment, I believe you're wrong. If you think it's immoral to give a patient sugar pills (or a pamphlet about guided imagery) despite them being useful, I disagree with you.
P.S.
Don't drink and post. And if you do, preview your HTML tags. :-)
The bit about the airplane nearly made my head explode. Where did he get his science reporter's credentials, in a Cracker Jack box?
Gilad writes:
But a placebo doesn't do "nothing"
Isn't it more the case that the patient does all the "work"? Placebos do, in fact, do nothing. They have no effect on an unconscious patient. It's the patient's mind that's "doing something."
Marcus Ranum said "The bit about the airplane nearly made my head explode."
That seems to an emerging misinformation bit amongst the woo. Perhaps because the bumblebee flight has been thumped on so many times. It is the second time I've seen it.
Last I looked at his blog, someone told him to go visit the National Air and Space Museum.
Just checked it again. I wish I had not. He claims to have read the book "Snake Oil Science"
The stupid, it burns.
Gilad, Concerning what you call sugar pills, see
http://www.sciencebasedmedicine.org/?p=29
All I ask is proof. Not what you believe; but facts.
I have to object to Comarow's "Argumentum ad Aphorism" - the argument that something is true because a smart and/or famous person said it:
"But absence of evidence, as the late astronomer Carl Sagan said, is not evidence of absence."
Carl Sagan may have been a genius, but he was not infallible. Nor are his utterances applicable in all situations.
Besides, even that aphorism is being used incorrectly. An absence of evidence may not be evidence of absence, but it isn't evidence of existence, either.
Prometheus
That's why Comarow's argument is truly the classic "argument from ignorance." He seems to be arguing that an absence of evidence is evidence that this woo works.;-)
Joe,
Nice, interesting link. In it, and in the previous entry he links to, Dr. Novella discusses the placebo effect and mentions that in certain, very specific fields, it does have an effect. Pain management, the field I discussed, is mentioned, but considered "hard to measure". I understand the effect of Naloxone on placebo pain relief as evidence of real action, albeit indirect, and would be very interested in an alternate explanation.
Don't misunderstand me - faced with people claiming that "guided imagery uses quantum mechanic principles to effect the body" (actually heard that) I find it hard to recommend anything like it, but feel one must do something. Should you just say "Get over it - if I lied to you and claimed I cured it you'd feel better"?