Posted by [This is a very long post, a reply to Orac's (my respected SciBling at Respectful Insolence) equally long response to my also long original post that invited him to tell us what he thought separated his brand of medicine from the "alties" he frequently posts about. Probably most of you won't have the patience to wade through this. But a challenge is a challenge and must be met. Anyway, its Christmas Eve. Who's reading, anyway?]
I had to smile at Orac’s response to my “bit of a pot shot” across his bow with my chicken soup provocation, because that’s what it was, a deliberate provocation. And he didn’t disappoint. The pot shot produced a nuclear response. Which I sought and appreciated.
Orac doesn’t like the terms alternative and conventional medicine, but I don’t like the term “woo.” I don’t know where it comes from and find it creepy. But each to his own. So I’ll use the more accepted phrase, alternative medicine, as in “Complementary and Alternative Medicine” (CAM) and Orac can use “woo.”
First, though, I must defend myself against his charge of erecting a “straw man” argument.
. . . revere uses a strawman so dry that it threatens to spontaneously ignite. (Yes, I know I’ve used that one before, but I like it; so I used it again.) The strawman is that my position with regard to alternative medicine is that I dismiss them because “the treatment has not been shown effective via a Randomized Clinical Trial or other acceptable study design to work.” Evidence-based medicine is all about using the best available evidence to guide treatment decisions, as I have explained before in my usual long-winded manner. When randomized double-blind clinical trials are available, their results should be considered the gold standard, unless there are methodological flaws severe enough to invalidate them or cast grave doubts on their findings.
I think I was quite clear I was using the chicken soup story, which was on the wires that day, as an opportunity to inquire into something that interested me whenever I have visited his site. I have no interest in alternative medicine, but I have a real interest in epistemology and I wanted to know what his was. It seemed to me that Orac’s views are very similar to many others and he is an articulate and feisty writer. So I took advantage of the opportunity, judging correctly Orac would rise to the bait and we would all be enlightened.
In any event, I don’t know what my alleged straw man was (I can’t help observing that dryness doesn’t produce spontaneous combustion, but when you are really fond of a quip, a little scientific inconsistency shouldn’t be a barrier). I was not attacking Evidence Based Medicine (why the capitals, though?). Who could be against using evidence? I was asking what kind of evidence counted and why. In any event, Orac has done some of what I hoped: tried to explain what bothers him about the “alties.” But he defends himself against the non-existent straw scarecrow nonetheless:
Let’s take an acutely hemorrhaging patient. It doesn’t take an RCT [Randomized Clinical Trial] to tell that taking the patient to the operating room to stop the hemorrhage will save the patient’s life.
[snip]
The fact is, in surgery, much of the evidence upon which we base our clinical decisions does not come from classical double-blinded randomized studies.
His hemorrhage example is a genuine straw man. I think it was clear I wasn’t talking about things for which common sense and patently obvious mechanisms are involved or for which one cannot ethically do an RCT. I’m an epidemiologist and believe in the value of controlled observational studies, not just randomized trials, but even there I don’t think you need an epidemiological study to show that having the slats in a baby’s crib too far apart is dangerous. One accidental strangulation case report is sufficient. Maybe Orac was misled by my adding the phrase “other acceptable study design” and perhaps it is too formal sounding. It was meant to include any kind of evidence he would warrant as informative or reliable and my purpose was to find out what he thought that was. I still don’t know.
I grant that RCTs are not common for surgical procedures, for good reasons. What then, are the grounds for many of the procedures that are done? In other words, what is the next “best evidence”? I’m not saying there isn’t any. I’m just asking, as a non-surgeon, for a surgeon to tell me what justifies a lot of the procedures he and his colleagues do since they are, by his own admission, untested by his gold standard. He doesn’t say whether many are actually tested in any way, although I believe many aren’t. What are Orac’s standards beyond his gold standard or some good observational study design? Is it clinical experience? Is it an inference from some known mechanism? And when there are options, i.e., different ways to carry out a procedure or even a choice between medical or surgical management, what justifies his choice? I am asking because I think it is fair to ask these questions of someone as highly critical of others as Orac is. He gets to some of this, which we will get to, too. Let me dispose of some of things that Orac said along the way.
Orac and others who inveigh against “alties” often accuse them of magical thinking, a charge I considered perfectly fair. But here was what I said about that charge:
A variation of this is that the proposed explanation is “magical thinking.” That’s perfectly fair as a criticism. Unfortunately, proposed counter-explanations [like placebo effect] are also vulnerable. If one takes “the placebo effect” to mean an interaction between the mind and the body that produces an effect, that’s at the level of magical thinking in my book. If you provide a neurophysiological explanation that’s fine (assuming you can give one). But then the remedy is actually having a physical effect so what’s the problem? It’s like a psychoactive agent. Counter-irritation as an assumed mechanism [to explain acupuncture anesthesia] is another irritating wave of the hand that seems quite magical to me. What exactly is the mechanism of counter-irritation, say, for acupuncture anesthesia? If you give me one then aren’t you giving me an explanation for how acupuncture really does work? If you can’t, why isn’t that a “magical explanation”? Or is the complaint about some alternative therapies that they might work but have the wrong reason? In which case much of conventional medicine is probably also in big trouble.
As an aside, it turns out that Orac is skeptical of both the placebo effect and acupuncture. His attitude toward placebos I find a bit extreme. His standard seems to be that there is no real effect since it don’t really work because it can’t do the really hard tasks like curing cancer. Obviously there is a well-documented placebo effect, however. That’s why we use them as comparisons in RCTs. The real point of my point, however, was to try to ferret out the role that the proposed explanation or underlying theory plays in telling alternative therapies from conventional ones. Here is his answer:
Let me lay it on the line here: I am not so much disturbed by treatments that are efficacious for which our understanding of the mechanism is incomplete or lacking as I am about the attribution of mystical B.S. to such treatments or to making claims for such treatments when there really isn’t compelling evidence that they “work.”. As revere points out, there are a fair number of treatments in conventional medicine for which the physiological mechanism is poorly understood or unknown. However, unlike the case for alternative medicine, we actually do have compelling evidence to support the efficacy of such treatments, and it is not true that possible explanations for their efficacy fall outside the realm of scientific plausibility based on our current understanding of human biology. For the example of asbestos, there are any number of plausible biological mechanisms based on our current understanding of chemical carcinogenesis by which asbestos particles could cause lung cancer. We just don’t yet know which one(s) to attribute its carcinogenic affect to [NB: this is incorrect about asbestos, but the error on Orac's part isn't germane to the argument.] Revere mistakenly conflates such scientific uncertainty with my overall message about the implausibility or virtual impossibility of much of alternative medicine. What I’ve been talking about is alternative medicine therapies where the mechanisms by which they are said to work are highly implausible physically, biologically, chemically, based on what we know. Good examples of this include homeopathy, which, if true, would violate our present understanding of physics, chemistry, and pharmacology to the point that many well-accepted theories would have to be overthrown, and something like Reiki therapy, for which no “life force” (qi) has ever been detectable or changes in such a force noted in response to the ministrations of a Reiki master.
Note that Orac has slid over the cases where there is no demonstration of efficacy in conventional medicine, perhaps the major category. For the rest, this does seem to be just what I said it was, a confusion between whether something does work or not with whether the reason given for any efficacy makes sense to Orac in terms of his own understanding of what is allowable scientific explanation. We see the same arguments given by physicists who say that non-ionizing electromagnetic radiation cannot possibly have a cancer effect because this is physically impossible: it is non-ionizing and therefore can’t have chemical effects. Meanwhile there is a respectable body of good epidemiology to show cancer effects and epidemiology that fails to show it. This has to be sorted out. But if we were to listen to the physicists (and Orac) we wouldn’t bother any longer to do any studies.
Orac is willing to listen, however, but only if the evidence is absolutely clear:
Even for twaddle like homeopathy or Reiki, I would still be willing to consider throwing out what we know and massively revising it in the light of new knowledge if compelling objective evidence for efficacy were presented for the real efficacy of either homeopathy, Reiki therapy, or any other alternative medicine to which an amazingly implausible mechanism of action is attributed. None ever has, as of yet. As Dr. Mark Crislip of QuackCast (someone who is even more–shall we say?–insolent than Orac is, but nowhere near as respectfully so, and that’s saying something) put it in his podcasts on the evidence regarding homeopathy and acupuncture, as the quality and rigor of the clinical trial increases, the “effects” attributed to homeopathy and acupuncture tend to decrease until they are no better than placebo in the best clinical trials with the largest number of patients.
The confusion between a plausible mechanism and efficacy is evident here, too. He’d believe evidence that something works if we could present him with compelling evidence. Yet he doesn’t demand compelling evidence for efficacy for the many things surgeons and others do for which he has one or more (possibly contradictory) plausible scientific explanations, however he understands what a scientific explanation is. We also see here that placebos are assumed to have an effect, at least by Dr. Crislip. So if placebos really do work (and if they don’t then why test against them?), then all these alleged failures of acupuncture show is that they work, but no better than placebos. Let me say again, I don’t know if acupuncture works or not, nor do I much care. My interest is in what Orac thinks counts as a scientific explanation, which is the linchpin of his whole system.
At this point, Orac brings in our mutual SciBling AbelPharmboy of Terra Sigillata here at ScienceBlogs:
The basis for generating a hypothesis to test whether an alternative remedy has biological efficacy must have some scientific basis that is sound. I’ve heard herbalists tell me that “whole herbs” are superior to purified compounds because of synergy among components or because one herbal component blocks a side effect of an active component. These are both reasonable scenarios, but there is a great paucity of evidence in the literature to support such suppositions…
However, if an herbalist tells me that a whole herb works better than a pure compounds because the herb is “energized” or contains a “life force” that is extracted away by laboratory purification, I call bullshit unless there is a method to quantify these forces and demonstrate their causality with biological activity.
That’s fine, as a consistent criterion, as long as it applies to everything. But by Orac’s own admission, much of what we do in medicine cannot be quantified in that fashion. Few of us doubt that “bedside manner” is an important component in much clinical success (well, maybe we should excuse surgeons from this), but we can’t quantify it most of us wouldn’t call it bullshit.
We now come to Orac’s battlecry:
My battle cry as far as these so-called “alternative” therapies is, “Show me compelling evidence that they work if you expect me also to radically alter my view of how human biology works!” My corollary to this battle cry would also be that, if you want to get grant funding — taxpayer money — to study your woo, you should be able to show at least one of two things: biological plausibility or compelling evidence that the woo in fact does what you claim that it does, preferably (but not necessarily) both.
I think this is a very conventional view, and one to which I don’t necessarily object, but I feel compelled to make some observations about it. For example, it is a recipe for “in the box thinking” that is OK for everyday science but not very promising for innovative science. Orac makes mention of spending taxpayer money to study alternative therapies (a reference, perhaps, to his on the record objection to the NIH grant program that studies the efficacy of Complementary and Alternative Medicine), but it appears to be a self-fulfilling prophecy. You shouldn’t study if it works if there hasn’t been a study to show it works or you have reason to think it might. The history of science is littered with examples of things that “shouldn’t work” or “shouldn’t happen” (and are often denied to be happening at all) but that do, and whose eventual explication results in a major step forward. I not only do grant reviews for NIH (as does Orac) but I chair grant review panels, so I am well aware of the standards and practices that go on there. The problem of a bias toward “safe” science is one most of us who have been around a long time recognize and are concerned about.
Finally, Orac takes on the Demarcation Problem (separating science from pseudoscience) and references Janet’s excellent explication of Popperism at Adventures in Ethics and Science. I should let Janet speak for herself, but it wasn’t my impression she was advocating Popperism’s falsifiability criterion as a workable standard. Few philosophers would do that because it is so obvious it doesn’t work in practice. For those of you unfamiliar with the ideas, I refer you to Janet’s fine post on the subject, but here is my short, crude version for purposes of making a point. Popper says that a theory that posits that all ravens are black is never established by repeated examples of black ravens (confirmation) but it can be overturned by one example of a non-black raven. Thus there is an asymmetry which argues for a strategy of falsifying your propositions as the true hallmark of a scientist. Like many scientists, Orac buys this. Of course it doesn’t work that way. The response of the scientist to his colleague’s demonstration of a non-black raven is more likely than not going to be something like, “A raven? You call that a raven?” There are many responses to this, such as Quinean pragmatism, but that’s for another post.
Here I will only observe that Orac seems to accept that telling science from pseudoscience is a simple matter of seeing whether the explanation accords with his idea of what is conventional science. Would that it were so easy. I wanted to know what his idea of what science was, but all I got is that science is whatever he says it is or at least, whatever the books say it is. The idea that science is whatever scientists do is fine, maybe even the right way to look at it, but then the question comes down to who gets to say who is a scientist and why.
Part of Orac’s attitude may be related to his notion of physician exceptionalism.
In fact, here’s where real humility comes in and why this demarcation problem is somewhat different in medicine than it is in the rest of science. The difference is, boiled down to its essence, patients, and if anything distinguishes “conventional” medicine from “alternative” medicine, even when evidence may be scant for some “conventional” interventions, it is the humility in presenting his treatments to the patient. Yes, humility. However much people joke about the “God complex” of doctors, this attitude is nothing compared to the attitude among true woos. Despite their seeming otherwise, quacks and pseudoscientists are anything but humble and honest about their limitations. In non-medical areas, pseudoscientists often claim that scientists are wrong, that they know nothing, and that their woo will replace current science, all on minimal evidence. While such hubris usually doesn’t do that much harm, other than to the pseudoscientist’s reputation, in medicine it is disastrous to patients. Quacks claim to have the cure for cancer or even the cure for all diseases. They often disparage conventional medicine, claiming that their woo is better and can cure the patient without the risk of all those potentially nasty complications or side effects that we conventional doctors have to acknowledge and warn the patient about. Patients, afraid, desperate, and all too often distrustful of “conventional” medicine because of its known shortcomings, too often fall for the blandishment of quacks and suffer, often from a delay in receiving effective therapy until it is too late, accompanied by a draining of their bank accounts. Dr. RW described this difference well while asking When Woo Overlaps Mainstream Medicine, Can Patients Sort it Out?:
It seems to me that this boils down to, “Maybe we aren’t great, but they are worse.” I have no doubt Orac is a truly excellent physician and also little doubt the humility which he describes is characteristic of his own practice. But to say it is the norm is really asking me to believe a lot. The average clinical encounter continues to shrink and there is less and less time to do the things most physicians never did well anyway. My son recently had (successful and uncomplicated) arthroscopic surgery for an athletic injury. He never saw the surgeon and never had the procedure or the condition explained to him. The surgeon is a famous orthopedist practicing in one of the best and most famous academic hospitals in the US. He did an excellent job (I hope) and my son’s shoulder should be stronger than before. But this surgeon is more the norm than the exception. Orac’s rendition of the humble doctor as representative is even more incredible than Reiki. I would guess (maybe Orac has some data on this) that more people visit alternative practitioners after exhausting (and being exhausted by) conventional treatments that failed than instead of them.
I have no doubt there are many crooks purveying quack remedies for profit or because for some reason they believe they work. Some of them do a lot of harm, and it isn’t difficult to find examples. But it isn’t difficult to find examples of conventional practitioners who can be similarly described. In fact, we could do a Global Search and Replace in Orac’s paragraph above and interchange conventional and alternative therapies and come up with an equally common and equally valid description of a great deal of conventional medical practice.
Orac continues in this vein, claiming ethical superiority for conventional practitioners by virtue of their science:
. . . one other huge area of “demarcation” between woo and non-woo in medicine. And Dr. RW is also very correct in pointing out another important aspect of differentiating woo from non-woo in medicine: ethics. Dr. Maurice Bernstein also points out that it is up to the physician to try explain to the patient what is and is not evidence-based about his recommendations because it is his fiduciary and ethical duty to the patient:
What has all of this to do with medical ethics? It has to do with the physician’s duty to be trustworthy to the patient and to aim to do good and not produce harm to the patient. Informing the patient, unless some patient rejects being informed, is also duty of the physician and is part of the consent to treatment process. It should be a standard of medical practice that the patient understand what portions of the physician’s decisions regarding diagnosis, prognosis or treatment is based on reliable evidence and which is based on hearsay or simply statistically not verified experience. But do most physicians have the time or skill to explain to the patient on what basis their decisions were made? Further, do most physicians even know which of the “facts” they are using have not been rigorously proven? Unfortunately, even with the best of physician attention to educate themselves on the evidence and to explain this to the patients, there is much in medical care that will remain “old tales” either because there is not sufficient interest to provide the time and expense to do scientific studies or realistically because some studies just can’t be accomplished because of technical or ethical issues.
The real Demarcation Problem between quackery and evidence-based medicine, I would argue, is the willingness to be always striving to test our therapies against new therapies in the search for better ways to care for our patients versus stagnation, dogma, and unquestioning faith in a treatment on the basis of little or no evidence. Scientific medicine embraces the former; alternative medicine embraces the latter.
Dr. RW’s ethical prescriptions, referred to by Orac, are mostly honored in the breech, leaving conventional practice no more ethical than alternative practice and frequently far less caring. This also is quite an arrogant position. “We are superior because we care enough about our patients to test our therapies.” But of course most aren’t tested, for one reason or another, and most doctors don’t even know whether they have been tested or not. We learned them in school and we keep doing them. Orac and I are academic physicians and not at all typical. Furthermore, everything hangs on Orac’s claim of a “willingness to test,” yet he doesn’t make clear what he means. This shows that rather than being exceptional, the Demaracation Problem is the same for physicians as for other scientists, and his version of it is “testable.” This is a highly problematic criterion for philosophers, if not for doctors.
Finally, we have this:
As far as I’m concerned, and as I have said over and over again since the very beginning and even before [?!], there is no such thing as alternative medicine. In my book, the approach to evaluating the therapeutic value claimed for a treatment should be the same, regardless of whether the treatment is considered “alternative” or not. The gold standard would be an RCT, but if it’s not possible to do an RCT then the best existing lesser evidence must be accepted and evaluated as the basis for using the treatment until better evidence is developed, lest we suffer “evidence-based paralysis” or “RCTomyopia.” The patient’s disease won’t wait, in other words, and we as physicians have to do the best we can and use the best scientific evidence available to decide upon the best course of treatment for any given patient. Sometimes that means, in cases where evidence is very sketchy or conflicting, to use primarily anecdotal or personal experience as a guide, even though that’s far from ideal.
Yes, it’s a dilemma, because frequently there isn’t any evidence of efficacy, only an argument of plausibility. Here is Orac’s solution:
Emily correctly points out that conventional medical treatments aren’t always evidence-based. She asks “Is that not, in effect, a form of woo, too?” The breach between evidence and practice to which she refers, let’s call it the quality chasm, while every bit as serious as woo, is not in fact woo in most cases. The quality chasm is not a result of outlandish or implausible claims, (e.g. that water has memory) and that’s what distinguishes it from woo. The quality chasm results from an extremely complex interplay of cognitive and system barriers to the consistent application of best evidence in practice. Mainstream medicine recognizes the need for widespread system change and promotion of evidence based medicine. Although there’s no simple fix to the quality chasm, many in mainstream medicine are trying, which is in ironic contrast to the fact that mainstream medicine also increasingly promotes woo. For the mainstream, especially academic medicine, to promote woo is to engage in unethical scientific pretense and active deception of patients.
In other words, “woo” and untested conventional practice are the same except that conventional practice doesn’t make “outlandish” claims about how it works. “We” are better because some of us are trying to figure out what works. At the same time, Orac doesn’t like NIH’s grant program to actually test alternative therapies because he already knows they don’t work, it’s a waste of money and it gives aid and comfort to the enemy.
What it comes down to for me is not so much that I disagree with Orac about whether much of what alternative therapies is correct as much as I don’t like the certitude with which he says it, often, it seems to me, with uncertain grounds. You can be right for the wrong reasons and wrong for the right reasons, one of the issues raised by the claims and counter claims of conventional medicine and its rivals (and I put it that way deliberately, for historical reasons).
It doesn’t seem to me Orac has escaped this, no matter his degree of certitude.
December 24, 2006
December 24, 2006
December 25, 2006
December 25, 2006
December 25, 2006
December 25, 2006
December 26, 2006
December 26, 2006
Pickering’s excellent “The one culture?” (and way before that Gilbert and Mulkay’s “Opening Pandora’s Box”) show that while there are some scientists that indeed show this kind of epistemological naivet�, there is also a large contingent that takes a much more nuanced approach.
January 16, 2007