One of the things that distinguishes evidence-based medicine (EBM) and science-based medicine (SBM) is how the latter takes into account prior probability that a therapy is likely to work when considering clinical trials. My favorite example to demonstrate this difference, because it’s so stark and obvious, is homeopathy. Homeopathy, as regular readers of this blog no doubt know by now, is a mystical, magical system of medicine based on two principles. The first is the law of similars, commonly phrased as “like cures like”; i.e., the way to treat symptoms is to use a smaller amount of something that causes the same symptoms. I know, I know, it’s rather silly, but homeopaths really believe it. Although on some occasions it may be that the drug used to treat a disease might be capable of causing the symptoms of that disease, there’s no scientific basis to support this concept as a generally applicable medical principle. The second principle is the law of infinitesimals, which states that diluting these remedies makes them stronger. This leads to the well-known criticism that homeopathy dilutes its solutions many, many orders of magnitude more than Avagadro’s number, meaning that there is not a single molecule of the original remedy left. Indeed, a typical dilution for a homeopathic remedy is 30C, meaning 30 serial 100-fold dilutions. This is equivalent to diluting the solution 1060. Given that Avagadro’s number is only on the order of 1023, I trust that you can see the problem.
Not that any of this keeps homeopaths from paddling boldly up the river of pseudoscience, usually by proclaiming that water has “memory.” According to homeopathy, water somehow “remembers” the presence of the remedy and transmits its healing power to the recipient of the remedy, while somehow magically forgetting all the poop, urine, dirt, and other contaminants that it’s been in contact with. In other words, homeopathy is water; its prior probability of doing anything other than what a small amount of water can do is essentially nil. Moreover, on the basis of basic science alone, we can conclude this with a very, very high degree of confidence because, for homeopathy to “work,” many well-established laws of physics would have to be not just wrong but spectacularly wrong, as would many principles of chemistry and biology. Given the amount and quality of scientific evidence demonstrating that homeopathy can’t possibly work, to prove otherwise would take at least as much high quality evidence from multiple different sources, evidence of high enough quality to call into doubt these same laws of physics. Equivocal or weakly positive clinical trials whose results could be due to bias, placebo responses, regression to the mean, and other confounding factors don’t cut it.
In real science-based medicine, there will, of course, be gray areas, areas where the prior plausibility is not so clear-cut. However, where it is so clear cut, such as in magic-based therapies such as homeopathy, reiki, and various other “energy healing” methods, prior plausibility considerations put a much higher hurdle for clinical trial data to overcome if efficacy is to be demonstrated. For instance, if a homeopathic remedy unequivocally healed stage IV pancreatic cancer in multiple patients, then, prior plausibility considerations aside, we would have to take it seriously. No such evidence has ever been presented.
It turns out that homeopaths are very sensitive to arguments based on prior plausibility, so much so that they’ve apparently tried to turn them against us nasty, dogmatic, reductionistic “Western” scientists. (Oh, wait, homeopathy is “Western” too, having been invented by a German.) I just came across a hilarious article by someone named Peter Fisher entitled Plausibility Bias and the Controversy Around Homeopathy. It comes from something called the Homeopathy Research Institute which, amusingly, claims to facilitate scientific research into homeopathy. First, Fisher whines about the criticism:
Homeopathy has long been surrounded by controversy. As long ago as 1846 it was denounced as ‘ludicrously absurd’ and an ‘outrage to human reason’ and more recently it has been claimed that ‘Accepting that infinite dilutions work would subvert more than conventional medicine; it wrecks a whole edifice of chemistry and physics’.
Particularly galling to Fisher appears to be the Commons Science and Technology Committee report in February 2010, which concluded that “There has been enough testing of homeopathy and plenty of evidence showing that it is not efficacious.” Which there has, based on basic science considerations that show that homeopathy can’t work and all the clinical trials, which, taken in their totality, are most consistent with nonspecific effects and placebo responses. Taking the nonexistent prior plausibility of homeopathy into account and the equivocal clinical trial data, the most reasonable conclusion is that homeopathy does not work. Then, taking its history into account, it is further reasonable to conclude that homeopathy is pseudoscientific nonsense based on purely magical thinking.
None of this stops Fisher then rapidly segueing into an appeal to popularity and presenting the central idea of homeopathy (the law of similars, or “like cures like”) as though it were a scientific fact rather than an ancient concept based on sympathetic magic. He even invokes the same tired (and scientifically inaccurate) comparison to hormesis. Finally, he finally gets into the meat of his “argument,” such as it is. First he references a paper he wrote for Webmed Central, entitled Homeopathy In Upper Respiratory Tract Infections? The Impact Of Plausibility Bias. Basically, Fisher and colleagues try to turn the prior plausibility argument around, but they do it in a manner designed more to appeal to fairness than actual science. For example, here’s how Fisher presents the argument:
Responding to these issues Lex Rutten, George Lewith, Robert Mathie and I have recently introduced the concept of ‘plausibility bias’ based on analysis of the discrepancy between evidence.
Fisher notes that a lot of clinicians still use antibiotics for upper respiratory tract infections (URTIs), even though most are viral infections and copious data demonstrate that antibiotics usually do little or no good and do not decrease the risk of serious complications. Fisher blames this on “plausibility bias” for antibiotics because we have a mechanism by which they work and bemoan a “plausibility bias” against homeopathy in URTIs and in acute otitis media, citing clinical studies suggesting that homeopathy is efficacious in these conditions. Here’s where the appeal to fairness comes in:
We concluded that the differing conclusions of the meta-analyses of the homeopathy and conventional medicine subsets, and particularly the homeopathy URTI subset, do not reflect the nature of the evidence, nor its quality, but negative ‘plausibility bias’. Negative plausibility bias obstructs a fair evaluation of the evidence around homeopathy; its extent and implications have not been adequately recognised or discussed. It should not impede further research, but we must recognise that such new research in homeopathy, if positive, may have limited impact on practice until a plausible theoretical framework is established.
This is almost identical wording to that found in Fisher’s Webmed Central article. The argument is also nothing more than a gussied up version of saying “two wrongs make a right.” Think of it this way. Fisher is unhappy that so many clinicians continue to overprescribe antibiotics for conditions for which they are unlikely to be effective. So am I, actually. I do not like it when physicians practicing EBM do not follow the evidence. However, the answer to this problem is not to start using magic water for these conditions instead or, to paraphrase Fisher, to give that magic water a “fair evaluation.” The answer is for physicians to have more backbone in not prescribing antibiotics for viral illnesses and to educate patients that antibiotics don’t do any good for these conditions so that they don’t pressure their doctors into prescribing them. Homeopathy has already had a “fair evaluation.” It’s had an even more than fair evaluation. It’s failed time and time again. The homeopathy proponents can come up with are occasional studies that with barely statistically significant results that are far more likely to have been due to either random chance, poor trial design, or bias. It can’t be emphasized too much that none of this is adequate to overthrow the mountains of evidence in physics, chemistry, and biology that render homeopathy, if not impossible, about as close to impossible as can be conceived.
Fisher and company are then fairly clever in proposing a clinical trial of homeopathy:
The specific research we propose, conducted in primary care, is to test the hypothesis that homeopathy averts, reduces or delays antibiotic prescriptions in the management of acute otitis media or other URTI.
There are a number of considerations to be taken into account in designing such a study. These include:
- Trials using an antibiotic treated group in uncomplicated URTI are unethical since the evidence suggests that they do more harm than good in this situation.
- Trials should be of relatively long duration: URTI are often recurrent and the disadvantages of antibiotics are most evident in this context.
- Homeopathic prescribing requires skilled practitioners.
- Homeopathy is associated with strong patient preferences; previous RCTs in primary care have foundered on this issue .
The research question would not be: ‘Can ultra-diluted medicines exert physiological effects?’. A clinical trial is a clumsy and expensive way of answering such a question, which would be much better answered by biological models. A number of such models have been described , and their further development might enable homeopathy to become framed in the plausible theoretical background that it requires.
Unbelievable. Fisher and company argue that it is unethical to test antibiotics in a clinical trial of uncomplicated URTIs because they are ineffective and might “do more harm than good” but somehow think it’s ethical to test an ineffective therapy based on magical thinking in a clinical trial.They then go on to conclude that there should be a “pragmatic” and “large scale” controlled clinical trial. As I and others have pointed out before, “pragmatic” trials represent an inappropriate methodology to determine if a therapy works, because they are not blinded, often not well controlled, and therefore subject to all the biases to which less rigorous clinical trials are subject. Rather, the purpose of pragmatic trials is to test a therapy in “real world” conditions after it’s been demonstrated to be safe and efficacious in proper randomized clinical trials in order to see how much its efficacy and safety decline away from the ivory tower environment under which many clinical trials are run. Even then, there are lots of problems with pragmatic trials due to their less rigorous nature. In any case, using pragmatic trials to test efficacy is a blatant attempt to lower the standard of evidence.
Fisher also gets it backwards. For real medicine (as opposed to fake medicine like homeopathy) the usual order of business is to demonstrate effects in preclinical models (i.e., cell culture, biochemical assays, and animal studies) first before even phase I clinical trials, much less randomized clinical trials, much less pragmatic trials are even considered. Of course, it is these very basic science considerations that render the entire concept of homeopathy so utterly ridiculous from a scientific standpoint and produce estimates of its plausibility that are so very, very low.
Basically, what Fisher is doing is a tried-and-not-so-true (albeit frequently effective) technique of obfuscation by arguing that, since EBM/SBM is not perfect that “alternatives” are needed and that SBM/EBM practitioners don’t practice what they preach. He even tries to turn the most potent criticism of homeopathy (namely its utter lack of plausibility) against SBM by claiming that plausibility is just a “bias.” It’s true that SBM is not perfect, and there are a lot of physicians who use methods that are not adequately science-based. There are even practices in “conventional” medicine that go against findings from clinical trials. There’s no doubt about it; SBM can be messy, and rooting out non-SBM practices (such as antibiotics for viral URIs) can be frustratingly slow and difficult. The answer is not to lower the standard of evidence to be “fair” to magical thinking like homeopathy; the answer is to maintain a high standard of evidence and use it to decrease the use of non-EBM/SBM medicine. In this specific case, that would mean eliminating the use of antibiotics for URTIs and eliminating homeopathy altogether unless homeopaths can come up with evidence supporting homeopathy as rich, varied, and convincing as the evidence showing it to be impossible on the basis of basic science plausibility alone.
The problem with homeopathy is that reality has a pro-science bias. What Fisher views as a “plausibility bias” is nothing more than science passing judgment on the pseudoscience of homeopathy.