Orac is the nom de blog of a (not so) humble pseudonymous surgeon/scientist with an ego just big enough to delude himself that someone, somewhere might actually give a rodent's posterior about his miscellaneous verbal meanderings, but just barely small enough to admit to himself that few will. (Continued 
When I first started blogging, I liked to refer to myself as a booster of evidence-based medicine (EBM). These days, I'm not nearly as likely to refer to myself this way. It's not because I've become a woo-meister of course. Even a cursory reading of this blog would show that that is most definitely not the case.
So what's changed? Basically, I've come to the realization that EBM is an imperfect tool. Don't get me wrong, EBM goes a long way towards systematizing how we approach clinical data, but there's one huge flaw in it. (I can just see a quack somewhere quote-mining that sentence: "Orac says EBM has a huge flaw!") That flaw is that it devalues basic science. In any hierarchy of evidence in the commonly used EBM systems, at the very top is, as they should be, are randomized, double-blind studies. Such studies control for the most potentially confounding variables and tries to rigorously isolate the difference between experimental groups to just the study drug or treatment. Thus, level 1a evidence is evidence from multiple randomized controlled trials with homogeneity of the trials. From there, the strength of studies falls by study type all the way down to the least powerful forms of evidence, such as case series. At the very bottom is the following:
Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles"
In other words, EBM devalues basic science.
So what, you ask? Why is that a problem? After all, why should basic science matter? Here's why. In the absence of a basic scientific basis to think that a treatment is at least plausible on a physiological basis, all sorts of mischief happens. That mischief takes the form of "complementary and alternative medicine," much of which has zero plausibility from a scientific viewpoint. I'm not just talking about mild improbability, either. I'm talking about relying on first principles that flout everything we know about physics and chemistry, that for them to be true would require the overthrow of major, well-established theories. Think homeopathy, for instance. For homeopathy to be true, not only would our entire understanding of the chemistry and physics of water have to be seriously in error, but so would our understanding of how cells respond to chemical compounds. Think reiki. For reiki to be true, not only would there have to be a life "energy" (which, by the way, no scientist has yet been able to detect or characterize), but "healers" would have to be able to manipulate it. And not just manipulate it either, they'd have to be able to make it do their bidding to heal. Or think acupuncture. For acupuncture's mechanism to be true, much of what we know about anatomy and physiology would have to be wrong. We know, for instance, that there are no such things as meridians, the mythical pathways along the body through which that magical, mystical life energy known as qi is claimed to flow and into which needles need to be inserted in order to "unblock" or "redirect" that flow in order to heal. True, it's marginally possible that acupuncture could "work" by another mechanism consistent with modern science (endorphin or opioid release, for example), but classical acupuncture is clearly pseudoscience. On the basis of enormous amounts of data gathered over literally centuries, we can say with confidence that all of the above CAM therapies are incredibly implausible, bordering on being downright impossible.
None of this matters in the EBM paradigm as formalized by the Cochrane Collaboration and others. First principles based on basic science, no matter how well supported that basic science is, fall under level 5 (the lowest level) evidence, and for purposes of EBM as prior probability doesn't matter. Even a poorly designed, badly carried out case series ranks higher on the scale of evidence than hundreds of years of science saying that homeopathy can't work. Worse, as John Ioannidis has shown us, clinical trials are so prone to producing "wrong" results more than the 5% of the time expected by random chance alone using the conventional 95% confidence interval, with the probability of such a result increasing with the implausibility of the treatment being tested. In other words, the less likely the prior probability of a positive result based on scientific principles, the less likely the positive result obtained is a "true positive" or that it can be explained by a real therapeutic effect. When the result is equivocal or only weakly "positive" (as virtually all of the few well designed studies of CAM modalities that report "positive" results are), it's even less likely that the positive result is due to a real effect.
What this blind spot in the EBM paradigm leads to is the appearance of enough "positive" trials of even incredibly improbable CAM modalities such as homeopathy when on the basis of prior probability alone it is not unreasonable from a scientific (not to mention ethical) viewpoint to reject these improbable remedies a priori while awaiting evidence that approaches being as compelling as the scientific evidence that says they can't work. That's a lot of evidence. It's also why I've started referring more to "science-based" medicine rather than EBM, or "science- and evidence-based" medicine (SEBM). To me SEBM takes into account both clinical trials, other forms of evidence, and, most of all, science in the form of estimating plausibility and prior probability. Rarely do I see anyone writing about such topics in literature directed at a lay audience. (Actually, come to think of it, I rarely see articles directed at professionals discussing such matters.)
That's why I was pleased to see an article in the New York Times yesterday that does about the best job I've seen at actually discussing the issue of how to rank the "believability" of clinical trials. Have you ever heard of the Avalon effect? The article uses it to demonstrate how three large, well-designed randomized trials that failed to find that beta carotene protected against cancer were trumped in the public consciousness by lots of lesser quality studies. As Frankie Avalon, while shilling for supplement manufacturers put it:
There were laboratory studies showing how beta carotene would work. There were animal studies confirming that it was protective against cancer. There were observational studies showing that the more fruit and vegetables people ate, the lower their cancer risk. So convinced were some scientists that they themselves were taking beta carotene supplements.Then came three large, rigorous clinical trials that randomly assigned people to take beta carotene pills or a placebo. And the beta carotene hypothesis crumbled. The trials concluded that not only did beta carotene fail to protect against cancer and heart disease, but it might increase the risk of developing cancer.
It was "the biggest disappointment of my career," said one of the study researchers, Dr. Charles Hennekens, then at Brigham and Women's Hospital.
But Frankie Avalon, a '50s singer and actor turned supplement marketer, had another view. When the bad news was released, he appeared in an infomercial. On one side of him was a huge stack of papers. At his other side were a few lonely pages. What are you going to believe, he asked, all these studies saying beta carotene works or these saying it doesn't?
That, of course, is the question about medical evidence. What are you going to believe, and why? Why should a few clinical trials trump dozens of studies involving laboratory tests, animal studies and observations of human populations
Why indeed. Too bad the average lay person doesn't understand that quality matters far more than quantity when it comes to deciding which clinical trials to believe. It is true that two or three well-designed studies do trump hundreds of weaker studies.
What follows is a fairly standard and well-described discourse on the rationale for what makes a medical study convincing. The importance of comparing populations as similar as possible except for the study intervention, randomization, controlling for confounding variables, and as large a sample size as possible. What shocked me is that Bayes' theory was next described:
The third principle, Dr. Goodman says, "is often off the radar of even many scientists." But it can be a deciding factor in whether a result can be believed. It's a principle that comes from statistics, called Bayes' theorem. As Dr. Goodman explains it,"What is the strength of all the supporting evidence separate from the study at hand?"
A clinical trial that randomly assigns groups to an intervention, like beta carotene or a placebo, Dr. Goodman notes, "is typically at the top of a pyramid of research." Large and definitive clinical trials can be hugely expensive and take years, so they usually are undertaken only after a large body of evidence indicates that a claim is plausible enough to be worth the investment. Supporting evidence can include laboratory studies indicating a biological reason for the effect, animal studies, observational studies of human populations and even other clinical trials.
That's science-based medicine he's talking about. Normally, the way in which treatments are developed and found to be effective in humans begins with a clinical observation or a scientific finding in the laboratory. It is then studied further using in vitro models, animal models, and all sorts of other forms of evidence. All these studies, known as "pre-clinical" studies, form the supporting basis that justifies small pilot studies in humans and ultimately larger randomized clinical trials. Dr. Goodman puts it very well, when he says that the guiding principle in interpreting clinical trials is "that "things that have a good reason to be true and that have good supporting evidence are likely to be true."
The article then does something I've never seen in such an article before in a major newspaper. It gives an example:
To teach students the power of that reasoning, Dr. Goodman shows them a paper on outcomes of patients in an intensive care unit, with every mention of the intervention blacked out. The study showed that the intervention helped, but that the result was barely statistically significant, just beyond the threshold of chance.He asks the students to raise their hands if they believe the result. Most indicate that they do. Then Dr. Goodman reveals that the intervention was prayer for the patient by others. Most of the hands go down.
The reason for the skepticism, Dr. Goodman says, is not that the students are enemies of religion. It is that there is no plausible scientific explanation of why prayer should have that effect. When no such explanation or evidence exists, the bar is higher. It takes more clinical trial evidence to make a result credible.
And that, my friends, is what science-based medicine is: EBM with science taken into account. It takes into account all the other preclinical evidence and evidence from other sources, such as basic science, that bear on the believability and plausibility of a therapy under study. That's all. It really is that simple.
Nor is it being close-minded, either, and rejecting out of hand the possibility that a therapy might work. It is simply weighing all the evidence, rather than pretending that any therapy under study is as likely to work as any other. It is using what we already know to decide where to set the bar for evidence. For a therapy that is highly plausible, the bar is relatively low: A couple of convincing randomized trials might be enough. For something as improbable as homeopathy, whose principles go against so much of what is known about chemistry and physics, the bar would be much, much higher. There would have to be multiple well-designed randomized clinical trials with very clear, compelling, and undeniable results to make it reasonable to start to conclude that there may be something wrong with our understanding of chemistry and physics rather than something wrong with the clinical trials. Of course, in the case of homeopathy there are no such trials. The "positive" ones are almost invariably small and/or poorly designed, and even the occasional randomized trial that appears "positive" generally demonstrates an "effect" that is barely above statistical significance. Meanwhile, the better designed and more rigorous a clinical trial of homeopathy is, the less likely it is to show a "positive" result.
Of course, that's the elephant in the room in discussions like these: How CAM research utterly ignores the issue of prior probability. True, Gina Kolata, the writer of the Times article, discusses prior probability and even interviews Dr. Goodman. In that, she goes further than virtually any other science or medical writer I've seen before in understanding how EBM should be applied. However, there's still that damned elephant that can't be avoided in CAM studies. It's an obvious connection, but she doesn't make it, and rarely does anyone else, it seems. In fact, we tend to pretend that it isn't there. Sometimes we bump into it but pretend it's something else. But it won't go away, and it's the reason that the vast majority of CAM research done under the auspices of the National Center for Complementary and Alternative Medicine and promoted by wealthy private foundations such as the Bravewell Collaborative is so often a huge waste of resources and an abuse of science.
Too bad someone didn't tell another Times writer, William J. Broad, who clearly didn't get the idea. On the very same day Kolata's article appeared, he published an article called Applying Science to Alternative Medicine. Although there are some reasonable bits, there's a lot of the double-talk used to justify wasting resources studying highly implausible CAM treatments:
Dr. Briggs said such investments would be likely to pay off in the future by documenting real benefits from at least some of the unorthodox treatments. "I believe that as the sensitivities of our measures improve, we'll do a better job at detecting these modest but important effects" for disease prevention and healing, she said.
If the effects are so modest, why is it justified spending so much money and in the process twisting the very process of scientific medicine, I ask? On the other hand, tight funding may eventually bring some sense to the endeavor of studying CAM treatments by forcing a more rigorous form of triage:
An open question is how far the new wave will go. The high costs of good clinical trials, which can run to millions of dollars, means relatively few are done in the field of alternative therapies and relatively few of the extravagant claims are closely examined."In tight funding times, that's going to get worse," said Dr. Khalsa of Harvard, who is doing a clinical trial on whether yoga can fight insomnia. "It's a big problem. These grants are still very hard to get and the emphasis is still on conventional medicine, on the magic pill or procedure that's going to take away all these diseases."
I hate that "magic pill" straw man. If there is a "magic pill" in scientific medicine, its effects are not magic; they're documented by rigorous science and clinical trials. It's CAM that is looking for magic in the form of that "magic supplement" or extravagant magical thinking in the form of modalities like homeopathy, craniosacral therapy, much of chiropractic, "detoxification," and so many others. If there's a silver lining that might come out the current dire NIH funding situation, it's that it might force some rigor in our thinking about CAM and in how we decide what CAM modalities to study.
I guy can dream, can't he? Or, if you will, think magically.
Medicine & Health
Comments
I've got your back on the Broad article
http://scienceblogs.com/denialism/2008/09/the_times_doesnt_know_bayes.php
Posted by: PalMD | October 1, 2008 10:10 AM
Hmm. Not sure I agree with you here. The yoga study sounds potentially productive, and if it is effective it would be better for a lot of people than sleeping pills. Are you counting the yoga study in with the nonsense? Is there not a focus on pills rather than alternatives that are not potentially profit-generating for corporations in this area? I don't know much about it, but it seems like she has a point at some level.
Good post, anyway.
Posted by: SC | October 1, 2008 10:33 AM
I read your blog entry with great interest. I share your belief that EBM is imperfect.
With regard to homeopathy - you make the following comment: For homeopathy to be true, not only would our entire understanding of the chemistry and physics of water have to be seriously in error, but so would our understanding of how cells respond to chemical compounds.
As a biologist, I too was a disbeliever. However, new research on water from top researchers at Penn State, The University of Washington, Stanford, and other institutions in Europe is dramatically changing our understanding of this substance and its potential to carry information into biological systems. Keep an open mind on this one.
Posted by: Peter Gold | October 1, 2008 11:09 AM
If you mean Rustum Roy or Jacques Benveniste, please spare me. None of that supports homeopathy or the "memory of water."
Homeopaths will have to do a lot--a whole lot--better than that!
Posted by: Orac | October 1, 2008 11:12 AM
I was blown away by the actual mention and decent explanation of the Bayes Theorem in that article. The writer did an amazing job of making it accessible to my mother, who is my standard layperson test.
Sadly, today Tara Parker Pope covers the Vitamin C/cancer trial results from Sloan-Kettering. Her article is neutral, but comment number four, from "Ruth," says: "Hmm. Vitamin C is not patentable. Could this have something to do with the findings?"
Which just shows that evidence be damned, misguided "gut instinct" will win out. All my warm and fuzzy feelings from the Times article vanished.
Posted by: kt | October 1, 2008 11:37 AM
EBM should have levels lower than level 5, expert opinion based on physiology, with correspondingly lower levels of weight and credibility.
I would put purely random interventions at having zero a priori probability. Also interventions which have zero basis in physiology would go here too. This would include prayer (no basis in physiology) and homeopathy. Interventions proposed with no basis in physiology would go here too.
Interventions that contradict expert opinion based on physiology would have negative a priori probability. I would put liver flushes here.
Interventions that have been shown to be harmful are lower still. I would put things like silver supplements below zero, they are known to be harmful.
Once an intervention is shown to be harmful, there is a negative expectation that it will help.
6. No basis in physiology (e.g. prayer, homeopathy, Riki, acupuncture) equivalent to zero a priori probability. Administration is probably useless.
7. Intervention expected to be harmful based on expert opinion based on physiology (coffee enemas, liver flushes, HBOT for seizure associated disorders, chelation) equivalent to negative a priori probability. Administration is probably harmful, people pushing it should be prosecuted for fraud.
8. Interventions shown to be harmful in humans (silver supplements, radioactive water). Known to be harmful people administering it should be prosecuted for criminal assault, poisoning and/or battery.
There are a number of interventions that are helpful at one level but which become harmful at another. Erythropoietin for example. At some level increasing hemoglobin in the anemia of kidney failure prolongs life, but at higher levels it shortens life.
EBM and SBM need to explicitly take such things into account.
Posted by: daedalus2u | October 1, 2008 11:41 AM
Thank you for this analysis. I have been thinking something similar but not in such a clear-minded way: how can people make claims about alternative medicines that defy our understanding of physics, chemistry and biology. If they are genuine effects, then obviously we'd need to adjust our understandings of those fields but those claims have not been demonstrated. I particularly liked your point about 5% or more of studies showing false positives as applied to weakly positive results. Interesting stuff! What I'm not sure about is how much well-designed studies as funded through NCCAM will do. People who believe in CAM won't be convinced. A negative result won't prove anything, while a positive result (I expect there are some CAM practices that do something) may reinforce the view that scientists have been trying to quash practices that actually work.
Posted by: bumblebrain | October 1, 2008 11:43 AM
Good post. I'm curious, what level you'd place "[e]xpert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles," if you were to recreate EBM as SEBM? Would you subdivide it?
Posted by: MKandefer | October 1, 2008 11:54 AM
It is a huge mistake to think alt med doesn't generate profits. It generates a huge truckload of profits, and that is why its practitioners will fight tooth and nail against any form of rigorous testing. And the best strategy they have found to defend themselves against negative results is a monetary argument based on a conspiracy theory.
The "magic pill" strawman is quite ironic in that you will see many alt med proponents, such as chiropractors, try and sell you those magic supplements, which according to them, are not "drugs", are "natural" and "have no side-effects". While the doctor will tell you that yes, you can take sleeping pills, but be careful because it can have such and such side effect. The simple, but inconvenient, truth is that no efficient drug, whether natural or synthetic, is devoid of side effect. Any substance that you take in the hopes of correcting a health problem is, by definition, a drug. Which one of the two drugs, based upon that description, seems more like "magic" ?
The other nasty strawman implied in what she said is that all doctors will resort to a pill with known side effects in all cases before suggesting that a patient tries relaxation methods, be they yoga or any other. Somehow that doesn't jive with the doctors I know of. I know someone whose longest night sleep are 4 hours long (that's when she's not slept for days) and somehow, she's not been prescribed sleeping pills yet, by all the doctors she's consulted. Good doctors are able to evaluate the risks and benefits of treatments, and will suggest trying alternatives (exercising, stop smoking, better food habits, relaxation) before prescribing drugs, especially if those drugs can have detrimental effects on the health of their patients. And ironically again, when they are refused a speedy prescription from their GP, that's when patients will go to an alt med peddler for the magical natural solution.
The health industry, be it pharmaceutical or "alternative", is driven by its clients' demands. And it is the client which demands a fast effortless solution to his/her problems. Thus, when the doctor honestly tells them there is no magical solution to their problems, patients will turn to alt med, which has no such compunction to be honest at all.
Posted by: Kemist | October 1, 2008 12:07 PM
Yoga is probably a bad example to use when discussing the value of CAM (esp. w/ the sleep study above). True, it's Eastern in origin, and many followers of CAM practice yoga, so it gets thrown into the mix--but at the end of the day yoga is exercise, mixed w/ breath awareness, etc., and studies have already shown exercise is effective in treatment of insomnia. It would be more surprising if yoga *didn't* have a statistically significant effect.
Posted by: Erika | October 1, 2008 12:15 PM
The fundamental fallacy of trials-driven epistemology is that no trial or conceivable series of trials can prove that something doesn't have an effect. Oh, sure, they can put an upper bound on the amount of effect that it can plausibly have but they can't, ever, show that there is no effect at all.
This isn't really a problem with Real Medicine (tm) because Real Medicine is based on risk/benefit tradeoffs so the threshold isn't, "does it have an effect" but instead "does it have a benefit materially greater than the risk" (or sometimes conversely.) That's much easier to test empirically.
However, with woo the "risk" is often the same as the "benefit:" zero. A hundred and fifty years ago that was a serious advantage for homeopathy, since the conventional medicine of the time was often worse than no treatment at all. Maybe elsewise it might have achieved the lasting obscurity it so richly deserves.
That leaves us with self-moving goalposts in the realm of woo-evaluation.
Posted by: D. C. Sessions | October 1, 2008 12:33 PM
You write:"much of what we know about anatomy and physiology would have to be wrong. We know, for instance, that there are no such things as meridians,"etc. I wonder how do we know that; meridiams, emergy points, etc. are not the elements of our western conception of man, they belong to a quite different system of knowledge, which seems to be hardly compatible with ours. You cannot disprove the claims of this eastern conception of man by using the criteria of our western science. But the fact that something is inconceivable within our western conceptual scheme does not necesserily mean that it is improbable or unreal. "The doctors- let me quote dr.D.J.Benor -have not yet learnt to accept that because something is different, it does not mean that the other is wrong."(A Clinician on Healing-An interview with dr.D.J.Benor in THE WORLD OF HEALERS by L. Dobrzanski)
Posted by: ld | October 1, 2008 6:01 PM
ld said "You cannot disprove the claims of this eastern conception of man by using the criteria of our western science."
I hate the "east vs. west" canard. Science is science... and the laws of nature act the same way in Japan as they do in Germany, and also in Africa and Australia. The is no other mystical plane of existance that makes "energy points" measurable or useful that exists on this entire planet.
In the last few months I have benefitted from a couple of medical interventions that originated in Japan. Are they "western" or "eastern"?
Posted by: HCN | October 1, 2008 6:23 PM
Let us NOT confuse ancient Chinese conceptions with the REALITY of meridians and acupuncture points demonstrated by NMR studies. Go to the Past Issues archives and www.medicalacupuncture.org and see for yourselves.
Issues prior to Volume 18 are available for all to view.
Pay particular attention to Dr. Niemtzow's articles, they are quite good articles and people will CONTINUE to get acupuncture treatments while theorists pout that the merdians and points are .... "not real".
Let us NOT confuse the reality of research in Homeopathy
with the uneducated and unscientific shouts of "it's just water" or "it's nonsense". ESPECIALLY in this era, one's high school chemistry knowledge is INADEQUATE to draw any conclusions whatever.
The REALITY is that M Ennis' experiments, demonstrating clear biological activity stimulated by a highly dilute substance with NO atoms of the stimulating substance remaining, remains unrefuted and without scientific explanation. This experiment was repeated in several international labs, with positive results by several of them and is described in the journal Inflammation Research vol 53, pge 181.
Please do NOT mention the BBC Horizon documentary which some believe purported to repeat her experiment with negative results. Months after the documentary the producers finally begrudingly admitted to Ennis and others that they never intended to reproduce her experiment though they did nothing to counter the widely held belief that this was so.
So we have at least ONE major experiment demonstrating biological activity from the supposed "nothing". How interesting. There are, of course, MORE.
There is real scientific research and real scientists and then there are the quackspotters, who see a ruse, a quack and a charlatan in their closet, under their beds and in the pages of the Guardian and who would not recognize REAL scientific research if it fell on them, nor, apparently would they want to.
Perhaps we could as easily reduce chemotherapy with the expression, it's just poisons, or pharmacology by exclaiming that, "it's just chemicals".
Posted by: James Pannozzi | October 1, 2008 7:16 PM
The argument you give is exactly the argument that Cardinal Bellarmine gave to Galileo: I don't care what your telescope shows, Aristotle proved that the Earth is stationary. Bayes theorem, indeed.
Posted by: Prolix | October 1, 2008 8:17 PM
@James Pannozzi
The REALITY is that M Ennis' experiments, demonstrating clear biological activity stimulated by a highly dilute substance...
I don't think "biological activity" means what you think it does.
@Prolix
Good analogy, except we're the ones with evidence, and you're the one deciding things by philisophical fiat.
Posted by: Clay | October 1, 2008 8:56 PM
Sorry, but I have a PhD in chemistry, I am working in medical research, and I'm afraid that's still "inadequate" to "understand" such nonsense.
The basic knowledge of chemistry one gets in high school is not nullyfied by what one learns at college and graduate levels. That you even speak like that betrays the fact that you don't understand science at all.
This is pseudoscientific gobbledygook which aims only at dishonest confusion by making it seem that you are so much more smarter than everybody else. Here's the news : you're not. You are talking nonsense.
If you knew anything of chemistry dude, you'd know everything is just chemicals. Here's the news, since you don't seem to have them yet : all the phenomenons of life consist of chemical interactions. All. of. them. And all chemical reactions obey the laws of physics. That is the true power of science. It all fits together ultimately.
Posted by: Kemist | October 1, 2008 9:31 PM
I agree completely. This was not, however, a mistake I was making. I was speaking specifically about the yoga study (and similar studies) and the researcher who was quoted - not "alt med" in any vague, general way. That's why I asked for clarification. Please revisit my comment; also see Erika's.
Posted by: SC | October 1, 2008 10:29 PM
From the BMJ (British Medical Journal):
Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials
Gordon C S Smith, professor, Jill P Pell, consultant
Abstract
Objectives To determine whether parachutes are effective in preventing major trauma related to gravitational challenge.
Design Systematic review of randomised controlled trials.
Data sources: Medline, Web of Science, Embase, and the Cochrane Library databases; appropriate internet sites and citation lists.
Study selection: Studies showing the effects of using a parachute during free fall.
Main outcome measure Death or major trauma, defined as an injury severity score > 15.
Results We were unable to identify any randomised controlled trials of parachute intervention.
Conclusions As with many interventions intended to prevent ill health, the effectiveness of parachutes has not been subjected to rigorous evaluation by using randomised controlled trials. Advocates of evidence based medicine have criticised the adoption of interventions evaluated by using only observational data. We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute
http://www.bmj.com/cgi/content/abstract/327/7429/1459
[ I saw this on a blog somewhere a while back, but can't remember where; sorry for the lack of credit. ]
Posted by: Patrick Caldon | October 1, 2008 11:30 PM
I suggest you all read the following, it is the BEST characterization of this new DENIALIST FUNDAMENTALISM which has taken hold. The Abstract is posted here,
see this link:
http://www.hpathy.com/papersnew/milogram-homeopathy-fundamentalism.asp
for the full article. And KUDOS to Orac
for admitting the limitations of EBM!
Homeopathy and the New Fundamentalism:
A Critique of the Critics
- Lionel Milgrom, Ph.D., F.R.S.C., M.A.R.H.
ABSTRACT
"Though in use for over 200 years, and still benefiting millions of people worldwide today, homeopathy is currently under continuous attacks for being "unscientific." The reasons for this can be understood in terms of what might be called a "New Fundamentalism," emanating particularly but not exclusively from within biomedicine, and supported in some sections of the media. Possible reasons for this are discussed. New Fundamentalism's hallmarks include the denial of evidence for the efficacy of any therapeutic modality that cannot be consistently "proven" using double-blind, randomized controlled trials. It excludes explanations of homeopathy's efficacy; ignores, excoriates, or considers current research data supporting those explanations incomprehensible, particularly from outside biomedicine: it is also not averse to using experimental bias, hearsay, and innuendo in order to discredit homeopathy. Thus, New Fundamentalism is itself unscientific. This may have consequences in the future for how practitioners, researchers, and patients of homeopathy/complementary and alternative medicine engage and negotiate with primary health care systems. "
from:
http://www.hpathy.com/papersnew/milogram-homeopathy-fundamentalism.asp
Again, see the above link for the full text of Milgrom's article.
Posted by: James Pannozzi | October 1, 2008 11:50 PM
James, let me recapitulate Orac's post using fewer words, since you seem to have lost the jist:
Homeopathy is a crock of shit. You don't need randomized double-blinded placebo controlled crossover trial with a battery of multivariate statistical tests to tell you that shit stinks.
Posted by: Patrick Caldon | October 2, 2008 1:34 AM
Patrick, let me recapitulate my post using fewer words, since YOU seem to have lost the jist:
Anti Homeopathy statements are a crock of shit.
You don't need anti-scientific innuendo about "woos" and "placebo" to tell you that the denialist fundamentalism at the heart of such un- scientific attacks to tell you that their shit stinks worse.
AGAIN, READ the comments of a scientist on this for the sordid details:
http://www.hpathy.com/papersnew/milogram-homeopathy-fundamentalism.asp
if you dare.
Posted by: James Pannozzi | October 2, 2008 7:29 AM
Let me recapitulate my perspective in brief:
Homeopathy is a crock of shit on a scientific basis. You don't need pseudoscientific hand-waving about "open-mindedness" and "memory of water" to realize that magical thinking are at the heart of such unscientific apologia that tell you that, whatever the deficiencies in "conventional" medicine, homeopathy's shit stinks far worse.
Posted by: Orac | October 2, 2008 8:01 AM
Please don't cite that guy, would you ? He's made a lot of theoretical chemists howl with pain at his abuse of quantum chemistry. He sounds like he's swallowed a quantum physics textbook backwards and proceeded to vomit it in the form of a crappy paper. Since quantum physics is not habitually part of a biochemist's or biologist's (or MD) curriculum, it is not easy for them to spot the glaring mistakes this crook makes.
For non-chemists and non-physicists : 99.999999% of the time when someone is applying quantum physics above nanoscale and below interstellar scale, you can safely assume he's talking nonsense.
Now do you want to discuss Rustum Roy's paper in which he demonstrates his inability to perform a calorimetric experiment properly ?
Posted by: Kemist | October 2, 2008 9:28 AM
The Cochrane Reports have described as "promising" the effects of Oscillococcinum in the treatment (not prevention) of influenza and influenza-like syndrome. These results were from FOUR large (over 300 subjects each) double-blind trials.
Jennifer Jacobs (University of Washington) has conducted 3 clinical trials, one of which was published in PEDIATRICS and a meta-analysis was published in the PEDIATRICS INFECTIOUS DISEASE JOURNAL.
David Reilly (University of Glasgow) has conducted four high-quality randomized and double-blind trials on allergic disorders and found significant results in each study. An editorials in the Lancet acknowledged these studies as high quality and as unlikely to have been the result of false positive results.
Iris Bell and colleagues conducted a high quality trial on patients with fibromyalgia: Improved clinical status in fibromyalgia patients treated with individualized homeopathic remedies versus placebo, Rheumatology. 2004:1111-5. Participants on active treatment showed significantly greater improvements in tender point count and tender point pain, quality of life, global health and a trend toward less depression compared with those on placebo. People on homeopathic treatment also experienced changes in EEG readings. "Helpfulness from treatment" in homeopathic patients was very significant (P=.004). The clinical results AND the objective findings from the EEG readings combine to show that SOMETHING was going on (beyond and different from a placebo effect).
The experimentation by V. Elia, an Italian chemist, on the thermodynamics of homeopathic medicines have been published in the Annals of the New York Academy of Sciences and in respect chemistry journals.
There IS a body of evidence that differentiates results of homeopathic medicine as different from placebo, and it is inaccurate to suggest that there is "no" evidence.
Finally, Shang's Lancet review (2005) noted that he found 22 high quality homeopathic trials (out of the 110 studies) but only 9 high quality conventional medical trials (out of 110 studies evaluated). The results from these randomized and double-blind trials show that there IS a difference in homeopathic treatment. However, only when Shang chose to select the largest trials, 6 of 8 of which had no external validity to the homeopathic system, did he find a "negative" result.
Posted by: Dana Ullman, MPH | October 2, 2008 10:03 AM
Re Dana Ullman's post:
So the evidence is there for all to see.
Pretending that positive Homeopathy evidence is not there or that articles appearing in respected scientific journals written by fully credentialed scientists were composed by quacks and charlatans simply won't work any more.
Sorry "Orac". Time for a little innuendo distraction?
Posted by: James Pannozzi | October 2, 2008 11:11 AM
Dana,
There have been literally hundreds of studies of homeopathy. It's absolutely unsurprising that we get a few studies with a positive result. Indeed if there were not a few studies with a positive result (having done 100s) that would be evidence that homeopathy was harmful. When we look at bigger more rigorous studies, the effect goes away.
The publishing pattern is absolutely consistent with the notion that consuming a few drops of water albeit with an elaborate preparation does sod all either way as a medical intervention.
Posted by: Patrick Caldon | October 2, 2008 11:37 AM
I should add if it's unclear, this is a simple property of using 95% confidence intervals. Periodically you will get "publishable" results coming up by chance alone.
Posted by: Patrick Caldon | October 2, 2008 11:43 AM
Hmmm...people above are referring to a post that I sent you earlier in the day, but it seems that you've chosen to delete this post. Why?
My comment provided specific references to research that you considered too supportive of homeopathy.
Are you supportive of evidence based medicine or not?
The bottomline is that there are more than a "few" clinical trials showing positive results from homeopathic treatment...including the "high quality" studies.
Posted by: Dana Ullman, MPH | October 2, 2008 12:44 PM
Patrick, I think you make a good point with your comment on parachutes. I think that issue could be addressed if EBM added rules based on theoretical harm and demonstrated harm as I outlined. Potential for harm based on good theoretical reasoning trumps complete ignorance, but doesn't trump a well run clinical trial that shows positive results.
The essence of the decision to treat and to choose a specific treatment is the risk/benefit ratio of that treatment for that condition. There is no intervention that has only benefits, parachute use not withstanding. People can get tangled in parachutes and drown, they can get sucked into jet aircraft air intakes, they can present a bigger target for dragons. Since dragons are either non-existent or exceedingly rare, that "risk" is very small, but is in principle calculable. As someone pointed out, clinical trials can only put bounds on the likelihood of an outcome and those bounds depend on the sample size. Extremely small effects take extremely large (and hence expensive) trials to measure.
EBM should have explicit rules for calculating harm as well as explicit rules for calculating benefits. Then the two can be presented to the patient and the patient can evaluate them based on the patient's risk/benefit/cost analysis.
There was a similar series of letters regarding the evidence for use of specific techniques in cleaning skin and the final sterilization with iodine. Does one scrub back and forth, or does one do a final iodine application in a spiral, starting in the center and moving outward? A clinical trial to distinguish which is better would likely have to be gigantic. The very occasional failures are most likely not related to technique, but to other circumstances such as a speck of dust falling in the wrong place, or someone already having an infection beneath the skin where the iodine didn't penetrate, or the clinician not washing hands or changing gloves first.
Posted by: daedalus2u | October 2, 2008 1:13 PM
The parachute argument doesn't hold much water for me; it's argument by analogy, and bad analogy at that.
It's possible to have a clinical trial of a parachute without making people jump out of airplanes. You don't have to intentionally expose a human being to anthrax after vaccination to test the efficacy of the vaccination.
How do you think they test new parachutes/ejection seat designs? How do you think they test new seat belt or airbag designs? Ever heard of a crash test dummy? Ever heard of animal trials? Ever heard of cadaver tests? I believe the first test of a modern parachute was done on a dog, not a human.
Posted by: Karl Withakay | October 2, 2008 2:35 PM
Arguments from physiology are considered acceptable for EBM, they are just the lowest level of evidence. There is a quite good theoretical analysis saying that parachutes are better than nothing, and no clinical trials saying that no parachute is better than having a parachute.
Posted by: daedalus2u | October 2, 2008 3:51 PM
hum, no ?
Let's see what we have:
Benveniste: A fraudster uncovered when his experiment was repeated under James Randi's supervision
Lionel Milgrom: A guy who publishes pseudo quantum physics in a crappy journal where the mistakes in his reasoning cannot be spotted in the absence of qualified peer review
Rustum Roy : A guy who's not only unable to perform calorimetric experiments properly, but isn't even able to report results correctly
Dana Ullman : a homeopathy crank banned from wikipedia who publishes books
A few "memory of water" cranks à la Dr Emoto, and a couple good papers which have nothing to do with homeopathy, because homeopathy does not deal with "dilute" solutions but with solutions which typically do not even contain one molecule of the active principle per dose.
Qualified scientist defending homeopathy ? Where ? Did I miss something ?
Posted by: Kemist | October 2, 2008 4:50 PM
Pardon me Orac, BUT
WHY DID YOU REMOVE DANA ULLMAN'S POST?
YOU CLAIM HOMEOPATHY IS NONSENSE BUT
WHEN SOMEONE POSTS SOME EVIDENTIARY
OR EXPERIMENTAL BASIS, YOU FIND IT
NECESSARY TO DELETE IT????
Posted by: James Pannozzi | October 2, 2008 7:20 PM
I delete nothing. I've only banned one person in the entire three and a half year history of this blog. I have no idea why it did that, but it was in the spam folder and I restored it. Stop typing in all caps. it makes you look like a crank.
Posted by: Orac | October 2, 2008 9:13 PM
Some people here like to call homeopaths and people who are pro-homeopathy "cranks." Because a "crank" is that mechanical device that was previously used to get a car moving, I have no problem being considered this catalyst.
By the way, Orac, I think that James Pannozzi was rightfully indignant because he and others saw my post at your site here, but then, all of a sudden, it disappear. He was trying to be a catalyst to encourage dialogue between points of view in the interest of information and scientific inquiry. Bless that type of crank.
Posted by: Dana Ullman, MPH | October 2, 2008 11:22 PM
Typing in all caps is the Internet equivalent of yelling and it is generally agreed that people who type in all caps come off looking like a crank.
Posted by: Orac | October 2, 2008 11:27 PM
Dana,
If I did 100 identical studies of homeopathy, and were looking for those with p-value
If we add to that a publication bias where people tend to publish studies which work more than studies which don't, what we get is absolutely consistent with the clinical trials.
What I've just described informally is what a meta-analysis does formally. Big meta-analyses say that it's bunkum.
But to repeat, we don't need high-falutin' statistics to get to this point, with thousands of people diligently quaffing down their micro-doses of nothing and recording it all at great expense. A great quantity of effort and energy which could be used on, for instance, healing the sick. All we need is the kind of chemistry we teach to 15 year olds.
Alternatively we could think to ourselves that near where I live, the big dam has a watercourse with a few cattle farms nearby, and the catchment is full of wild animals with all sorts of intestinal parasites. Most cities are the same. This water fills up the dams and gets massively diluted. We should all be completely homeopathically proofed against giardia and cryptosporidium, just from tap water.
Posted by: Patrick Caldon | October 3, 2008 12:29 AM
Kemist, you are a valuble chap to have around in these discussions and perhaps with your help we can focus our perspective in this debate.
What have you missed?
Well, you might have missed pharmaceutical researcher M. Ennis, a skeptic of Homeopathy who set one day to repeat some experiments of a French researcher regarding the stimulation of basophil cells to produce histamine by a highly dilute substance which no longer contained any atoms of the stimulant. She was of course expecting negative results but instead got positive results. The experiment was repeated in several international labs, many of them getting positive results, some negative.
So.... what is your hypothesis regarding the mysterious production of the histamine demonstrated in this experiment. Cosmic rays? Enhanced nothingness of the high diliution substance? Ernst anomoly neutrino reaction?
Or do you admit that the experiment did show histamine release triggered by the "nothing" of the high dilution substance.
Ennis herself published the results in "Inflammation Research" and stated that the results were unexplainable by known science and that others should repeat the experiment to discover what's what.
Was she on to something or was it much ado about nothing?
Posted by: James Pannozzi | October 3, 2008 2:04 AM
Patrick, your logic is whack. By your logic, the atomic bomb was a placebo because there are atoms around us right now that are not "exploding." Duh. Please consider doing some homework on the mystery of water. Please also consider:
Vigorous shaking of the water in glass bottles can cause small amounts of silica (silicate) fragments or chips to fall into the water (this has been verified in physics journals). Homeopathic drug manufacturers use a double-distilled water in making their medicine, and whatever medicinal substance is placed in the water interact with the silicate fragments and can change the structure of the water in different ways.
It has been shown that micro-bubbles and nano-bubbles, caused by vigorous shaking, can "burst" to produce microenvironments of higher temperature and pressure. One group of material scientists have estimated that the vigorous shaking involved with making homeopathic medicines changes the pressure in the water, akin to water being at 10,000 feet in altitude. These scientists suggest that the homeopathic process of using double-distilled water and then diluting and shaking the medicine in a sequential fashion changes the structure of water.
Posted by: Dana Ullman | October 3, 2008 11:55 AM
Mr. Panozzi, I'd like to inform you of the way science is discussed among scientists. First, do not rant and insist that you're smarter than everybody. Second, post a reference for a paper, that we might actually have a valuable discussion about something that we can all examine for ourselves.
Several crappy science papers, for many reasons, go through peer review unchallenged. One recent example of this is a paper pushing hastily and obscurely deduced creationist ideas that made it into Proteomics, a well repected science journal. Another less recent example involves a certain individual called Sokal whom without even a degree in physics managed to publish crap in a respected physics journal.
Another more to the point example concerns the unamed french scientist that you happen to mention, which actually commited something that amounts to scientific fraud and published this, of all places, in Nature.
It is possible to clear some weeds by dismissing such crap heaps as Homeopathy, Journal of Medical Hypotheses or JPANDS, but, as you can see, that's not the only criteria for good science.
Anybody who's familiar with how biological activity is assayed knows that unexplained results are a frequent occurance. However, in the vast majority of cases they have less to do with weird new things that will make you win the Nobel prize than with uncontrolled variables and reagent contamination (of which any scientist, even the most rigorous and honest, can become a victim).
So until more is available, consider me very underwhelmed by somebody who repeated the fraudster Benveniste's experiment and found positive results, even if she calls herself a "skeptic about homeopathy". So, yes, unfortunately (because, like most scientists, I happen to like Nobel-worthy weird new ideas), much ado about nothing.
Posted by: Kemist | October 3, 2008 12:11 PM
To Kemist:
Inflammation Research, vol 53,
Histamine dilutions modulate basophil activation
pp. 181-188(8)
Authors: Belon, P.; Cumps, J.; Ennis, M.; Mannaioni, P.; Roberfroid, M.; Sainte-Laudy, J.; Wiegant, F.
The unnamed French researcher to which I was referring, contrary to your mistaken supposition and rant about "fraudsters", was J. Saint-Laudy, the scientist whose experiments were repeated by Ennis.
Consider me underwhelmed by your response until you provide some explanation, theory, hypothesis, or hopefully, an admission that you don't know why the basophil cells' histamine release was stimulated by a high dilution substance having no atoms of the stimulant physically present, and admit that it is worthy of continued research.
Do not mistake my disgust at the easy talk of "fraudsters", "quacks" and "charlatans" applied to genunine scientists, for arrogance.
Posted by: James Pannozzi | October 3, 2008 1:22 PM
M'sieu P Belon, of course, works for French Homeopathic remedy manufacturer Boiron. Coincidentally (or not) Belon was also an author of the infamous / incompetent Jacques Benveniste Nature paper that someone already mentioned. Benveniste had people in the lab funded by Boiron. The person in Benveniste's lab who got positive results with homeopathic dilutions was - would you believe it - an enthusiastic supporter of homeopathy. And so on, and so on.
Or, if you prefer it in French:
Take the blinkers off, homeopaths. Oh, sorry - I forgot I was asking the impossible.
It is not even necessary for these people to "fix" the experiments deliberately. It is merely necessary for them to convince themselves (quite sincerely) not to exercise the customarily necessary critique of experimental design, or of their own fallibility. Or convince themselves they can edit out the non-positive results for homeopathic dilutions because "something is wrong there", or not do proper controls "because we don't need them" (both famously occurring in the Benveniste case).
As if it needed saying yet again, this is classic Cargo Cult Science - which can, of course, be found every day across the pages of numerous Journals of Alternative Medicine.
Posted by: Dr Aust | October 3, 2008 3:51 PM
You started by accusing everyone here of being fundamentalists and denialists. That is not indignation, that is just plain arrogance, especially when you're the one who's bringing an opposing opinion to the table. And then, you brought up Milgrom. As a qualified scientist. It doesn't start a conversation on good terms, especially for one in my field.
But now, we have a proper paper to look at.
Two minor quibbles in this paper :
A potential conflict of interest which sort of negates what you said previously of Ennis. But, frankly that doesn't really matter, since we shouldn't juge science results according to opinion but rather quality (but hey, we humans all have prejudices of one sort or another).
Another thing which appears in this paper (one of the most annoying thing ever in discussions with homeopathy proponents), is the comparison of liquid solution phases with solid phases. It's... wrong. Both are very different and you just can't extrapolate significant results from solid phase to solution phase. But, ok, I'll let that pass, since it's just a weird sort of unrelated discussion of the results.
Now the more serious things:
The way the results are reported is a little strange from my POV. First we're told that the 4 labs performed the same experiments, with great emphasis upon the use of the same reagents, labware and training of technical staff... And then the experiments themselves become different. One lab reports only extra high dilutions, the other a wide range... There is a humongous variation on the error bars... (some consistently have very small, others huge, for what essentially seems to be the same experiment). It makes it difficult to get a feel of the noise on these experiments. I understood that there can be variation in the sensitivity of basophils from different donors but... Couldn't the samples be pooled together before the assay, so all labs would share the same source (It's been done with human lymphocytes in my lab, that's why I ask) ? Maybe it would make those error bars behave.
There is a sort of game going on also with statistical tests. That is also... strange. Why are the labs using different tests to find significant differences ? When they are supposed to be performing the same experiment (Maybe a statistician can answer me for that one)?
And in most instances lab #2's results are ditched, like they're not important or something. Indeed, lab #2's results are not discussed at all.
To be completely honest, it's a weird paper. Am I curious to see other people try things like this ? Mildly. Would I approve investing money in it ? heeshh... not really, just based on this particular paper.
Posted by: Kemist | October 3, 2008 7:36 PM
Many thanks to Kemist for giving us some considerations regarding the Ennis paper and for giving us an overview of some questionable issues with it in the light of his experise.
And Kemist also has had the honesty to state:
"But, frankly that doesn't really matter, since we shouldn't judge science results according to opinion but rather quality (but hey, we humans all have prejudices of one sort or another)."
I agree.
And thank the maker, an entire post with no tirades against Benveniste.
Related papers you might find of interest follow:
Lorenz I, Schneider EM, Stolz P, Brack A, Strube J.
Influence of the diluent on the effect of highly diluted histamine on basophil activation.
Homeopathy. 2003 Jan;92(1):11-8.
PMID: 12587990 [PubMed - indexed for MEDLINE]
Sainte-Laudy J, Belon P.
Use of four different flow cytometric protocols for the analysis of human basophil activation. Application to the study of the biological activity of high dilutions of histamine.
Inflamm Res. 2006 Apr;55 Suppl 1:S23-4. No abstract available.
PMID: 16705375 [PubMed - indexed for MEDLINE]
Sainte-Laudy J, Boujenaini N, Belon P.
Confirmation of biological effects of high dilutions. Effects of submolecular concentrations of histamine and 1-, 3- and 4-methylhistamines on human basophil activation.
Inflamm Res. 2008;57 Suppl 1:S27-8. No abstract available.
PMID: 18345504 [PubMed - indexed for MEDLINE]
Sainte-Laudy J, Belon P.
Improvement of flow cytometric analysis of basophil activation inhibition by high histamine dilutions. A novel basophil specific marker: CD 203c.
Homeopathy. 2006 Jan;95(1):3-8.
PMID: 16399248 [PubMed - indexed for MEDLINE]
Guggisberg AG, Baumgartner SM, Tschopp CM, Heusser P.
Replication study concerning the effects of homeopathic dilutions of histamine on human basophil degranulation in vitro.
Complement Ther Med. 2005 Jun;13(2):91-100.
PMID: 16036166 [PubMed - indexed for MEDLINE]
Lorenz I, Schneider EM, Stolz P, Brack A, Strube J.
Sensitive flow cytometric method to test basophil activation influenced by homeopathic histamine dilutions.
Forsch Komplementarmed Klass Naturheilkd. 2003 Dec;10(6):316-24.
PMID: 14707480 [PubMed - indexed for MEDLINE]
Thanks
JP
Posted by: James Pannozzi | October 3, 2008 8:28 PM
Dana,
Understandably you completely missed the point of my posting, because the blog software apparently has a problem with displaying a "less than" sign, presumably because it mistakes it for HTML.
The first paragraph should have read something along the lines of:
If I did 100 identical studies of homeopathy, and were looking for those with p-value < 0.05 I would expect to find something in the order of 5 studies showing that homeopathy had an effect. Indeed if I found zero studies showing that homeopathy had an effect given the hundreds that have been done, that would be evidence that something odd was going on.
You have to do a lot more than show half a dozen studies showing that homeopathy works. You have to account for the overwhelming majority of studies that show that it's ineffective.
But to re-iterate, why bother? It's close to self-evidently bunkum from the basic chemistry. You first need to fundamentally transform basic chemistry before anybody with half a clue will even begin to listen to you; ill defined hogwash about "silica" doesn't cut it I'm afraid. Let's quit analyzing something so obviously stupid and spend our tax dollars/research dollars/grant money on something which at least has the potential to heal sick people.
Posted by: Patrick Caldon | October 4, 2008 2:38 AM
Yup, he's a pretty good example of it. Have you seen his claim that double blind RCTs of homoeopathy fail because of "a kind of quantum superposition between the remedy and placebo"?
Posted by: Mojo | October 4, 2008 11:15 AM
You forgot to mention the attempt by a team at Southampton to replicate Reilly's findings.
Posted by: Mojo | October 4, 2008 11:28 AM