What brought these thoughts to mind is a study that I recall seeing a few days ago in the Annals of Internal Medicine on placebos in randomized clinical trials coming out of the University of California San Diego and the University of Oxford. The study is entitled, provocatively enough, What’s in Placebos: Who Knows? Analysis of Randomized, Controlled Trials. Basically what Golomb et et al did in this study was something incredibly simple. They simply asked: How often did published randomized clinical trials (RCTs) provide sufficient detail about the composition of the placebo used that another investigator could replicated them? The answer was, in essence: Disturbingly less often than I would have expected. I filed the article away as something I should consider blogging about. Then I filed it away and, thanks to the other things going on this week, forgot about it.
Until, that is, I saw Mike Adams at that even more wretched hive of scum and quackery than The Huffington Post, NaturalNews.com, decide to argue on the basis of this study that the entire scientific basis of science-based medicine is now in doubt. I kid you not. That’s what he is arguing in an article he entitled Placebo fraud rocks the very foundation of modern medical science; thousands of clinical trials invalidated. It’s a screed of flaming stupid so over the top that only Mike Adams could have produced it, firing napalm into the sky to form the words, “I’m an idiot” in letters big enough for the state of Texas to see.
Before we get to Mike Adams’ rather fevered interpretation of Golomb et al, let’s settle down a moment and look at the study itself. To get at the answer of how many , Golomb et al screened four clinical journals with high impact factors between the period from January 2008 to December 2009, looking for placebo-controlled RCTs. Articles were eligible if they were RCTs and used a non-active control (i.e., placebo). They excluded articles where the primary findings were cited as having been published in other articles, in other words, papers that recommend additional findings of a clinical trial that had previously been reported. In addition, they compared how often placebo ingredients were disclosed for pills versus injections. Their findings are summarized well in the abstract:
Background: No regulations govern placebo composition. The composition of placebos can influence trial outcomes and merits reporting.
Purpose: To assess how often investigators specify the composition of placebos in randomized, placebo-controlled trials.
Data Sources: 4 English-language general and internal medicine journals with high impact factors.
Study Selection: 3 reviewers screened titles and abstracts of the journals to identify randomized, placebo-controlled trials published from January 2008 to December 2009.
Data Extraction: Reviewers independently abstracted data from the introduction and methods sections of identified articles, recording treatment type (pill, injection, or other) and whether placebo composition was stated. Discrepancies were resolved by consensus.
Data Synthesis: Most studies did not disclose the composition of the study placebo. Disclosure was less common for pills than for injections and other treatments (8.2% vs. 26.7%; P = 0.002).
Limitation: Journals with high impact factors may not be representative.
Conclusion: Placebos were seldom described in randomized, controlled trials of pills or capsules. Because the nature of the placebo can influence trial outcomes, placebo formulation should be disclosed in reports of placebo-controlled trials.
Primary Funding Source: University of California Foundation Fund 3929–Medical Reasoning
Personally, I don’t find it surprising at all that the ingredients of more injections than pills were disclosed. After all, injections are more invasive than taking a pill. Even so, placebo content should be disclosed.
In the discussion, Golomb et al engage in a discussion of the issues behind placebos. What has to be considered is that there is no such thing as a perfect placebo. Consider this. If a drug has a characteristic taste or smell and the placebo used in a study doesn’t also have that characteristic taste or smell, it’s quite possible that patients in the control group might figure out that they’re receiving a placebo. Consequently, it’s often necessary to add something to the placebo to make it taste and/or smell like the real drug. The authors use a specific example of a drug that leaves a fishy aftertaste, meaning that, to make a convincing placebo, something that adds a fishy aftertaste would have to be added. It’s also important that the placebo have the same color and texture, or at least be as close as possible in color and texture, to the drug.
More problematic for drug trials is that some ingredients used in placebos can actually have physiological effects. The results can be as follows:
However, negative, positive, or same-direction effects of a placebo can result in the misleading appearance of positive, negative, or null effects of the experimental drug (7). For instance, olive oil and corn oil have been used as the placebo in trials of cholesterol-lowering drugs (7, 10, 11). This may lead to an understatement of drug benefit: The monounsaturated and polyunsaturated fatty acids of these “placebos,” and their antioxidant and antiinflammatory effects (12, 13), can reduce lipid levels and heart disease (13, 14). In one of these studies (11), the authors commented that “The lack of any overall effect in patients with myocardial infarction might be related to the unexpectedly low mortality rate in the placebo group.” The possibility that the placebo composition may have influenced this “unexpectedly low mortality” was apparently not considered.
Another example cited by the authors is that of megestrol acetate for anorexia associated with cancer, a trial in which an unexpected benefit for megestrol in gastrointestinal symptoms was found. It turns out that the placebo control contained lactose. It further turns out that lactose intolerance is prevalent in cancer patients, excacerbated by chemotherapy and radiation, leading to speculation that the lactose in the placebo might have produced the appearance of benefit for adverse GI symptoms. Was the amount of lactose in the placebos enough to have actually provoked or worsened GI symptoms in the lactose intolerant? Who knows? It’s a possibility, but only that–a possibility. Certainly Golomb cites nothing that demonstrates it to be more than just a possibility. In fact, I looked up the references. The references cited don’t demonstrate that this is what in fact happened, which actually irritated me a bit. All they showed was that, yes, cancer patients probably a higher than normal incidence of lactose intolerance that might be related to chemotherapy and radiation. The other references cited included the megestrol study, which didn’t even speculate about the lactose in the placebo and a bioethics paper in which Golomb discussed the possibility of active ingredients in placebos. In any case, nothing she presented shows that the small amount of lactose that was probably in the placebo used in the megestrol study was even likely to have contributed to GI complaints that could have made megestrol look better by comparison. Moreover, this study was not without weaknesses. For instance, its focus is on only four journals. True, they may be high impact journals, but that’s a limited sample nonetheless and “high impact” does not necessarily equal “more rigor.” Basically, what we have here is an interesting preliminary study that might have found a potential problem with placebo-controlled clinical trials.
Worse, clearly Dr. Golomb has at the very least a bit of naÃ¯vetÃ© about how cranks use her work to try to justify her pseudoscience. For example, in June Dr. Golomb appeared on Joe Mercola’s YouTube channel in an interview:
Still, even given Dr. Golomb’s apparent affinity for (or at least inability to recognize) cranks coupled with the preliminary nature of her study, I still have a hard time not agreeing with her when she advocates that high impact journals change their CONSORT (Consolidated Standards of Reporting Trials) guidelines to include new rules on the reporting of placebos. At the same time, I wish Dr. Golomb would be a bit more careful about whom grants interviews to. Let’s just put it this way: Giving an interview to Joe Mercola is not a good way to burnish your credibility. Specifically, they propose adding a section requiring the specification of placebo ingredients that answers these questions:
- Was the (placebo) control treatment described in detail?
- If a chemical compound, were its full constituents given (by weight)?
- Were its appearance and any differences from the test drug described (or absence of differences stipulated)?
- Was it stated what other factors might render the experience of the control distinctive from the test agent (or absence of other factors stipulated)?
All of these are reasonable questions. Overall, I was left with the impression that there might be a problem, but I wasn’t convinced that it was as huge a problem as Golomb et al were trying to argue. Even so, I thought that their proposal to modify the CONSORT guidelines was a good first step in correcting the situation. After all, as Golomb et al argue, it can only increase the usefulness and rigor of reporting clinical trials if placebo ingredients are reported.
Obviously, Ã¼ber-quack Mike Adams has different ideas. In fact, I’m going to jump to the hugest howler in a howling sea of stupidity right now. As he is wont to do, Adams leaps on one thing Golomb et al mentioned, namely that there is no government regulation of the contents of placebos. Because of that (get a load of this), Adams concludes:
You see, if there are no regulations or rules regarding placebo, then none of the placebo-controlled clinical trials are scientifically valid.
It’s amazing how medical scientists will get rough and tough when attacking homeopathy, touting how their own medicine is “based on the gold standard of scientific evidence!” and yet when it really comes down to it, their scientific evidence is just a jug of quackery mixed with a pinch of wishful thinking and a wisp of pseudoscientific gobbledygook, all framed in the language of scientism by members of the FDA who wouldn’t recognize real science if they tripped and fell into a vat full of it.
Big Pharma and the FDA have based their entire system of scientific evidence on a placebo fraud! And if the placebo isn’t a placebo, then the scientific evidence isn’t scientific.
Step back a minute. Step back and cover your head to try to protect yourselves against the waves of burning stupid washing over you. I’m sorry I didn’t warn you. In retrospect, I really should have warned you, so that you could have gone to your kitchens to get some tin foil to construct a hat to keep the conspiracy waves mixed with the waves of burning stupid from frying your neurons. Adams’ post is truly a black hole of neuron-apoptosing stupid. I can’t make up my mind if he really believes it when he argues that a problem with the reporting of placebo ingredients means that all placebo-controlled RCTs are invalidated, that they’re all pseudoscience and quackery, or whether Adams contempt for his audience and erstwhile customers is so great that he thinks they’ll believe anything, even whoppers like the one he just told. I’m also quite sure that he has no idea that the contents of placebos for these trials are known and could be discovered. Records are kept; the IRB application for the clinical trial has to reveal the placebo as well. If anyone has any reason to question any individual clinical trial, it is quite possible to go back to the trial records and find out what the placebo used was. True, this is not ideal, being far more difficult than simply being able to read in the journal article describing the results of the trial, but the fact that too many clinical trials don’t adequately report the placebo used does not automatically invalidate every clinical trial that uses a placebo.
Think about what Adams is trying to get his marks to believe. He’s trying to argue that, just because a low percentage of studies report the ingredients of their placebos, that all science-based medicine is not just wrong, but a fraud. It’s hard for me not to chuckle, of course, at Adams’ thinking that the lack of FDA regulation of placebos is such a horrible thing, given that he has frequently likened the FDA to Nazis and worse on his website. After all, any time the FDA tries to rein in quackery, Adams is so fast off the mark in attacking the FDA as a bunch of jack-booted thugs that he shatters windows with his sonic booms. Yet, here he is, railing that the FDA doesn’t regulate the contents of placebos.
More hilarious is how Adams represents this issue as being a massive conspiracy involving–who else?–big pharma to manipulate placeboes in placebo-controlled RCTs in order to get the results that they want. Amusingly, in doing so, he can’t even get it right about what statistical significance means:
As the key piece of information on its regulatory approval decisions, the FDA wants to know whether a drug works better than placebo. That’s the primary requirement! If they work even 5% better than placebo, they are said to be “efficacious” (meaning they “work”).
Is it just me, or is Adams confusing the convention that statistical significance requires a p-value of less than 0.05 (i.e., 5%), meaning that there is only a 5% chance that the differences observed are due to random chance alone? (I realize I’m simplifying a lot, but this is Mike Adams we’re dealing with here.) Instead, Adams misstates the concept as meaning that the FDA just requires a 5% difference between drug and placebo. Someone who can’t even state with reasonable accuracy the concept of what a statistically significant difference means shouldn’t be lecturing anyone about how to do science. Yet lecture scientists (more like harangue them) is exactly what Adams does, with this broadside against those evil skeptics:
It really makes you wonder about so-called “skeptics,” doesn’t it? If they’re skeptical of homeopathy, tarot cards, psychic mediums and people who claim they can levitate, I can at least understand the urge to ask tough questions about all these things. I ask tough questions, too, especially when people tell me they’ve seen ghosts or spirits coming back from the dead or other unexplained phenomena. (And I’ve already publicly denounced so-called “psychic surgery” which it quite obviously little more than sleight-of-hand trickery combined with animal blood.)
But most conventional skeptics never step out of bounds of their “safety zone” of popular topics for which skepticism may be safely expressed. They won’t dare ask skeptical questions about the quack science backing the pharmaceutical industry, for example. Nor will they ask tough questions about vaccines, or mammography, or chemotherapy. And you’d be hard pressed to find anything more steeped in outright fraudulent quackery than the pharmaceutical industry as operated today (and the cancer branch of it in particular).
That’s why I’m skeptical about the skeptics. If a skeptic doesn’t question the loosey goosey pseudoscience practiced by Big Pharma, then they really have no credibility as a skeptic. You can’t be selectively skeptical about some things but then a fall-for-anything fool on other scams just because they’re backed by drug companies.
This passage, more than any other, amuses the hell out of me. Why? First, it amuses me because Adams pats himself on the back because he’s skeptical of paranormal phenomena and has actually figured out that psychic surgery is nothing more than sleight-of-hand. Wow! That took a lot of effort and knowledge! Apparently, though, he used up all that skepticism on psychic surgery, because there isn’t a form of other quackery that Adams won’t defend. Indeed, his website is a one-stop shop for all things quackery. More amusingly, even according to Adams’ self-serving criteria, I’m still a skeptic. After all, I’ve attacked the pharmaceutical industry when it’s misstepped. I’ve written about mammography and its shortcomings extensively and given realistic appraisals of what chemotherapy can and can’t do.
Adams parodies skeptics so that he can dismiss them and then pat himself on the back for doing so, but he doesn’t realize that most skeptics who write about quackery of the sort he so loves. Certainly I don’t. Neither does Steve Novella. Neither does Mark Crislip. Ben Goldacre, whom I met last week for the first time, criticizes homeopathy and “alt-med,” but in reality his true notoriety comes from criticizing pharmaceutical companies. Indeed, last week I heard him accuse the pharmaceutical companies of lying to him about their drugs. No, the real difference between skeptics and pseudoskeptics is that skeptics base their skepticism on science and evidence. Pseudoskeptics like Mike Adams do not; rather, they base it in ideology.
Of course, skeptics have different interests, and there is nothing wrong with that. Some of us are more interested in the claims of alt-med than others, and some aren’t very interested in the misdeeds of pharmaceutical companies. I’ve heard some argue that there are plenty of watchdogs who criticize pharmaceutical company misdeeds but not so many taking on the claims of alt-med. Again, there is nothing wrong with that. More importantly, we are consistent in our application of evidence, regardless of whether we’re applying our skepticism to Mike Adams’ quackery of the perfidy of pharmaceutical companies.
One last point: For all the railing Mike Adams does against science and science-based medicine, remember this. It was not Mike Adams who discovered this potential problem with placebos. It was not one of Mike Adam’s merry band of quacks who discovered this potential problem. I daresay it was not even one of Mike Adams’ readers who discovered this problem. No! It was scientists, examining the methods of science-based medicine who did the hard work of carrying out the study. In essence, it was science criticizing itself.
The contrast with Mike Adams and is ilk couldn’t be more stark.
Golomb BA, Erickson LC, Koperski S, Sack D, Enkin M, & Howick J (2010). What’s in placebos: who knows? Analysis of randomized, controlled trials. Annals of internal medicine, 153 (8), 532-5 PMID: 20956710