Even with cable, there isn't much on TV before six in the morning. On the stationary bike today I was flipping through the channels and I just had to watch the infomercial for No Evil Oil. According to the testimonials, this stuff not only keeps out the Devil, but gets rid of shingles and makes you rich. The smiling preacher with the mullet will send it to you for free (but there are plenty of donation buttons).
You may be asking yourself, "How dumb/desperate/gullible do you have to be to use the No Evil Oil?" Even my most credulous readers would agree that this stuff is snake oil.
While we probably can't design a study to measure it's Satan-shunning capabilities, we certainly could investigate its medical properties. What if a double-blind, randomized placebo-controlled trial, conducted technically well, showed that No Evil Oil was superior to placebo in reducing the severity and duration of flu symptoms? Should we start prescribing it?
There are many ways to explain seemingly unexplainable results. Here are some options for our hypothetical study:
- No Evil Oil works. We don't know how it works, but it does.
- The study was subject to bias: perhaps randomization was poorly done, or the operational measures were systematically flawed, or there were unequal attrition rates in the groups---there are many ways to screw up an experiment.
- There was a confounding variable responsible for the disparity between groups
- There was reporting bias: maybe fifteen other studies failed to find a significant effect.
There are, in fact, many possible explanations for a "positive" study other than "No Evil Oil Works"---that is but one of many possibilities. How are we to figure out which explanation best describes reality?
One way is to look at our basic premise and assess how likely it is that "NEO works" is the right answer. We can look at this a number of ways. Let's do another thought experiment. Let's say that we didn't do a randomized controlled trial but instead a sort of cohort study. We dig up the charts from a clinic and grab out all the flu patients and divide them into groups of "got better fast" and "got better slow". We then comb through the charts to see if there are significant differences between these two groups. We may find a lot of differences: some are smokers, some have asthma, some are older, and some used the ever-popular No Evil Oil. All of these differences we find to be statistically significant. How do we decide which, if any, are important clinically?
The answer to both thought experiments is that we must consider some sort of basic plausibility. We know that No Evil Oil is just some canola with red dye #40 in it and is for all intents and purposes biologically inert. The only difference between No Evil Oil and the placebo is the preacher's magic blessing. We could posit that the blessing makes all the difference, but that would require us to ignore the bulk of our understanding of physics, chemistry, and biology. Since the overwhelming evidence supports our current understanding of science, we are unlikely to abandon this in favor of one or two odd experiments.
But, you may argue, what if our science really is wrong? Should we dismiss these results just because we don't "believe" in them?
The nature of science is to be very skeptical. If we test our hypothesis over and over, using different methodologies, and repeated experiments are done by others that find the same thing, we might---just might---be on to something. Or we might still be missing an alternate explanation or confounding variable.
One thing that we should not do is change the way we treat the flu based on one or two tests of a very improbable hypothesis. The practice of medicine deals with real people, and messing with their health and their hope is immoral and wrong. We work from the best evidence available because it is the right thing to do. That is what produces predictable outcomes and allows us to make intelligent decisions that helps our patients. If we jump on every cockamamie idea, we are performing a great disservice.
Oh, and if you got the idea that when I said "No Evil Oil" I really meant "Occillococcinum", you were right.
- Log in to post comments
Which is why my HD setup is connected to a computer with a LOT of e-books. Baen WebScriptions has a long way to go to make up for all of the novels I've read while lying immobile, but it's off to a good start.
There's a "morning" before six? I don't think I understand.
From May through September it's the only halfway-comfortable time of day to be outside.
Didn't you forget to re-dilute the No Evil Oil and then shake it really hard? You know that could have thrown off the entire experiment!
Ah, but the placebo effect is real.... perhaps people getting red oil thought better of it than people getting plain oil.
Or perhaps it's no placebo, but since it was red, they spent longer washing it off.
I'm reminded of a trial that attempted to prove or disprove the benefits of ear candling, but didn't take account of the fact that the instructions for the ear candles told people to rub some oil into their ears first -- maybe *that* was the clinically significant part. Ya never can tell.
Whaaaaat...the font on that pastor's banner looks oddly familiar.