Adventures in Ethics and Science

A little while ago, PalMD put up a post at Whitecoat Underground about the current state of the National Center on Complementary and Alternative Medicine (NCCAM), especially at a moment in history when the federal government is spending loads of money (and thus maybe should be on the lookout for expenditures that might not be necessary) and when health care reform might actually happen. Pal wrote:

The whole idea of setting up such an agency is a bit quixotic—after all, the National Institutes of Health already study health science. …. Many, many studies have been funded which fail basic tests of plausibility and ethical propriety. Also, a huge percentage of the studies funded fail to ever publish their results. Still, some studies have been published, and more often than not, they find that the “alternative” modality being studied fails to behave better than placebo. That’s probably the sole redeeming quality of the agency, but not enough to keep it open, as these studies could have been done under the auspices of the NIH.

Since I have not been a close follower of NCCAM, I figured I’d have a look at their website. (I was a wee bit apprehensive given the trauma the website inflicted on Dr. Benway.) Whether of not the agency you think the agency ought to exist, it does lay out some relevant terminology very clearly. For example, what is CAM?

CAM is a group of diverse medical and health care systems, practices, and products that are not presently considered to be part of conventional medicine. Complementary medicine is used together with conventional medicine, and alternative medicine is used in place of conventional medicine. Conventional medicine is medicine as practiced by holders of M.D. (medical doctor) or D.O. (doctor of osteopathy) degrees and by their allied health professionals, such as physical therapists, psychologists, and registered nurses. Some health care providers practice both CAM and conventional medicine.

The list of practices that are considered CAM changes continually, as those therapies that are proven to be safe and effective become adopted into conventional health care and as new approaches to health care emerge.

And, what is NCCAM’s role in the field of CAM?

The National Center for Complementary and Alternative Medicine (NCCAM) is the Federal Government’s lead agency for scientific research on CAM. NCCAM’s mission is to explore complementary and alternative healing practices in the context of rigorous science, train CAM researchers, and disseminate authoritative information to the public and professionals.

So, the practices people use to deal with illness, injury, or health maintenance can be divided into two piles: those that are accepted by conventional medicine (as practiced in the U.S. at this point in history – NCCAM is an agency of the U.S. federal government), and those that are not accepted by conventional medicine.

Those two piles correspond at least approximately to another distinction: practices whose safety and efficacy are well-supported by empirical evidence, and practices whose safety and efficacy are not so supported.

It’s worth pointing out here that an intervention whose safety and efficacy are not currently supported by empirical evidence may find such support if the relevant research is done. In the absence of empirical evidence one way or another, it may be the case that we just don’t know – at which point, there may be good reason to set about getting the empirical evidence that would answer the question for us. (There may also be good reasons to be cautious in trying to get this empirical evidence; more on this below.)

It also seems likely that there are some parts of conventional medical practice that are not driven entirely by the evidence from placebo-controlled double-blind clinical trials. There are lots of pieces of the interaction between patient and health care provider that feel more like unscripted interpersonal interaction than well-tested scientific intervention. In a word, these conventional practices may be conventions, practiced because that’s what people in the health care community seem to do (and it seems to work reasonably well). Possibly these conventions have no effect whatsoever on patient outcomes. If they do, though, presumably there might be some value in mounting empirical trials to determine what kind of effects they have.

Nonetheless, the idea is that conventional medicine is striving toward developing an arsenal of health care interventions that are grounded in good empirical evidence – so much so that once an “alternative” treatment gets this kind of grounding, it is taken on board by practitioners of conventional medicine.

So why not take stock of all the alternative treatments out there, set up clinical trials, and get the evidence that will sort out the good from the bad?

The short answer is that clinical trials are experiments with human subjects, and there are ethical and regulatory constraints on the kinds of experiments you can conduct on humans. If the foreseeable risks to the human subjects of the experiment are great enough, or if they outweigh the foreseeable benefits to the human subjects, then you can’t do the experiments – even if the knowledge they might generate would be really useful.

The NCCAM website actually has a pretty clear explanation of what a clinical trial is (and of the different phases of clinical experimentation). Here’s how that site describes the possible benefits of being in a clinical trial:

  • You will receive expert medical care.
  • Your health will be closely watched throughout the study.
  • Clinical trials can be one treatment or prevention option for a disease or condition.
  • In some types of trials, you may be among the first to benefit from a new treatment or new knowledge about a current treatment.
  • You will help others by helping to advance medical and scientific knowledge.

And here’s how it describes the possible risks to being in a clinical trial:

There are possible risks to being in a clinical trial, as there are with any form of treatment and with illness itself. In a clinical trial:

  • The treatment under study does not always turn out to be better than, or even as good as, standard treatment. The researchers hope that it is, but they need to do the study to find out for sure.
  • The treatment may have side effects that are unknown to the researchers or different from what they expect.
  • If you are in a randomized trial, you may be assigned to the control group, where you will not receive the treatment under study.
  • Like standard treatments, the treatment under study may not work for everyone.
  • Participation may require more tests and more visits or treatments than regular care.
  • There may be costs to participate, and these costs may not all be covered by health insurance plans. Be sure to talk with the research team about any costs involved and your situation.

Depending on the disease or condition being treated, receiving an ineffective treatment can be a very big harm. Especially if there’s a reasonably safe and effective treatment that’s already part of the conventional medical practitioner’s bag of tricks, the benefit of establishing the safety and efficacy of an unproven alternative may turn out to be quite small relative to the risks. Of course, if the established treatment is not so effective for particular patient populations, or if the side-effects for particular patient populations are very serious, that may chance the risk-benefit calculus. But the safety of the human subjects of the research is prioritized – which means, in some cases, the (limited) information we have makes us judge the costs of building more complete knowledge are just too high to warrant proceeding with the research.

Going forward with a clinical trial that we have good reason to think would be harmful to the human subjects crosses the line ethically. I think there’s also a feeling that plausibility ought to play a role in the evaluation of whether a clinical trial is ethical. Plausibility is tricky – it’s a question about whether a particular kind of intervention could possibly be effective given the current state of our understanding (of human biology, of this sort of intervention, etc.). Clearly, the current state of our understanding is incomplete and subject to revision; this is why scientists are still involved in conducting research.

Does “X could plausibly treat condition A” mean that there must be some clear mechanism by which X might act to cure or improve condition A? There have been compounds whose efficacy was widely accepted before we had anything like a detailed understanding of the mechanisms by which they worked (think aspirin). Does “X could plausibly treat condition A” mean that there already exists a body of empirical data demonstrating its efficacy? If this were the standard, no compound that wasn’t already being used by a significant number of people could ever make it to clinical trials.

Plausibility does play some sort of role in the evaluation of the risks an intervention might pose to human subjects. As I’ve discussed before, researchers have to make their best guesses of likelihood of benefit and likelihood of harm given the knowledge that exists. Since humans are subjects of the research (and because we recognize that human subjects are entitled to certain protections), the current state of the knowledge might rule out research to build a new piece of knowledge on the basis that building that new knowledge might come at too high a price for the human subjects.

It’s worth noting, though, that at least some of the “alternative” treatments that turn up in NCCAM studies involve stuff a good number of people do everyday that they might not consider a medical treatment – drinking cranberry juice or herb tea, eating tofu, getting a massage, meditating, doing yoga. At least from my vantage point in the land of veggie restaurants and yoga studios, I don’t imagine that these “alternative treatments” are feared to be harmful in themselves. Rather, they are probably viewed as “risky” to the extent that they may replace medical interventions that are well-grounded in empirical evidence – or to the extent that funding to study them in clinical trials might waste funds that would be better directed to studying interventions judged more likely to be effective.

There are interesting questions here about the challenges of distinguishing the causal impacts of all the kinds of things people actually do with their bodies (ingesting medications, ingesting food, exercising, sleeping, getting stressed, getting germy) on the state of their health. Our presumptions about which of these things are likely to have the biggest effects – and the medical community’s presumptions – may shift in the light of new information. Sometimes this information comes from well-controlled experiments, but sometimes it might come from other places (like our own personal experience or anecdotes from others). It’s also worth asking whether some of the causal impacts of the things people do are judged as probably-not-that-important because it’s wicked hard to control them in experiments.

Let me close with a question for the biomedical researchers in the audience: What is the practical difference in getting a clinical trial approved and funded through NCCAM versus NIH? Could the sorts of research overseen by NCCAM be folded into NIH without too much trouble? (If not, why not?)


  1. #1 Orac
    March 17, 2009

    Could the sorts of research overseen by NCCAM be folded into NIH without too much trouble?

    Yes, scientifically. Politically, not so easy, thanks to Tom Harkin.

    Perhaps I’ll address this tomorrow, as I was planning an NCCAM post anyway for other reasons. I’ll also mention that NCCAM is one of the two primary promoters of CAM, its mission statement notwithstanding. You might want to start here:

  2. #2 Kimball C Atwood
    March 17, 2009


    No time for a comprehensive reply, but a couple of things:

    First, you are absolutely correct to worry about human studies ethics, but what you read on the NCCAM website does not necessarily reflect what the Center does. The largest, most expensive, and most unethical trial yet funded by the NCCAM is the Trial to Assess Chelation Therapy (TACT). To find out the specifics, please start here:

    Here you’ll see that recruiting for the TACT has recently been halted:

    You’ll find nothing about that on the NCCAM website.

    Another example:

    Here you’ll find an overview of human studies ethics applied to trials of highly implausible claims:

    Your discussion of plausibility is mistaken, I’m afraid. “Plausible” needn’t mean that we know the mechanism, only that it’s plausible that one exists. Thus aspirin was plausible from the outset, because chemicals can interact with other chemicals. “Balancing qi” is implausible because qi is not detectable or measurable in any reproducible way. The overwhelming likelihood is that qi doesn’t exist except as a poetic metaphor, but that isn’t even the point; as long as it is undetectable and unmeasurable, it isn’t useful. Many other claims that the NCCAM takes seriously are equally absurd. Some have already been subjected to decades of fruitless study (psychokinesis, for example), but you’d never know that by looking at the NCCAM website.

    Even molecules that may interact with human physiology venture beyond the plausible when proponents insist that “only the whole leaf will do,” or that herbs have co-evolved with humans in order to provide us with ‘natural’ remedies—common beliefs of NCCAM-boosters. Those who would look for the active ingredient in an herbal preparation in order to purify it, to measure it for rational dose-reponse activity, and perhaps to improve upon it in the lab, are not welcome at the NCCAM.

    Kimball Atwood

  3. #3 Dr Benway
    March 17, 2009

    Those two piles correspond at least approximately to another distinction: practices whose safety and efficacy are well-supported by empirical evidence, and practices whose safety and efficacy are not so supported.

    The real divide is deeper. There’s a fight over the rules of evidence. On one side, people agree that personal clinical observation is weaker than well controlled trials. On the other side, the rules are more elastic and can be bent to suit. Most MDs are on one side, but some are on the other side. Nearly all chiropractors and naturopaths are on the other side.

    Thus the tu quoque we science-lovers hear all the time: “You say we don’t have good evidence for cracking people’s necks. Well, you don’t have good evidence for such-and-such!”

    The science lovers reply, “Yes, we’ve got a ways to go; we’d better get busy.” The other side says, “The rules are arbitrary; why bother?”

    The science-lovers try to avoid tarting up new studies. They’d really like to restrict novel treatments to registered trials. The other side gets a great idea and whips up a batch of remedy in someone’s garage, to be distributed widely by anyone who “really cares about the children.”

    The other side are backed by a longer physicianly tradition than the science-based. Doctoring has been dominated by superstition and arguments from authority for most of human history until about the 1920s when the Flexner report was released.

    I’m no historian, but it’s my impression that Flexner was this dude who surveyed medical training in the US and immediately had a stroke. What he found was so disturbing that he managed to convince everyone that the entire system needed reform. Most medical schools were closed, and those that remained had their curricula standardized.

    The chiropractors and naturopaths were not well pleased with these changes. They felt they were on to something big and didn’t want to give it up. “Screw you and your stinking standards,” they said.

    The osteopaths and podiatrists took a less confronational, more Canadian approach. The podiatrists said, “Hello doctor. Are you tired of thick toenails and toxic socks syndrome? We can help!”

    The osteopaths said, “We’re not crazy like the Brits –we promise! And we’ll make your back feel great.”

    Friendships blossomed.

    Later other allied health professions emerged: physical therapists, occupational therapists, speech therapists, vocational therapists, rehab therapists, respiratory therapists, nurse practitioners, physician assistants, psychologists, behaviorists, dieticians, etc., etc. Each new specialty accepted the language and procedures of medicine (e.g., “subjective-objective-assessment-plan”) and added to the general system of care.

    Unfortunately, medicine today has no home for the chiros and naturopaths. Dieticians know more about nutrition and physical therapists make chiropractors kinda redundant.

    It’s all very sad, as I’m sure many young people who train to be chiropractors are honest and smart and genuinely want to help people. But the adversarial arrangement with the medical community has been a deal breaker. How does a doctor work with someone who believes that vaccines are poison?

    There’s a big chiropractic school in Iowa, and the senator from Iowa has made himself the champion of that profession. NCCAM, that channel of sweet Federal dollars, is his thank-you for their support.

    Unfortunately, elevating the profession with a little science at this late date is not enough. If the chiropractors can’t talk to the MDs, if they can only compete, their prognosis will remain poor.

    Senator Harkin has said he wants to end the “discrimination” against chiropractors. He compares the amount of federal funds going to “conventional” medical research and the amount going to research in “natural methods” to prove his point.

    In my view, anyone who can’t discriminate between a crack-yer-backer and medical doctor needs an eye exam.

    Competition with respect to efficiency is fine. Competition with respect to what is true is daft.

  4. #4 Thomas
    March 18, 2009

    Some of the herbal medicine trials seem reasonable — people are already using the stuff, and in some cases there is (weak, but non-zero) evidence that it works.

    I think the gingko trial is a good example — I know some of the people involved at the coordinating center for the trial, and their attitude was that it probably wouldn’t work, but it might, and since all the currently available treatments suck, it was worth trying.

    In some ways the biggest problem is the small effect of negative results. The gingko trial will only discourage a few people from taking gingko. In contrast, the Women’s Health Initiative results showing that estrogen supplementation doesn’t prevent chronic disease in post-menopausal women caused a dramatic collapse in sales (and probably has already paid for the huge cost of the study).

  5. #5 Dr Benway
    March 20, 2009

    Whether of not the agency you think the agency ought to exist, it does lay out some relevant terminology very clearly.

    Really? I thought it was obvious the definitions on that NCCAM site were stupid.

    Son: What is plumbing?
    Dad: It’s what plumber’s do.
    Son: What do plumber’s do?
    Dad: Plumbing.

    Not terribly informative. A tautology, I think.

    So, the practices people use to deal with illness, injury, or health maintenance can be divided into two piles: those that are accepted by conventional medicine… and those that are not accepted by conventional medicine.

    Except that isn’t actually correct. By the that definition, the “alternative” pile would have to include all research in progress involving new chemotherapies, surgical devices, etc.

    Dad: There are two kinds of plumbing: the kind plumbers do and the kind plumbers do not do.
    Son: What’s the difference?
    Dad: Well they’re not the same thing.

    Why was NCCAM created? For the sake of crazy stuff like homeopathy, chiropractic subluxations, and energy healing. We already had scientists studying herbs and stress reduction methods like meditation prior to NCCAM.

    In the case of weird, impossible stuff, the mere appearance of research provides more legitimacy than any proponent could ever hope for. And while the pseudo-research remains in progress, no definitive smack-down against the nonsense is possible.

    NCCAM is the most brilliant pro-quackery idea since Flexner. Expect more chiropractors and naturopaths to graduate from newly refurbished colleges (thanks, NCCAM grants!) around the US over the next half-century.

    Expect that one day your primary care physician will be a chiropractor.

  6. #6 Dr Benway
    March 21, 2009

    I ought not drink and post. I meant, “since the pre-Flexner era.” What I wrote seems to say Flexner had pro-quackery ideas.