Acupuncturists mistake insufficient rigor for bias against them

Of all the modalities of alternative medicine currently in use, arguably acupuncture is the one that has achieved the most mainstream acceptance in medicine. I've often asked why it has become so common in academic medical centers and elsewhere, despite the evidence being overwhelmingly in favor of the conclusion that it is nothing more than a theatrical placebo. It doesn't matter that acupuncture is part of a prescientific system of medicine now known as traditional Chinese medicine (TCM), whose concepts are rooted in vitalism. It doesn't matter that what has passed for acupuncture since the 1940s and 1950s bears little resemblance to what was practiced hundreds or thousands of years ago and even 100 years ago more resembled the bloodletting that was common in ancient European medicine more than the use of thin needles along "meridians" to "redirect the flow" of the life energy qi to alleged healing effect. It doesn't even matter that acupuncture, along with the rest of TCM, was basically foisted upon the world when Chairman Mao Zedong, unable to provide real medicine for his people after World War II, decided to make up the difference with folk healers even though he himself basically didn't believe in TCM and chose "Western" medicine instead. At that point, the history of TCM and acupuncture were effectively "retconned" into the form we know them today. In the case of acupuncture in particular, the retconned version, which posits acupuncture as being thousands of years old and having been practiced in basically the same form over all that time, has little resemblance to the original, which wasn't documented until the 17th century. Basically, TCM is more philosophy than medicine. None of this has stopped China from continuing to support and do its best to export TCM to the world, even in 2017. Meanwhile media falsely portray acupuncture as having been proven effective for a number of conditions.

I've discussed acupuncture many, many times on this blog, sometimes finding some truly spectacular examples of silliness, because it's an excellent alternative medicine modality to use to discuss placebo effects, shortcomings of clinical trials, and how alternative medicine is sold to the public—even revisionist history. In particular, it's interesting to note how very many forms of acupuncture have developed, and how they are used, and how many conditions that acupuncture can supposedly treat, which encompass basically everything. It's also a great jumping off point to discuss appropriate placebo controls, something that many acupuncture studies conveniently leave out, and how, the larger and more rigorous a study is, the smaller the effect observed is, until the largest, most rigorous studies show no difference between "sham" acupuncture and "real" acupuncture, and how meta-analyses sold as showing the therapeutic efficacy of acupuncture in actuality show no such thing. Basically, it doesn't matter where you stick the needles or even if you stick the needles in; the effect is the same. And I haven't even pointed out how studies of acupuncture from China are virtually always positive.

Not surprisingly, acupuncturists and believers in acupuncture always think that their favorite modality is being treated unfairly. Not surprisingly, many of them publish in the Journal of Alternative and Complementary Medicine (JACM). Unfortunately, at least one of them writes for Popular Science.

In which medicine is accused of a "double standard" with respect to acupuncture

What made me think about these issues was a recent article by Kendra Pierre-Louis in Popular Science entitled Why it’s so hard to figure out if acupuncture actually works: Should you stick a needle in it? Notice the statement in the title. Yes, I agree that it's hard to figure out whether acupuncture works. Past tense would be better, though, as figuring out whether acupuncture works is something that has already been done over the last three decades. It doesn't.

Pierre-Louis, however, doesn't seem to think so:

Does medicine have a bias against acupuncture?

That's the verdict of a paper (and an accompanying commentary) published earlier this week in The Journal of Alternative and Complementary Medicine. While there’s still no medical consensus on acupuncture, and most reputable medical organizations do not support its use for arthritic knee pain, the authors' critique lends interesting insight into the process by which medical procedures are accepted—and which are excluded.

How does alternative medicine get to be plain-old medicine?

In fairness, that last question is a very good question that plays off a common adage in skepticism and science-based medicine: What do you call alternative medicine that has been scientifically proven to work? Answer: Medicine. But what does "scientifically proven to work" mean? If Pierre-Louis had focused on that question, using acupuncture as an example, even if I ended up disagreeing with her conclusion, the article would have been much more worthwhile. Unfortunately, Pierre-Louis is not about exploring that question; she's about proving that acupuncture actually works and the UK's National Institute for Health and Care Excellence (NICE), which provides guidelines for recommended treatments for diseases and conditions, has a bug up its butt about acupuncture:

Led by Stephen Birch of Kristiania University College in Norway, the researchers behind the new study allege that the United Kingdom’s National Institute for Health and Care Excellence (NICE), which details recommended treatments for given ailments, holds acupuncture to a higher standard than it does traditional medical modalities.

If all of the treatments that NICE recommend for knee arthritis—including weight loss and nonsteroidal anti-inflammatory (NSAID) drugs like ibuprofen—had to meet the minimum required standards that NICE sets for acupuncture, “opiates would become the first line of drug prescription,” wrote Birch et al.

I'd hardly call Birch's article a "study." It's more of an opinion piece with some data backing it up kvetching about how NICE came up with its guidelines for the treatment of osteoarthritis of the knee. Before coming back to the rest of Pierre-Louis' piece, let's take a look at Birch's article, whose title gives away the game, The U.K. NICE 2014 Guidelines for Osteoarthritis of the Knee: Lessons Learned in a Narrative Review Addressing Inadvertent Limitations and Bias.

Bias. You keep using that word. I do not think it means what you think it means.

Birch's complaint is stated most succinctly in the abstract:

Several systematic reviews suggest that acupuncture is effective for knee osteoarthritis (OA), and furthermore a safe and cost-effective treatment for this condition. A recent clinical practice guideline (CPG) from the National Institute for Health and Care Excellence (NICE), in the United Kingdom, recommended against the use of acupuncture on the grounds that the effect size (ES) in comparison with sham acupuncture is too small.

I read this, and couldn't help but think, "You say that as though it were a bad thing." And, yes, Birch does think it's a bad thing:

The 2014 NICE guideline on OA (CG177) was not a complete revision of the 2008 guidelines, it was more limited. Although originally intended to be more extensive including a broader review of drugs such as NSAIDs, it was decided to review those at a later date. Not only did the update focus on only a few of the interventions, it also applied different criteria for evaluating and accepting in- terventions. In effect, as will be described, the process se- lected for evaluating acupuncture required the treatment to meet higher standards than many other included treatments. A key problem in this NICE update is the introduction of a focus on the development of recommendations based on the consideration of which interventions make ‘‘minimal important differences’’ (MIDs) to patients as a replacement for usual CPG comparisons of evidence. The MID was set as an effect size (ES) of 0.5 or greater. The analysis of the evidence for acupuncture in this review emphasized results from sham studies with a de-emphasis on evidence from pragmatic comparator studies. Despite the fact that the assessment of MIDs should include analysis of benefit and harm, this review did not include an analysis of safety data on acupuncture. This coupled with the lack of usual CPG head-to-head comparison of interventions excluded data relevant to the use of acupuncture as an intervention for knee OA in comparison with other standard recommended care options, namely that it is much safer than many other accepted interventions for knee OA pain. This shift in methodological approaches might have inadvertently biased against acupuncture.

Notice the main complaint about the NICE guidelines: That they're too rigorous. The dead giveaway in the paragraph above is this sentence: The analysis of the evidence for acupuncture in this review emphasized results from sham studies with a de-emphasis on evidence from pragmatic comparator studies. I've discussed before how much acupuncture advocates love pragmatic studies. Pragmatic studies are designed to demonstrate the "real world" effectiveness of a treatment. Often they are not randomized and don't have a placebo or sham control. Clinical trials are very regimented, with clearly defined protocols, rigorous inclusion and exclusion criteria for subjects, endpoints, and reporting. When a treatment that's been shown to be efficacious in randomized clinical trials gets out into the real world, the inclusion criteria inevitably expand and the exclusion criteria contract. When that happens, patients who wouldn't have been eligible for the clinical trials used to demonstrate the efficacy of the treatment receive the treatment, and the possibility of the treatment not working in these patients or causing adverse reactions becomes a real concern. The idea of a pragmatic trial is to study how well a treatment that has already been demonstrated to be effective and safe works in the real world. Here's the thing. Pragmatic trials are not meant to be the first to demonstrate efficacy. They are meant to see if a treatment proven in randomized clinical trials still works as well when released "into the wild," so to speak.

Unfortunately, acupuncture advocates flip the order and put the cart before the horse, emphasizing pragmatic trials. The reason is obvious. Pragmatic trials, because they often don't have a placebo control arm or a sham treatment arm, by their very nature, will produce positive results. That's acceptable for pragmatic trials used properly, because they aren't meant to be the first demonstration of efficacy of a trial. That's exactly what Birch et al are doing here, and the reason they focused on osteoarthritis of the knee is because they can point to a meta-analysis that acupuncturists love to cite as evidence that acupuncture "works." It was discussed many times by skeptics when it was first published, and its author, Andrew Vickers, was most unhappy about the criticism.

See what I mean in the accompanying commentary by Hugh MacPherson:

Second, a new methodological approach was used in 2014. Different thresholds were used for the recommenda- tions in 2008 from those made as part of the 2014 update. The new guidance introduced the concept of the minimum clinically important difference (MCID), and required those interventions such as acupuncture that were within the scope of the update to show an MCID of an effect size of 0.5, a difference that is commonly considered ‘‘moderate.’’ When acupuncture is compared with usual care, there is an effect size of approximately 0.5, as has been shown in an individual patient data meta-analysis of acupuncture for osteoarthritis.

As I pointed out, the Vickers meta-analysis did not show an MCID for acupuncture compared to sham acupuncture. That's why MacPherson harps on an MCID between acupuncture and no treatment/usual treatment, a measure that maximizes placebo effects.

Does acupuncture do anything besides leaving little holes in the skin?

I'll have more to say about Birch et al, but this seemed like a good point to jump back to Pierre-Louis' article, because she asks the very pertinent question: Does acupuncture do anything? She seems quite taken by the fact that earlier this year the American College of Physicians listed acupuncture as a minimal invasive treatment of low back pain. But did it? Yes, it sort of did, but it was hardly a ringing endorsement. The actual studies used to justify the recommendations. For example, this study showed that there was no difference between “true” acupuncture and sham acupuncture (which is what pretty much all well-designed acupuncture studies show). In other words, it showed that acupuncture is a placebo intervention. This study tested a sort of “scalp acupuncture” that I’ve never heard of before. Another study was from China, and, as has been discussed before, pretty much all acupuncture studies out of China are positive.

What about chronic low back pain? Take a look:

Low-quality evidence showed that acupuncture was associated with moderate improvement in pain relief immediately after treatment and up to 12 weeks later compared with sham acupuncture, but there was no improvement in function (125–130). Moderate-quality evidence showed that acupuncture was associated with moderately lower pain intensity and improved function compared with no acupuncture at the end of treatment (125). Low-quality evidence showed a small improvement in pain relief and function compared with medications (NSAIDs, muscle relaxants, or analgesics) (125).

I laughed. At least, I laughed when I looked over the list of references. The authors actually included the GERAC study in its list, which basically showed that acupuncture does not work, given that sham acupuncture was indistinguishable from acupuncture. Another study was a “bait and switch” in that it studied “electroacupuncture,” which is in reality TENS. We’re talking thin gruel indeed.

However, that's not enough. Pierre-Louis has to pick out another study that really impressed her.

Carpal tunnel and acupuncture a-go-go

The study that impressed Pierre-Louis so much is one that has special resonance for me, because I once suffered from the condition for which acupuncture was tested. I'm referring to carpal tunnel syndrome. Yes, around 16 years ago, I developed a really bad case of carpal tunnel syndrome in my left hand. Given that I'm a surgeon, you can appreciate the anxiety and fear that my symptoms and diagnosis caused me. I saw my career as potentially being over. The pain and numbness were manageable for a while, but I saw the progressive nature of my carpal tunnel syndrome threatening my ability to operate. I even endured a nerve conduction study, which is basically torture with electricity. In the end, I was basically begging the hand surgeon to operate on me because I couldn't take it any more. Eventually, he did, and I recovered promptly. My career was saved. I still get a twinge every now and then, but haven't had any major carpal tunnel symptoms since.

So Pierre-Louis invokes a recent study that I meant to blog about when it first appeared, but for some reason never did:

A more intriguing example is a March study that appeared in the journal Brain, which found that acupuncture improved the outcomes for carpal tunnel syndrome by literally remapping the brain. Researchers came to that conclusion after exposing subjects diagnosed with carpal tunnel syndrome—broken into three groups—to acupuncture treatments.

Patients in the first group received an acupuncture treatment as prescribed by traditional Chinese Medicine—that is, needles were inserted at the site of the pain. The researchers exposed the second group to something known as distal needle acupuncture, in which acupuncture needles aren’t inserted where it hurts, but rather at other sites that practitioners say are connected to the painful regions by “channels of energy.” Yes, we know this sounds like snake oil—and the study authors aren’t alleging that the so-called channels exist. But if you want to know if something—a drug, a workout regimen, or in this case an acupuncture technique—has any effect, you have to test it.

Finally, a third group received what’s known as sham acupuncture, which is essentially the sugar pill of acupuncture. In this case, sham acupuncture involved non-penetrating placebo needles designed to convince participants that they had undergone a real acupuncture treatment. Each participant received 16 treatments of their designated form of acupuncture over the course of eight weeks.

At the end of the study, all groups equally reported that their symptoms had improved. That's proof that acupuncture is a sham, right? Not exactly.

This is the study. Each group of patients with mild to moderate carpal tunnel syndrome received 16 sessions of acupuncture over eight weeks, with before-and-after carpal tunnel symptom questionnaires, median nerve conduction studies, and, of course, functional MRI (fMRI). In any event, the reason why all three groups improved in the short term should be obvious. All are placebo. This result is exactly what I would have predicted. Of course, the other thing that is obvious is that this is a typical "bait and switch" acupuncture study. It's not acupuncture at all. It's electroacupuncture, which is in reality a form of transcutaneous nerve stimulation (TENS). Let's just put it this way: There was no electricity hundreds or thousands of years ago. Hooking up electrodes to acupuncture needles is a 20th century "innovation" that serves no apparent reason other than to "sex up" acupuncture. It also makes no anatomic sense. The distal acupuncture, for instance, was administered to acupuncture points in the lower leg; there is no anatomic reason to believe that this should have any effect on nerves in the distal arm. Remember, carpal tunnel syndrome is due to a very specific anatomic issue, entrapment of the median nerve as it goes through the carpal tunnel, defined by bones and ligaments in the wrist. That's why the surgical treatment, cutting the transverse carpal ligament, works; it frees the entrapped nerve.

Now here's where this study has some interesting results. They are not results that have anything to do with acupuncture, as this is in reality the application of electrical current at acupuncture points. But they are somewhat interesting. First, there was a modest (and not strongly statistically significant) improvement in median nerve conduction in both the distal and local acupuncture groups compared to the sham. This result makes little sense from an anatomic standpoint, although it's not entirely implausible that running a weak electrical current along the lower arm might potentially affect nerve conduction studies in the same arm. In fMRI studies, the investigators reported brain remapping associated with symptom improvement only in those receiving the local acupuncture, a result that's somewhat plausible. Of course, again, none of these results has anything to do with acupuncture, the existence of meridians, or TCM, even as they are sold as slam dunk evidence that "acupuncture" remaps the brain.

This study also has some significant flaws. For instance, although it was blinded, it appears not to have been truly double-blinded. Patients were blinded to allocation, as were the study physicians who evaluated patients for inclusion. Experimenters, the people who did the nerve conduction studies and evaluated the fMRIs, were blinded as well. You know who weren't blinded? The acupuncturists. I'll give the investigators credit for inserting all the needles in the distal and local acupuncture groups to hide from the patient the allocation. However, the acupuncturists knew which experimental group each patient was in at the first treatment session. So that's a huge problem right there, especially since adequacy of blinding was only evaluated after the first treatment and the patients were unblinded after the fMRI. There's also another huge defect in that the sham acupuncture specifically does not pass current through the needles. A much better control would have been to use random acupuncture points and actually run the same current through the needles. My guess is that this group, had it been included, would have shown the same results as the distal and local acupuncture groups.

So basically, what this study shows is mildly interesting, but is not evidence that acupuncture relieves symptoms and function in patients with mild to moderate carpal tunnel. The most generous interpretation is that passing current through the forearm 16 times over eight weeks might improve nerve conduction somewhat. That's about it. Unfortunately, in a statement issued by Massachusetts General Hospital (which ran the study) and Harvard, the study principal investigator Nataly Napadow that basically sells this as showing that "true acupuncture" works by a different mechanism than the "placebo" of sham acupuncture:

Even after years of clinical research, controversy continues as to whether acupuncture works primarily by the placebo effect, especially given the slight differences between the efficacy of real and sham acupuncture. The findings of the Brain study help to address this question. Sham acupuncture may produce a stronger placebo effect than a pill because it sends inputs to the brain via skin receptors and is coupled with a specific ritual. But the symptom improvement produced by sham treatment for conditions like CTS also might derive from entirely different mechanisms than those elicited by real acupuncture, the mechanisms of which may more specifically target CTS pathophysiology.

No, no, no. Maybe running electricity through the leg or arm causes an effect, but acupuncture doesn't. After all, what was the main difference between both "true" acupuncture groups and the sham acupuncture group? It was the application of electrical current!

Studies like this and how they are sold infuriate me and are not evidence that acupuncture "works." In this case, though, the press pretty much credulously ate up the given narrative, and I couldn't find a single skeptical take on the study anywhere.

When is a placebo not a placebo?

Pierre-Louis, unfortunately, completely buys into Napadow's narrative:

Recall that if you were to throw out the objective measure’s—the fMRI and nerve conductivity tests—Napadow’s study looks like a dud, because patients expressed equal levels of pain reduction whether they experienced real acupuncture or sham acupuncture.

The whole point of a placebo is that it's inert. It doesn't actually have an effect on the body, making it an excellent control. But physical interventions aren't quite the same as sugar pills: even in sham acupuncture, there is some pressure or sensation being inflicted on the patient's body. In fact, some cases of "sham" acupuncture even involve the insertion a needle, though Napadow's study did not.

“There's a controversy as to what sham acupuncture is,” said Napadow, “you still have a tactile sensation and a somatosensory input as a result of sham acupuncture.”

No, there really isn't a controversy over what sham acupuncture is. At least, there shouldn't be. The idea behind placebo is to isolate the aspect of the treatment that has a specific physiologic effect, which sham acupuncture does in studies of acupuncture studies. Napadow failed to do that in his carpal tunnel syndrome study. The way his study was designed, whether you believe the results or not, all it could do was to isolate the electricity, not the "true" acupuncture, as the part of the treatment modality that causes a physiologic effect. His study shows nothing about acupuncture. Again, repeat after me: It doesn't matter where you stick the needles. It doesn't even matter if you stick the needles in.

In his commentary, MacPherson argues that acupuncture for knee osteoarthritis was given a bum rap by NICE, that NICE applied a double standard. She points out that the evidence for surgical interventions for this condition, such as arthroscopic debridement and lavage, is not very strong and that these interventions don't reach the threshold for effect size required by NICE, even though NICE still recommends these procedures, asking. Like so many alternative medicine apologists, he frames it in terms of the opioid epidemic:

As news of these questionable recommendations spreads, many people will be misled, such that acupuncture services may be commissioned less often, conventional medical practitioners may refer their patients less, and patients might find access to acupuncture more difficult. Moreover, misleading guidance related to chronic pain may inadvertently drive up the utilization of opioids, which is already emerging as an epidemic in the United States.

Birch et al go even further, using the NICE recommendations as an excuse to advocate pragmatic trials:

If NICE were to apply the same criteria to OA [osteoarthritis] treatments, only opioids would be left and with their adverse events profile that may not be acceptable to some patients suffering from OA of the knee. Furthermore, in agreement with other authors, it is time to stop performing sham acupuncture as a control treatment in acupuncture studies.

Just a reminder: No sham acupuncture control = pragmatic study.

So basically, Birch et al and MacPherson are basically arguing that the new NICE criteria are too strict and exclude acupuncture without excluding conventional surgical treatments that don't reach the designated effect size and are not superior to sham surgery. They use these observations to argue that acupuncture should be included in NICE recommendations for osteoarthritis, too.

Pierre-Louis then asks the right question, but implies the wrong answer:

The researchers ask a simple question: if most of the recommended treatments fail to meet this threshold, why is acupuncture excluded while arguably more invasive procedures are included? But as patients, we should perhaps be asking a different question. Why are we being recommended procedures that don’t work?

An excellent question, actually. Unfortunately, the answer given by Birch et al and MacPherson—for NICE to start recommending acupuncture for osteoarthritis—is the wrong answer. The correct answer is that, if surgical interventions for osteoarthritis truly don't reach the new NICE threshold for MID, then the answer is not to include acupuncture in NICE recommendations for osteoarthritis, but rather to stop including those surgical interventions. That's not how alternative medicine practitioners and apologists see the world, though. When conventional therapies don't reach a given standard of evidence for efficacy and safety, instead of advocating that those treatments be abandoned, as those of us who advocate for science-based medicine do, they use that as a reason for accepting their quackery as valid treatments too. If it requires weakening the standards of evidence to let their quackery stand "integrated" with science-based medicine, so much the better.