If there’s one theme that’s run through this blog since the very beginning, it’s that the best medical care should be based on the best science. In other words, I like to think of myself as being far more for science- and evidence-based medicine, than I am against against so-called “complementary and alternative medicine” (CAM).
Unfortunately, even though the proportion of medical therapies not based on science is far lower than CAM advocates would like you to believe, there are still more treatments in “conventional” medicine that are insufficiently based on science or that have never been validated by proper randomized clinical trials. This is true for some because there are simply too few patients with a given disease; i.e., the disease is rare. Indeed, for some diseases, there will never be a definitive trial because they are just too uncommon. For others, it’s because of what I like to call medical fads, whereby a treatment appears effective anecdotally or in small uncontrolled trials and, due to the bandwagon effect, becomes widely adopted. Sometimes there is a financial incentive for such treatments to persist; sometimes it’s habit. Indeed, there’s an old saying that, for a treatment truly to disappear, the older generation of physicians has to retire or die off.
That is why I consider it worthwhile to write about a treatment that appears to be on the way to disappearing. At least, I hope that’s what’s going on. It’s also a cautionary tale about how the very same sorts of factors, such as placebo effects, reliance on anecdotal evidence, and regression to the mean, can bedevil those of us dedicated to science- and evidence-based medicine just as much as it does the investigation of CAM. Moreover, it’s a story that I first encountered four years ago and wrote about at that time. As such, writing a followup on it is one of the advantages of having been blogging for nearly five years now. It’s one of those times when blogging is very satisfying, because it allows me to follow the development of of a story over years and comment on developments as they appear.
This latest wrinkle should serve as a warning to those of us who might feel a bit too smug about just how dedicated to SBM modern medicine is. What will physicians do with the most recent information from very recently reported clinical trials that clearly show a very favored and lucrative treatment does not work better than a placebo?
I’m talking vertebroplasty, baby:
Two new studies cast serious doubt on a widely used and expensive treatment for painful fractures in the spine.
The treatment, vertebroplasty, injects an acrylic cement into bones in the spinal column to ease the pain from cracks caused by osteoporosis, the bone-thinning disorder common in older people. Doctors began performing it in this country in the 1990s, patients swore by it — some reporting immediate relief from terrible pain — and it soon caught on, without any rigorous trials to determine whether it really worked.
The new studies are exactly the kind of research that health policy experts and President Obama have been calling for, to find out if the nation is spending its health care dollars wisely, on treatments that work. A bill passed by Congress this year provides $1.1 billion for such so-called comparative effectiveness research.
The studies of vertebroplasty, being published Thursday in The New England Journal of Medicine, found it no better than a placebo. But it remains to be seen whether the findings will change medical practice, because they defy the common wisdom and challenge a popular treatment that many patients and doctors consider the only hope for a very painful condition.
These are the two studies in question, and they were published in last week’s New England Journal of Medicine. Whenever a journal as prestigious and widely read as the NEJM publishes two studies of the same clinical question in the same issue, as companion studies, it’s trying to send a message. This time, the message is loud and clear:
- Kallmes DF, Comstock BA, Heagerty PJ, Turner JA, Wilson DJ, Diamond TH, Edwards R, Gray LA, Stout L, Owen S, Hollingworth W, Ghdoke B, Annesley-Williams DJ, Ralston SH, Jarvik JG (2009). A randomized trial of vertebroplasty for osteoporotic spinal fractures. New Engl. J. Med. 361:569-579.
- Buchbinder R, Osborne RH, Ebeling PR, Wark JD, Mitchell P, Wriedt C, Graves S, Staples MP, Murphy B (2009). A randomized trial of vertebroplasty for painful osteoporotic vertebral fractures. New Engl. J. Med. 361:557-568.
Before I get into the meat of the studies, let’s take a look at what vertebroplasty is. Vertebral fractures in patients with osteoporosis represent a very difficult problem to deal with. The reason is that they can be extremely painful and, worse, very difficult to get to heal using conventional methods. Indeed, the pain can be so severe that powerful narcotics are the only thing that can control it. Given that it can sometimes take months for such fractures to heal (if they heal at all), “conservative” treatment is not very satisfactory. Not surprisingly, a search for treatments that could decrease the disability, pain, and even death from these fractures was imperative. In the 1990s, a procedure known as percutaneous vertebroplasty, which involved the injection of polymethylmethacrylate directly into the vertebral compression fracture under the guidance of fluoroscopy, was first reported and popularized. Initial anecdotal reports suggested a very rapid and effective relief of pain after the procedure. The other aspect of vertebroplasty that led physicians to believe that it could work to relieve pain from compression fractures is that the technique had some degree of prior plausibility. The concept was that the injection of cement into the fracture would provide rapid stabilization of the fracture and therefore rapid relief of pain. It’s never been proven that that is the mechanism, but, according to the principles of prior plausibility, there was at least a reasonable hypothesis as to how vertebroplasty might stabilize fractures and relieve pain.
Now let’s take a look at the introduction of Buchbinder et al:
Observational studies suggest that there is an immediate and sustained reduction in pain after this procedure is performed,5 but data from high-quality randomized, controlled trials are lacking. The best currently available evidence for the efficacy of vertebroplasty comes from one randomized, open trial involving 34 patients and two quasi-experimental, open, controlled, before-after studies that compared vertebroplasty with conservative treatment. Although each study showed an early benefit of vertebroplasty, methodologic weaknesses cast doubt on the findings. In particular, the lack of blinding and the lack of a true sham control raise concern that the observed benefits reflected a placebo response, an effect that may be magnified with an invasive procedure.
Does this sound familiar? For long-time readers of SBM, it should. I’ve written about it before, as have others, in two contexts. First, consider acupuncture. Time and time again, I’ve pointed out how acupuncture studies tend to be positive for early, smaller, less rigorously controlled studies but in larger and better-controlled studies the observed treatment effect disappears. This sort of progression is very typical for interventions that don’t produce an effect that is measurably greater than that of a placebo. Four years ago, I described how vertebroplasty was in the phase where anecdotal evidence and small clinical studies suggested that it was useful. Indeed, I even referred to vertebroplasty as an example of just how hard it can be sometimes to avoid scientific delusions. Before the studies I’m about to discuss the evidence supporting vertebroplasty was actually very similar to the state of evidence for acupuncture in that the best evidence for the efficacy of vertebroplasty were two unblinded trials comparing vertebroplasty with medical management. Not surprisingly, they were positive trials showing a benefit for vertebroplasty over medical management. That so many physicians could convince themselves that an unproven treatment works on the basis of personal observation and small pilot studies simply shows how easy it is to persuade oneself to believe what one wants to believe, no matter how scientific one views oneself.
Cold, hard science has a way of cutting through all that and revealing that we, too, can fall for placebo medicine just as easily as any CAM practitioner.
The first blast of cold, hard science is a multi-institutional study (Kallmer et al) involving five centers in the United States, five centers in the United Kingdom, and one center in Australia, led by David F. Kallmes, M.D. at the Mayo Clinic and Jeffrey G. Jarvik, M.D., M.P.H. at the University of Washington. According to the methods, “sites were selected on the basis of having an established vertebroplasty practice for osteoporotic fractures, an enthusiastic local principal investigator, and an available research coordinator.” The study was designed as a randomized, controlled, study, in which the experimental group underwent standard vertebroplasty, while the control group underwent a sham procedure in which local anaesthesia was infused, and verbal and physical cues, such as pressure on the patient’s back, were given. To complete the sham, the methacrylate monomer normally used to make up the glue was opened to simulate the odor associated with mixing of the cement. However, the vertebroplasty needle, the needle was not placed and the cement was not infused into the frature. Patients with fractures due to bone metastases, bleeding disorders, and a few other criteria were excluded. Overall it was a sound design, but the investigators had trouble recruiting patients to the trial. Indeed, the NYT article from 2005 describes just this problem for Dr. Jarvik’s trial:
In 2002, a group of researchers received a federal grant for a clinical trial that would be the first to rigorously assess vertebroplasty. But their study is faltering.
Patients in severe pain have proved unwilling to enter such a trial, in which they might be randomly assigned to get a placebo, and their doctors have been reluctant to suggest it. In 18 months, the investigators have been able to persuade just three medical centers to recruit patients, and only three patients have enrolled.
Now the investigators are looking for centers overseas, but they agree that the study’s prospects are dim and that its failure would leave critical questions unanswered.
Once a belief has taken hold that a procedure works, it’s sometimes hard to persuade patients and physicians that a randomized study is ethical. Imagine how hard a sell this study would be to a patient at the apogee of pain from a compression fracture. That’s why there was a crossover design, in which patients, after one month, were allowed to crossover to the other group if they did not have adequate pain relief at one month.
In any case, this study was a resoundingly negative study. There were 131 patients enrolled (68 vertebroplasties and 63 simulated procedures), and the two groups were well matched. Both groups noted immediate and comparable improvement in disability and pain scores immediately after the intervention. At one month, there was no statistically significant difference between the control and vertebroplasty groups. The only hint of a possible effect is that there was a trend towards a higher rate of clinically meaningful reduction in pain in the vertebroplasty group, although, quite frankly, this result strikes me as a bit of the old trick of looking at multiple outcomes, trying to find one that comes out statistically significant. Let’s put it this way. This study is about as “positive” as any of the acupuncture studies we’ve dissected.
The second study (Buchbinder et al) was performed in Australia by a group led by Rachelle Buchbinder, Ph.D. It, too, was a multi-institutional study, with a control group that underwent vertebroplasty. The sham procedure was slightly different for this group, and, in my opinion, more rigorous. They underwent the same procedures as those in the vertebroplasty group up to the insertion of the 13-gauge needle, and to simulate vertebroplasty the vertebral body was gently tapped. As was done for Kallmes et al, the cement was prepared so that its smell permeated the room. Unlike Kallmes et al, there was no crossover was permitted in the design of this study. A total of 78 patients were enrolled and 71 completed the 6-month follow-up.
Buchbinder et al reported results that were just as unimpressive as those reported by Kallmes et al. In fact, they were more so, so much so that I will quote the abstract:
Vertebroplasty did not result in a significant advantage in any measured outcome at any time point. There were significant reductions in overall pain in both study groups at each follow-up assessment.
Can you say “placebo”? Sure, I knew you could.
Like Dr. Jarvik, Dr. Buchbinder crunched the data, looking for any outcome that could be considered even trending towards positive. She found none. This was about as negative a study as you can imagine, even more negative than most studies of homeopathy and acupuncture. This study found zero evidence that vertebroplasty is anything more than an elaborate placebo. Like acupuncture and homeopathy, come to think of it, the only difference being that there was some degree of prior scientific plausibility. Prior plausibility made vertebroplasty worth studying; however, these two trials provide emphatic evidence that it does not work, so much so that it surprised the authors, even though the very reason that the study had been done was because of nagging doubts that reported results seemed too good to be true:
Kallmes did the study because he always though the results reported for vertebroplasty seemed too good to be true–everyone got good results no matter how much cement was injected or what technique was used. And the mechanism through which vertebroplasty provided pain relief was a bit of a mystery. He figured if vertebroplasty was as good as promised, it would be easy to prove. He never expected results as stone cold negative as they were.
This is, of course, quite true. If vertebroplasty worked so well, it shouldn’t have taken much of a randomized trial to show it. If there’s one principle in medicine and clinical trials, it’s that dramatic treatment effects are much easier to demonstrate than weak ones. If vertebroplasty was as potent a treatment as advertised, these two trials should have easily shown its benefits. They did not. Indeed, in last week’s NYT article, there were a couple of more revealing quotes from the lead authors of these two studies:
“I’m going to be the most reviled radiologist on the planet,” said Dr. David F. Kallmes, the first author of one of the studies and a professor of radiology at the Mayo Clinic.
Indeed he is. He just provided very strong evidence that a favored cash cow among interventional radiologists is nothing more than an expensive placebo. So did Dr. Buchbinder:
“It does not work,” said Dr. Rachelle Buchbinder, a rheumatologist and epidemiologist at Monash University in Melbourne, Australia, and the leader of the Australian team. Dr. Buchbinder does not perform vertebroplasty and would “absolutely not” recommend it to patients, she said.
Dr. Kallmes, who helped develop vertebroplasty and has been performing it for 15 years, said his team was “shocked at the results.”
Shock is understandable. As had been pointed out in both papers, vertebroplasty had become the standard of care for the treatment of vertebral compression fractures. Medicare and insurance companies had, on the basis of unblinded studies, begun to reimburse for the procedure. Dr. Burbacher put it well:
Despite evidence that is acknowledged to be inadequate as a basis for justifying reimbursement, public institutions have recommended reimbursement for vertebroplasty. A recent position statement from various American radiologic and neurologic surgical societies also recommended funding the procedure.6 These endorsements have resulted in a dramatic increase in the number of vertebroplasties performed. For example, an examination of aggregate fee-for-service data from U.S. Medicare enrollees for the period from 2001 through 2005 showed that the rate of vertebroplasties performed during that time almost doubled, from 45.0 to 86.8 per 100,000 enrollees.14 There are also reports of repeat procedures for unrelieved pain at previously treated vertebral levels16 and of the prophylactic use of vertebroplasty in normal vertebrae that were deemed to be at high risk for fracture.
Not only is the short-term efficacy of vertebroplasty unproven, but there are also several uncontrolled studies suggesting that vertebroplasty may increase the risk of subsequent vertebral fractures, particularly in vertebrae that are adjacent to treated levels, sometimes after cement has leaked into the adjacent disk; controlled studies have shown conflicting results.
In other words, there’s money to be made. A lot of money. Boatloads of money. Combine the financial incentive with “personal anecdotal experience” that suggests to both patients and the physicians who do the procedure that vertebroplasty “works,” giving interventional radiologists the feeling that they’re “doing good while doing well,” and you can see why the procedure’s use has taken off.
Dr. Jensen knows firsthand how powerful such stories can be. In the late 1990’s, when vertebroplasty was new and many doctors were looking askance at it, she gave a talk to a group of doctors in Chicago.
“I could tell by looking at the audience that no one believed me,” she said. When she finished, no one even asked questions.
Finally, a woman in back raised her hand. Her father, she told the group, had severe osteoporosis and had fractured a vertebra. The pain was so severe he needed morphine; that made him demented, landing him in a nursing home.
Then he had vertebroplasty. It had a real Lazarus effect, the woman said: the pain disappeared, the narcotics stopped, and her father could go home.
“That was all it took,” Dr. Jensen said. “Suddenly, people were asking questions. ‘How do we get started?'”
Even after these two studies, there are patients who will not be convinced, as demonstrated in last week’s NYT article:
One patient in the study, Jeanette Offenhauser, 88, said she was convinced that the cement had helped her severe back pain, even after hearing the results.
One thing that people often don’t realize and even physicians sometimes forget is that surgery and invasive procedures provide one of the most powerful placebo effects there is; so it is not surprising that a procedural intervention like vertebroplasty could appear effective at the anecdotal experience level. A lesson in history might be in order here. There was an operation that was popular in the late 1930s through the 1950s for treating angina due to coronary artery disease. It was called pericardial poudrage, and it involved opening the chest and sprinkling sterile talcum powder on the heart. The idea was that the inflammation would cause angiogenesis (the influx of blood vessels into the heart to revascularize it). We know today that the inflammatory reaction thus caused is far too minor to make up for the loss of blood flow from a major coronary artery. Still, patients reported marked improvement of their symptoms. It wasn’t until the 1950s that an actual randomized blinded trial (the patients could be blinded but the surgeons couldn’t) was performed. It found no difference between patients who had poudrage and those who just had their chests opened and closed. (Imagine trying to get that one through the IRB today.) In other words, poudrage was no different than sham surgery.
So it appears to be with vertebroplasty. Indeed, if I may be so bold, these studies should render vertebroplasty dead. To paraphrase what John Cleese would say, “E’s passed on! This procedure is no more! It has ceased to be! It’s expired and gone to meet its maker! It’s a stiff! Bereft of life, it rests in peace! If docs wouldn’t keep nailing it to the perch it’d be pushing up the daisies! Its metabolic processes are now ‘istory! It’ off the twig! ‘E’s kicked the bucket, It’s shuffled off ‘is mortal coil, run down the curtain and joined the bleedin’ choir invisibile!! THIS IS AN EX-PROCEDURE!!”
Sorry. I got a little carried away.
Even doctors are having a hard time accepting these results. Even Dr. Kallmes himself argues that patients who want the procedure should still be able to get it, but only under the auspices of a clinical trial. That’s not entirely unreasonable, but Dr. Burbacher disagrees and says that these studies should be enough to abandon vertebroplasty. In an accompanying editorial, Dr. James Weinstein of the Department of Orthopaedics, Dartmouth Medical School, argued:
President Barack Obama has called for more comparative-effectiveness research as part of the American Recovery and Reinvestment Act. Although clinical trials are an integral part of such research, from a safety and effectiveness standpoint, data from clinical trials combined with those from registries or other large longitudinal databases are necessary to provide the best evidence. Americans prize advances in technology. However, if in major medical challenges, such as osteoporotic vertebral compression fractures, the alternative is to pay the cost of perpetual uncertainty, we need to support the research necessary to provide sufficient efficacy and safety information for patients to make a truly informed choice. Although the trials by Kallmes et al. and Buchbinder et al. provide the best available scientific evidence for an informed choice, it remains to be seen whether there will be a paradigm shift in the treatment of vertebral compression fractures with vertebroplasty or similar procedures.
That is indeed the question, isn’t it? Here’s the challenge. Given these two rather definitive randomized studies showing in essence zero benefit for vertebroplasty over a placebo intervention, what will we as science-based physicians do? Will we, as we should, abandon a procedure against which the evidence has been accumulating, culminating in two large randomized studies that found no real benefit in terms of pain relief? Unlike homeopathy, for instance, vertebroplasty has a complication rate. It’s low, but it’s there. Harm can be done, and these studies suggest there is no benefit to make the risks worthwhile. Or will we behave as CAM advocates behave and refuse to believe because we know from our own experience that it “works”? Will we become acupunturists using fluoroscopy to guide much larger needles into the spine in order to redirect the flow of qi?
What will we do?
Dr. Weinstein, ironically enough, suggests what may well happen:
In an interview, Dr. Weinstein, who does not perform vertebroplasty, suggested that rather than abandoning the procedure, doctors could let patients decide for themselves, by telling them, “This is a treatment option no better than a placebo, but if you want to consider a placebo because you might benefit from it, you might want to know that.”
A $3,000 placebo? If given the choice between a pricey $3,000 placebo and a CAM placebo like acupuncture, which is likely to cost less than that and not have the potential for complications, you know what? I’d pick the acupuncture! If placebo medicine is going to be the wave of the future, at least with homeopathy, acupuncture, and reiki the potential for harm is less than sticking needles into vertebrae and injecting cement. Really, if allegedly science-based practitioners make this sort of argument, then why not just offer CAM to patients as a placebo and get in on the action?
Here’s my prediction. These studies will not be enough to change practice, at least not in the short term. I wish I could say otherwise, but I can’t. A procedure as well embedded into the standard of care and as profitable as vertebroplasty is won’t disappear overnight or without a fight. Less mercenary, vertebral compression fractures are a serious problem for which other treatment options (bed rest, pain killers, and back braces while the fractures heal) are both slow and not palatable to patients or physicians, who want immediate results. However, unlike CAM practitioners, eventually physicians will yield to the weight of the negative evidence. There may be a few more studies, and it’s even possible that there may be found a subgroup of patients who actually do benefit from vertebroplasty, but eventually the procedure will either be abandoned or scaled back to patients who might actually benefit from it. The process may be far messier than we would like. It may take far longer than we would like. It may even take a turnover to a new generation of physicians for the process to be complete. But, make no mistake, science will eventually win out.
If it doesn’t, then we are no better than the homeopaths.