I'm currently in Las Vegas anxiously waiting for The Amazing Meeting to start. Believe it or not, I'll even be on a panel! While I'm gone, I'll probably manage to do a new post or two, but, in the meantime, while I'm away communing with fellow skeptics at TAM7, I'll be reposting some Classic Insolence from the month of July in years past. (After all, if you haven't been following this blog at least a year, it'll be new to you. And if you have I hope you enjoy it again.) This particular post first appeared in July 2008.
Last week, The New York Times started a rather unusual series in its medical section entitled, The Evidence Gap, described thusly:
Articles in this series will explore medical treatments used despite scant proof they work and will consider steps toward medicine based on evidence.
When I first saw the series, I was prepared for a crapfest. My experience has generally been that when reporters start examining the evidence for and against a treatment they usually do a pretty lousy job. This is most obvious when it comes to "complementary and alternative medicine" (CAM), where we are routinely treated to utterly credulous articles touting the latest poorly designed study demonstrating that this or that fairy dust cures cancer and the common cold (I exaggerate, but not by much) or "exposes" on the latest misdeeds by big pharma, in which risks are either exaggerated or not put into perspective. Oddly enough (or not so oddly enough) seldom does the misinformation go in the other direction, with excessive credulity towards pharma-produced evidence and excessive skepticism for evidence relating to the efficacy and safety of CAM modalities, but that's a topic for another day. I don't so much mind the tighter skepticism regarding the claims of big pharma, given its history; I only wish that similar skepticism applied to the claims of big CAM. In any case, the point is that generally medical reporting tends to be very poor at doing a fair, objective analysis of the state of the actual evidence regarding a treatment modality, so much so that I sit up and take notice when I find an exception.
Depending on how the the rest of the articles in the series play out, this New York Times series may very well be one of those exceptions. As a maven of science- and evidence-based medicine, I approve--thus far, anyway. The first two entries are on cardiovascular testing and cancer chemotherapy. I'll spend the most time on the latter, but I can't resist a brief treatment of the former before I get to the latter.
The first installment in the series was Weighing the Costs of a CT Scan's Look Inside the Heart, which provided a mostly balanced look at the latest bandwagon fad among cardiologists, CT angiography. Unfortunately, the article gave Dr. Harvey Hecht of the Lenox Hill Heart and Vascular Institute of New York license to say some spectacularly stupid things, like:
"It's incumbent on the community to dispense with the need for evidence-based medicine," he said. "Thousands of people are dying unnecessarily."
Bullshit. More like there are hundreds of ideologue doctors with a financial incentive that distorts their objectivity convinced that their shiny, expensive new machines are the latest and greatest--doctors like Dr. Hecht, who has, as far as I can tell, not produced one whit of evidence that using his device to guide therapy saves even a single life.
Fortunately, the NYT actually did a pretty good job of pointing out the price and lack of evidence supporting the widespread use of these scans for screening. When used indiscriminately, they also could risk leading to more unnecessary angioplasties and coronary artery stenting in much the same way that ever more sensitive tests for cancer can lead to overtreatment. Unfortunately, Dr. Hecht appears no better than woo-bosters Dr. David Katz, who wants a "more fluid concept of evidence" supporting CAM or Dr. Andrew Weil, who wants to emphasize "uncontrolled clinical observations" (i.e., anecdotes) in evaluating evidence. The only thing different is what kind of modality he's trying to change the rules to support.
Cardiologist blogger Dr. Wes, as well as medical bloggers Matthew Holt, Dr. Ray Poses, and Dr. Guy Clifton have already ripped Dr. Hecht a much-deserved new one for his idiocy. I also happen to know that Dr. Harriet Hall over at Science-based Medicine is planning an installment for tomorrow. The issue is well-covered, even by medical students, and I've also touched on many of the same issues before the last time this sort of technology was overhyped. From my perspective, the only shortcoming of the NYT article is that it didn't adequately discuss the question of what the best science-based recommendations are for when a CT-angiogram is appropriate.
More interesting and relevant to me is the second article in the series, Costly Cancer Drug Offers Hope, but Also a Dilemma. It's right up my alley in that (1) it's about cancer and (2) it's about my area of research interest, angiogenesis inhibitors. Specifically, it's about one angiogenesis inhibitor in particular, namely Avastin.
Avastin is the trade name for bevicuzimab, which is a humanized monoclonal antibody against a protein called vascular endothelial growth factor (VEGF). A "humanized" antibody is an antibody raised in mouse cells the nonspecific or invariable part of which (in other words, the part that doesn't recognize and bind to the target antigen) have been replaced with its human counterpart in order to reduce the likelihood of an immune reaction against a foreign protein. VEGF is a protein that is secreted by both normal cells and cancer cells in order to stimulate the ingrowth of new blood vessels, a process known as angiogenesis. In the case of normal cells, VEGF is only secreted in response to injury or a physiologic need for more nutrients and oxygen. Tumors, on the other hand, hijack this normal physiological function to their own nefarious ends, often by cranking out huge quantities of VEGF or other angiogenesis-stimulating proteins to feed their ravenous glucose and oxygen requirements. Targeting tumor angiogenesis as a means of "starving" tumors has been an area of my research interest for over ten years now, and it is the area of research pioneered by one of my scientific and surgical heroes, the late Dr. Judah Folkman, who demonstrated the feasibility and effectiveness of antiangiogenesis as a therapeutic strategy against cancer.
Antiangiogenic therapy is attractive because, in theory at least, it could target almost any solid tumor. In addition, in the mouse and, as we now know, in humans it is a therapy that generally produces low toxicity, at least compared to chemotherapy. Hypertension, edema, poor wound healing, and, paradoxically, bleeding appear to be the most common side effects. Another less common but very serious side effect is gastrointestinal perforation, thought to be due to compromise of the blood supply to an area in the GI tract. Still, all in all, the drug has a favorable side effect profile compared to many cytotoxic chemotherapy regimens, and it's been shown to be effective in prolonging life for certain metastatic tumors, particularly when combined with chemotherapy, a phenomenon I've described before.
The problem, as the NYT article points out, however, is that Avastin is very, very expensive. It can cost as much as $100,000 a year, depending on the specific tumor, which influences the dose regimen necessary. It's here, as the article describes, where a question that we as Americans find far more uncomfortable than perhaps many other nations with nationalized health care systems do, rears its ugly head: How much are an additional few months of life worth?
Antiangiogenic therapy was all the rage 10 years ago in the research realm. Dr. Folkman's dramatic results in mice using angiostatin and endostatin fueled hype in the mainstream media that a "cure" for cancer was at hand or that, at least, Dr. Folkman's dream of turning cancer into a chronic, manageable disease like diabetes was finally being realized. Avastin was the first of the new generation of antiangiogenic drugs to make it into widespread clinical testing, thanks to the efforts and R&D money of Genentech. Unfortunately, its results when used alone against various cancers were disappointing to nonexistent. Fortunately, against metastatic colorectal cancer, combining Avastin with standard chemotherapy prolonged median survival from 15.6 months to 20.3 months. While chemophobes (as I like to call those who can't ever admit that chemotherapy does any good) would scoff, I'd point out that that represents a 30% increase in survival, generally with good quality of life.
However, as the article points out, newer chemotherapy regimens seem to be more effective on their own and less synergistic when combined with Avastin:
In February 2004, 15 years after Dr. Ferrara's initial discovery, the Food and Drug Administration approved Avastin for patients with advanced colon cancer. A blockbuster was born.
But now there is a question mark over that evidence. That first exciting result compared Avastin with a type of chemotherapy that has since been widely replaced by a more effective regimen.
In a later, larger study comparing Avastin with current chemotherapy, Avastin slowed the growth of tumors but did not extend life by an amount considered statistically significant.
Although GI cancers are not my specialty, I do know enough to say that the therapy of metastatic colorectal cancer and the evidence supporting it is in a state of flux, and this may indeed be the "true" case rather than a fluke in a study. Still, the evidence for the efficacy of Avastin is probably the strongest for colorectal cancer, both in terms of magnitude of the benefit in terms of the increase in life expectancy and quality of evidence. On less compelling evidence, Avastin was also approved for lung cancer, as well.
Perhaps the approval for Avastin that caused the most controversy was one that occurred earlier this year. I really should have written about it at the time, but for some reason I never did. Here's what how the article describes the approval:
The third approval for Avastin, for advanced breast cancer, came in February of this year. The clinical trial found it significantly slowed the progression of cancer but did not significantly extend life. The F.D.A. went against its own panel of outside experts, who had voted 5 to 4 against approval.
The agency's action has not sat well. Senator Charles E. Grassley, Republican of Iowa, asked the Government Accountability Office to look into the F.D.A.'s approval of Avastin and some other drugs that "appear to have little to no effect in protecting lives and increasing health."
Dr. Lee Newcomer, an oncologist and executive at the insurer United HealthCare, said patients were not well served, and neither were insurers, nor the public, which ultimately foots the bill. If a drug just stops tumor progression, without the woman's living longer or feeling better, without her noticing anything different, Dr. Newcomer said, "you're treating an X-ray."
I must admit that I was even skeptical of this approval. Basically, what is being discussed here is whether a drug affects overall survival (OS), which is mortality from all causes in a cancer patient, versus whether it affects progression-free survival (PFS), which is the period of time before a given tumor progresses. Surprisingly, at least to non-oncologists and lay people, OS and PFS are often unrelated. If, for example, a drug slows tumor growth sufficiently to demonstrate a significant affect on PFS, it doesn't necessarily mean that OS will be better too. Sometimes it will, sometimes it won't. Moreover, it's long been a debate over whether PFS is a valid endpoint for approving a drug. Traditionally, the thinking has been that if a drug does not improve OS, then it probably shouldn't be approved as a first line agent given up front to new cancer patients who have not been treated yet, although it can be approved as a second-line or third-line agent, to be tried after first line agents fail.
However, thinking has been evolving over the last few years towards accepting a somewhat looser standard of valuing PFS. Dr. Kathy Albain of Loyola University Medical Center and the Cardinal Bernardin Cancer Center, a passionate defender of using using PFS as an endpoint, makes the argument this way:
"Assuming that there is a real treatment benefit for whatever agent you are testing, I believe that PFS should be accepted as a proper surrogate," Albain maintained. She pointed out that in metastatic breast cancer in particular, it may never be possible to prove surrogacy for overall survival, given the thousands of patients that would be needed.
Says Dr. David Miles, lead investigator on the AVADO study on Avastin and breast cancer:
"As a practicing physician," AVADO lead investigator David Miles said he views PFS as a reflection of the period of time where the disease is controlled and the patient remains relatively well. There is an expectation that the patient is "doing better than if their disease is not controlled," he told the press briefing.
"We have studies where PFS does reflect overall survival and studies where it doesn't," Miles observed. "I think when you have a patient whose disease is well-controlled, there is benefit to that, there is utility to that. Particularly when studies are not powered for overall survival." None of the Avastin breast cancer studies have been designed for a full survival analysis.
Indeed, Dr. Albain polled oncologists about whether they accept PFS as a valid endpoint, and this is what she found:
Albain took it upon herself to conduct a survey, sending out a questionnaire to a sample of 47 experts. She was overwhelmed by the volume and vehemence of the replies. The clear consensus was that PFS should be considered a meaningful approval endpoint. Getting to "the crux of the matter," Albain's group suggested PFS should be used in Phase III trials, but maybe it should be prospectively determined how large a benefit is desired. For full approval perhaps two trials should be required, or perhaps there should be a mandatory cross-over design.
In a world where cost is no option, I'd agree. However, resources are not endless, and one has to ask how much a few months of PFS without a concomitant increase in OS are worth. I don't know the answer to that one, as this is a very difficult debate that we in the U.S. have thus far been able to avoid. Our colleagues in nations with nationalized health care systems cannot avoid it, though. For example, in Canada, it's been estimated that the addition of Avastin to breast cancer and lung cancer treatment would add $299 million a year to Canada's health care costs. In a country like Canada, the only choices are to cut money out of other programs to pay for this or to raise taxes. In the U.S., our insurance premiums just go up.
These issues are difficult, and the NYT article gives a fairly balanced overview up until this point. However, when it comes to discussing off-label use of Avastin, the bioethical and financial issues become harder still:
Doctors are free to prescribe Avastin, or any other drug on the market, for unapproved uses, at their discretion. As much as 75 percent of cancer drug use is of this "off label" variety, according to an estimate by the National Comprehensive Cancer Network, a group of big cancer centers. And some doctors say that with patients dying, they simply cannot wait for airtight evidence.
"Of course we want everything to be evidence-based," said Dr. Yashar Hirshaut, an oncologist in Manhattan. "I also like the American flag and apple pie."
But, he explained, "You say, 'This person is dying right here and I need something that will help, and there's a logical construct that I can see how it will help.' "
I must point out that in this case "off-label" use does not usually mean non-evidence-based. What it usually means is that the evidence supporting the treatment choice is weaker or that the clinical trials haven't been done yet to show if a therapy works or not. Even the most die-hard advocate of EBM understands that if you have a patient in front of you dying right now, you're willing to try something for which the evidence may not be particularly strong. That's why I'm not making sarcastic comments about Dr. Hirshaut the way I did about Dr. Hecht; the two situations are apples and oranges.
The problems, however, are two-fold: First, there's a financial incentive on the part of the drug company to encourage such off-label use, or at least not to object to it. Second, off label use makes doing the correct clinical trial difficult, and the NYT article describes one way how:
But the anecdotes and evidence from small trials that may seem to justify off-label use sometimes turn out to be misleading. That happened with pancreatic cancer. After patients and doctors decided Avastin had to be helping, cancer researchers themselves conducted a large study. So did Roche. Avastin, both studies concluded, did not prolong life for people with cancer of the pancreas.
The moral of the story above? Anecdotes can deceive "conventional" physicians as much as they can CAM practitioners. Never forget that. This is in reality not much different than the issues I discussed about whether or not patients have a right to experimental therapy. The only difference is that it's much easier to prescribe drugs off-label than it is to prescribe experimental therapy.
One thing that bothers me about this trend is indeed the potential harm it can do to science- and evidence-based medicine. One can make an argument that perhaps PFS is a valid endpoint for approving a cancer drug (if we're willing to pay for it), but it's hard for me to accept arguments like this:
As word spread, Dr. Friedman at Duke and Genentech organized studies of a type generally considered less than definitive. There was no control group that took another drug or got a placebo. Everyone got Avastin. Otherwise, no one would enroll in the study, doctors argued.
Then the investigators compared the results with what they thought would have happened without Avastin. The patients lived a median of about nine months, about three months longer than the researchers estimate would have been expected.
But such comparisons have led scientists seriously astray in the past because the people being treated with a new drug often are very different from previous patients who did not take it and because overall medical care steadily improves. Nonetheless, Genentech has said it planned to apply this year to the F.D.A. for approval for Avastin to treat glioblastoma, the deadliest form of brain cancer.
Single-arm studies like this using historical controls can sometimes be used as valid evidence, but the difference between the treated patients and historical controls have to be much, much more impressive, to the point where nothing other than a major effect from the drug is a reasonable explanation for the improvement in survival. After all, selection bias alone could potentially account for a 3 month survival benefit compared to historical controls. Trying to argue for approval of a drug on the basis of such thin gruel isn't in essence any different than Andrew Weil or David Katz, either.
With complex medical technology and expensive drugs whose benefits seem modest in relationship to their price, our health care system appears to be heading for a crisis. We in the U.S. absolutely hate to address real world questions such as whether how much these drugs cost is worth their benefit. A cancer patient will often answer a resounding yes. We'd all like to say yes to that question as well. Unfortunately, such idealism comes crashing against fiscal reality, and it's not clear where we will draw the line and to which patients we will end up having to deny potentially life-prolonging treatments because they are just too expensive.
The preamble to this post has a typo, "While I'm gone, I'll probably manage to do a new post or twoy..." and I have to admit, for a good 5 seconds I was sitting here thinking "twoy" was some kind of Twitter-related neologism.
"Orac did a wonderful twoy the other day!"
You must be new around here. Orac hates it when people point out obvious typos.
Re: Disinfo and Marilyn
I agree with Marilyn about showing restraint in pointing out obvious typos (unless they lead to some real uncertainty in what is actually meant by the writer).
However, I suggest that occasionally something trivial like a typo or merely an odd juxtaposition may tweak our sense of humor, and pointing that out may similarly amuse others and perhaps even encourage our own sense of humor.
(Yesterday, while driving around, I saw a sign for a clinic named "Chiropractic Art". Does that mean it is not science?)
I can even see where calling something a oney or a twoy might even be convenient in text messaging or casual speech. In a few months, we may be watching the Winter Olympics and wondering if a particularly good skating performance was a sixy. Given modern performance techniques and standards, it might even be a sexy sixy. (Sorry, I couldn't resist sticking that one in there.)
I also suggest that developing a sense of humor about our own mistakes and shortcomings may make it easier for us to accept them and less defensive about them. I know from my own limited research the importance of being absolutely clear about what the real data are (everything else is built on them). Also, I know that sometimes things don't turn out as you expected or planned and then you have you have to figure out if there was a real reason why that happened or you just made a stupid mistake or maybe it was just in the random noise of the data and you really don't know.
My final suggestion is that being defensive about our data and results may lead to more attempts to manufacture a cause where none exists and hence to crankism and woo.
Sorry if this turned out more like a blog-post and less a comment.
Thanks for accepting my post and my complements to the people who run the service that supports your blog.
Perhaps for the benefit of other new posters, I offer the following note.
When attempting to post my comment, I clicked "Post" but nothing seemed to happen, so I clicked it again.
When nothing still happened, I went for insurance and copied the post to an e-mail to my wife in case it somehow vanished into the "cloud".
However, when I went back to the tab for Respectful Insolence, I saw a new window which suggested the post had merely been delayed and recommended clicking the back button on my browser and then refreshing the window.
It worked! And, my post was accepted. Thanks, again.
Regarding PFS, I agree we have to look at and decide on higher standards for the magnitude of it, not just accept small positive differences, which was touched on in your excellent piece, but maybe not given enough emphasis.
Regarding off-label use, I was told by a drug company representative that it's not done in Europe, which provides an incentive for the drug company to do the clinical research.
Anyhow, given the avalanche of drug candidates emerging based on discovery of disease pathways, and finite/shrinking public resources, I think we need to raise the bar, but I don't see a way to avoid using PFS, particularly for the indolent lymphomas, where measuring survival benefit is confounded by use of many subsequent therapies.
I can even see where calling something a oney or a twoy might even be convenient in text messaging or casual speech.
Reminds me of Bill Cosby. "She doesn't need to know if I'm onein' it or twoin' it!"
Re: Marilyn Mann
Apologies then; didn't mean to be pedantic, I just thought it was funny.
I hope you won't be offended if I use this opportunity to plug my own blog? This may not be the right place to do so, as anyone who understood this post may not be interested, but I've started a series of posts aimed at explaining the important principles in cancer biology to the layman. My first post deals with the concept of cancer as a feature of evolution in action, and Hanahan/Weinberg's Hallmarks of Cancer ideas, which are, to my mind, the most fundamental things to understand. If you like it, there's lots more to come
Hope it's ok to drop by and post to suggest that Dr. Weil's web site and Dr. Weil answers his questions with evidence based science--study after study and one will find little or no "woo" factor. Thanks!
Now there's a joke-- little or no woo factor at Dr Weil's site!! Guess again
Now there's a joke-- little or no woo factor at Dr Weil's site!! Guess again
Karl Schwartz, what do you mean by "no off-label use in Europe"? Do you mean it's impossible (which would be stupid and cruel, but I'm used to our insane laws, so it wouldn't surprise me at all) or just not as common place? I suppose a lot of it is simply not covered by insurance?
I can't make much sense of the German wikipedia article. It seems it could be actively discouraged, because of liability issues. Would be really disconcerting if terminally ill patients could not get off-label perscriptions.
Kismet wrote: what do you mean by "no off-label use in Europe"? Do you mean it's impossible (which would be stupid and cruel, but I'm used to our insane laws, so it wouldn't surprise me at all) or just not as common place? I suppose a lot of it is simply not covered by insurance?
I was noting only the comments of a representive of a drug company on the subject, and don't have experence with European subscribing policies to offer more on it. Probably it varies also by country.
My assumption (from said comments) is that it's used less often in Europe than in the US - to a degree that provides more incentives for sponsors to do the studies in Europe, which is the good side of such policy, because sometimes you do more harm than good when giving drugs that are not sufficiently studied.
But I agree with you that there can be circumstances where off-label use is appropriate ... would be difficult or unethical to withhold ... such as for orphan conditions, or when a cancer is refractory to standard treatment and the use of an new agent is rationally determined, such as by mechanism of action ... A lot depends on the details of the case.