The truth? You can't handle The Truth (when it comes to mammographic screening)!

Occasionally, there are topics that my readers want -- nay, demand -- that I cover. The topic of this post, it turns out, is one of them. It's a link to a TED Talk. I'm guessing that most of our readers have either viewed (or at least heard of) TED talks. Typically, they are 20-minute talks, with few or no slides, by various experts and thought leaders. Many of them are quite good, although as the TED phenomenon has grown I've noticed that, not unexpectedly, the quality of TED Talks has become much more uneven than it once was. Be that as it may, beginning shortly after it was posted, readers of both this blog and my other super-not-so-secret other blog started peppering me with links to a recent TED Talk by Dr. Deborah Rhodes at the Mayo Clinic entitled A tool that finds 3x more breast tumors, and why it's not available to you.

At first, I resisted. In fact, I resisted so much that this talk is nearly a month old now. Moreover, to put it bluntly, there really isn't anything radically new in Dr. Rhodes' talk, at least not to anyone who's been in the field of breast cancer for a while. Certainly, there's no new conceptual breakthrough in breast imaging and screening described. As I will discuss in more depth later in this post, there's an interesting application of newer, smaller, and more sensitive detectors with a much better spatial resolution. It's cool technology applied to an old problem in breast cancer, but something radical, new, or ground-breaking? Not so much. What Dr. Rhodes describes in her talk is the sort of device that, when I read about it in a medical journal, produces a reaction along the lines of, "Nice technology. Not ready for prime time. I hope it works out for them, though. Could be good." So it was with molecular breast imaging (MBI), which is the topic of Dr. Rhodes' talk. So I continued to resist for about two or three weeks.

Eventually, as you can tell, I gave in. The reason I gave in is that the way Dr. Rhodes tried to sell the audience on the idea of her technology brings up a lot of issues important to our audience and a lot of issues related to discussing science with the lay public, not to mention observations that irritate the crap out of me. I also thought it was important to put this technology in perspective. So here I go. First, I'll start by describing what really set my teeth on edge about Dr. Rhodes' talk. Then I'll go to the primary literature (namely her brand, spankin' new article in Radiology describing the technology) and discuss the technique itself.

The truth? You can't handle the truth!

What irritates me about Dr. Rhodes' TED Talk starts right at the beginning:

There are two groups of women when it comes to screening mammography -- women in whom mammography works very well and has saved thousands of lives and women in whom it doesn't work well at all. Do you know which group you're in? If you don't, you're not alone. Because the breast has become a very political organ. The truth has become lost in all the rhetoric coming from the press, politicians, radiologists and medical imaging companies. I will do my best this morning to tell you what I think is the truth. But first, my disclosures. I am not a breast cancer survivor. I'm not a radiologist. I don't have any patents, and I've never received any money from a medical imaging company. And I am not seeking your vote.

Later in the talk, Dr. Rhodes says:

If this technology is widely adopted, I will not benefit financially in any way. And that is very important to me, because it allows me to continue to tell you the truth.

I bet you can guess what irritates me about these statements. Actually, it's two things. First, it's Dr. Rhodes' invocation of "The Truth." Strictly speaking, there is no "truth" in science or medicine. There are hypotheses that are supported by evidence, experimentation, and, in medicine, clinical trials, and there are hypotheses that are not. Most scientific hypotheses are not black and white "true" or "false," either. Rather, individual hypotheses fall somewhere closer to being true or false, based on the evidence, and they can move closer to or father away from being "true" as new evidence comes in. Indeed, the object of scientific investigation is to falsify hypotheses. Hypotheses that are easily falsified fall by the wayside quickly. Those that are not advance to more intense testing. Those that have withstood the most attempts to falsify them and provide highly useful explanatory and predictive value might eventually graduate to being full-fledged theories.

The next thing that irritates me is Dr. Rhodes' implication that everyone else has an ax to grind (and is therefore probably lying to you), where as she does not because she doesn't receive money from medical imaging companies, isn't a breast cancer survivor, and is not a radiologist. Don't get me wrong; financial conflicts of interest (COIs), particularly undisclosed ones, are very, very important to know about, because they can (and all too often do) warp the perspective of even the most diligent, honest, rigorous scientist. However, financial COIs are not the only COIs. Just because someone proclaims that she has no financial COIs (or professional COIs) does not mean that she does not have biases or COIs that can be just as strong as the financial COIs of scientists who stand to make a lot of money if their research results in a marketable drug, treatment, or medical device.

Before I examine Dr. Rhodes' invention (and her claims for it) in a bit more detail, in the interests of full disclosure, I'll point out that I actually have a non-financial interest in a competing imaging technology for the breast. Researchers at our cancer institute have developed what I consider to be a truly innovative and promising breast imaging device. It's based on ultrasound and can produce images of the breast almost as striking as those produced by breast MRI. This device has even resulted in a startup company that was featured in our governor's state of the state address last week. As you can see, it's a big deal to our cancer center. Since, through some fluke of flukes, I've somehow managed to find myself in leadership positions within the clinical and research breast cancer programs at my cancer center (obviously, the cancer center director hasn't realized his mistake yet), I have a stake in the success of this device. More importantly, not only do we see the chance to have a major positive impact on women's health if this device is validated but it would bring all sorts of prominence to our institution in general and my programs in particular, just as the success of Dr. Rhodes' MBI device would improve women's breast screening and bring all sorts of glory to the Mayo Clinic and her programs. None of this means that either Dr. Rhodes or I are likely to be lying or stretching the truth, but we both have COIs based on our belief in our respective devices. Arguably, my COI is less intense, because this device had been developed before I accepted my current job, and I am not directly involved in its commercialization. A COI does, however, exist nonetheless, and I acknowledge it.

Another thing that bugs me about Dr. Rhodes' talk is her implication that the radiology world is somehow closing ranks to keep her from bringing this technology to the masses. While she does have a point that some radiologists were shamelessly disgusting in protecting their turf and launching what can only be called histrionic attacks on the new guidelines (the quote by Dr. Daniel Kopans, a very prominent breast imaging radiologist at Harvard, about the USPSTF guidelines that Dr. Rhodes cites at 2:49 in the video being an excellent example), it's a bit of a stretch to claim that somehow radiologists are so biased against her technology that they won't give her a fair shake. Before I explain, I mention a couple of points that Dr. Rhodes makes that are correct. First, Dr. Rhodes is correct that breast density appears to be an independent risk factor for breast cancer that has only relatively recently been appreciated as such. Unfortunately, it is in dense breasts where mammography has the biggest problem in detecting cancer. Indeed, that's part of the reason why it's not as good in women under 50; their breasts tend to be denser. Second, she is most likely correct that digital mammography is probably not more sensitive or specific for detecting breast cancer, particularly in women with dense breasts, although I will point out that the evidence is not as cut and dried for this assertion as Dr. Rhodes makes it out to be.

Dr. Rhodes also fails to mention that digital mammography does have some major advantages over conventional film mammography and that they are not inconsequential advantages, either. These include permanent storage and duplication of as many copies of a study as needed. In other words, if a woman goes for a second opinion, instead of taking a jacket full of films (what I was used to until relatively recently), she can take a CD and provide an exact copy of her suspicious mammogram to the consulting surgeon. In the old days, she would have to sign out her jacket or bring an inferior film copy. Lost films are a thing of the past, and instead of huge rooms full of large manila envelopes stuffed with X-ray films, the studies can now be stored on hard drives and backed up off site. Digital mammography also allows for the digital manipulation of the image, not to mention the development of image analysis algorithms that can assist the radiologist in detecting suspicious lesions. Finally, digital mammography probably requires less radiation, particularly in women with dense breasts, the very group in whom Dr. Rhodes is trying to increase sensitivity and specificity of breast cancer detection, although this benefit hasn't been fully verified yet.

Now, I wouldn't be all that surprised if there was some skepticism over her idea. I'll explain why in a bit more detail in the next section. However, I really do think Dr. Rhodes goes overboard in implying in at least two parts of her talk that the radiology world and the mammography world are somehow conspiring (or at least so resistant to new ideas that the practical effect is the same as conspiring) to prevent new breast imaging technologies (like hers) from gaining a foothold. Perhaps the most egregious example is this passage, where she talks about having submitted her article to four different journals and having it rejected by each one:

After achieving what we felt were remarkable results, our manuscript was rejected by four journals. After the fourth rejection, we requested reconsideration of the manuscript, because we strongly suspected one of the reviewers who had rejected it had a financial conflict of interest in a competing technology. Our manuscript was then accepted and will be published later this month in the journal Radiology.

At least she didn't mention Galileo or Ignaz Semmelweis. I'm grateful for small favors.

In any case, this is a highly explosive charge to make so casually, without describing the evidence that led the authors to suspect that one of the reviewers had a financial COI in a competing technology. In fact, if three journals rejected her manuscript before Radiology apparently rejected it and then reconsidered it, did it ever occur to her that perhaps her manuscript just wasn't very novel? Actually, the manuscript that ultimately was published in Radiology was pretty decent, but there are other reasons that papers, sometimes even good papers, have trouble being published. For instance, she didn't say which journals she tried first before settling on Radiology. For instance, if she had tried the New England Journal of Medicine, The Lancet, and the Journal of the American Medical Association (JAMA), the reason her manuscript was rejected would be fairly obvious. It's an interesting new technology with promising preliminary results, but not interesting enough to a broad enough audience to be likely to be published in such high impact journals. None of this implies that Dr. Rhodes' work isn't solid work, but the reluctance of journals to publish her results doesn't suggest that MBI is a technology "they" don't want you to know about, either.

Everything old is new again

There is no doubt that Dr. Rhodes is charming and a very effective advocate for her preferred breast imaging technology. Nowhere is that more clear than the middle portion of her talk, where she describes the genesis of the MBI. Who wouldn't be moved at her description of a friend who found a lump in her breast while she was pregnant (and Dr. Rhodes was pregnant as well), the fear it engendered, and how that event inspired Dr. Rhodes to wonder if there was a better way to detect breast cancer? Who didn't find her account of serendipitously bumping into physicists, who told her about a new kind of gamma detector that was much smaller than previous generations of such devices, compelling? Who wasn't appreciative of her description of the first device the medical physicists and Dr. Rhodes cobbled together with duct tape and those early primitive tests of its ability to detect radiotracer concentration in the breast? Certainly not me. The image of a bunch of brand new tiny gamma detectors cobbled together with duct tape was priceless. Nor do I in any way want to detract from the hard work and development that went into her MBI device. I do, however, have a small problem with how Dr. Rhodes discussed it in her TED Talk.

That problem is that scanning the breast with radiotracers as a means of looking for breast cancer is not a new technology at all. It's been around for at least three or four decades. Go back to the 1960s, in fact, and it's not difficult to find references to the detection of breast cancer using various injected radioisotopes. Since then, at various times such techniques been called scintimammography or sestamibi breast imaging, and they're all based on the same concept as many nuclear medicine imaging modalities: Inject a small amount of radiotracer that is differentially taken up (or not taken up) by the cell of interest or by cells exhibiting the disease process of interest (i.e., cancerous cells), and then take pictures. Positron emission tomography (PET) scans work this way. So do MUGA scans and bone scans. About 20 years ago, ^99mTechnetium sestamibi became the most commonly used radioisotope for breast cancer detection in the breast (as opposed to looking for metastatic disease).

Indeed, if you read Dr. Rhodes' recently published study in Radiology, you'll quickly see that what Dr. Rhodes and her team are doing is nothing more than sestamibi breast scanning. Specifically, she is using technetium (^99mTc) sestamibi scanning combined with mammography. Currently, only one ^99mTc sestamibi compound, Miraluma, which is manufactured by DuPont Pharmaceuticals, is FDA-approved for breast imaging in the United States, which is why such scans are sometimes referred to as Miraluma scans. The same isotope is sold for cardiac imaging under a different name (Cardiolite). The problem with the various nuclear medicine breast scans for screening purposes has always boiled down to an unacceptable lack of specificity and sensitivity. That's why I have to wonder if the reason for the skepticism that greeted Dr. Rhodes' results isn't at least in part due to a collective shrug of the shoulders, as reviewers thought, "Been there, done that." Certainly, fair or unfair, that probably would have been my initial reaction if this paper had come across my desk.

Not that it's a bad study or a technology without promise. Neither is true. In fact, as a proof-of-principle study it's perfectly acceptable, and that's all it's billed as in the paper. The findings of the paper are summarized quite well in this figure (click to enlarge):

As you can see, the study started out with 1,007 women with heterogeneously dense breasts enrolled and ended up with 936 women who completed all imaging whose data were available for analysis and whose cancer status was verified. Positive cancer status was defined as a positive biopsy showing cancer, while negative cancer status was defined as women who had a subsequent negative mammogram in the following year, a negative biopsy, or a negative prophylactic mastectomy. Reported sensitivities were 27% for mammography alone, 82% for gamma imaging (MBI) alone, and 91% for a combination of mammography and gamma imaging. The corresponding specificities were 91%, 93%, and 85%. All in all, these are good numbers, particularly when compared to mammography alone in women with dense breasts. Diagnostic yield was 3.2 per 1,000 for mammography alone, 9.6 per 1,000 for gamma imaging alone, and 10.7 per 1,000 for both. Finally, the positive predictive value (the chance of having cancer if the test is abnormal) was 3% for mammography, 12% for gamma imaging, and 8% for both. Basically, the study suggests that the addition of MBI with ^99mTc sestamibi can increase the sensitivity and specificity of breast cancer detection in women with dense breasts.

Still, it has to be pointed out that there were only 11 women with breast cancer in the entire population. This is lower than one would normally expect for a typical study of mammography, likely because of the large number of younger women (as young as 25) in the population studied. The reason for this was in order to include a lot of women with dense breasts. Unfortunately, this means that it doesn't take very much to skew the numbers one way or another. What this implies is that a much larger study is very much indicated in order to get a more precise estimate of what the sensitivity and specificity of this test is and how much it really adds to mammography. My guess is that it will add something to the mammographic screening of dense breasts, but probably not as much as this initial study suggests. Sadly, the decline effect will likely rear its ugly head.

There is another consideration here, namely the question of radiation. Dr. Rhodes touches on this issue in her talk:

So now that we knew that this technology could find three times more tumors in a dense breast, we had to solve one very important problem. We had to figure out how to lower the radiation dose. And we have spent the last three years making modifications to every aspect of the imaging system to allow this. And I'm very happy to report that we're now using a dose of radiation that is equivalent to the effective dose from one digital mammogram. And at this low dose, we're continuing this screening study, and this image from three weeks ago in a 67 year-old woman shows a normal digital mammogram, but an MBI image showing an uptake that proved to be a large cancer. So this is not just young women that it's benefiting. It's also older women with dense tissue. And we're now routinely using one-fifth the radiation dose that's used in any other type of gamma technology.

When I heard this, I wondered something. What is the effective total body dose? Injecting a radioisotope is a different thing than aiming an X-ray beam at the breast. ^99mTc has a half-life of around six hours, which means that it takes 24 hours for the radiation levels to fall to 1/16 of the original. In this study, Dr. Rhodes administered ^99mTc equivalent to 20 mCi. According to the chart included in this drug information, the estimated radiation dosimetry for a dose of 30 mCi of this tracer is 0.2 rads (approximately 0.2 cGy) to the breast, meaning that the dose used in this study was approximately 0.13 rads. Dr. Rhodes is correct that this is approximately the same amount of radiation as a single mammogram administers. However, she leaves out consideration of the dose of radiation to which other organs are exposed. For instance, 20 mCi of Miraluma results in a dose of approximately 2 rads to the small intestinal wall, 2.6 to 3.6 rads to the wall of the large intestine, 1.0 rads to the ovaries, and 0.3 rads to the bone marrow. In a single dose, these doses are not very high, but remember that we are talking about a screening test that is meant to be administered repeatedly, possibly even yearly, to women with dense breasts. Over the course of 30 years (or even more) of screening, the radiation dose to tissues other than the breast could rapidly add up, in addition to also adding to the dose of radiation to the breasts. These are not trivial concerns, particularly the potential for a significant cumulative whole body dose of ionizing radiation over decades of screening.

The return of the revenge of the Will Rogers effect

Finally, I hate to be a spoilsport, but some of the images that Dr. Rhodes displayed to me did not impress me that much. For example, she showed an image where a mass was not seen on mammography but showed up on MBI. Here's what bugged me: It was a 5 cm tumor, and, quite frankly, the signal from MBI was not that impressive at all. I also wonder if anyone actually -- oh, you know -- examined the patient. Most 5 cm tumors are palpable as masses. True, I've seen the occasional patient where such a large tumor is difficult to detect, but these patients are relatively uncommon. In another part of the talk, Dr. Rhodes showed one slide in which the mammogram showed one lesion, but the MBI showed three, one of which was only 3 mm in diameter. Unfortunately, she did not say whether pathology of the resected tissue verified that these were indeed separate foci of cancer.

What this leads me to believe is that Dr. Rhodes either doesn't acknowledge or doesn't seem to understand the concept of the Will Rogers effect, more formally known as stage migration. This is the phenomenon where a new imaging modality detects tumor that couldn't be detected before. The name is based on Will Rogers' famous joke: "When the Okies left Oklahoma and moved to California, they raised the average intelligence level in both states." This little joke describes very well what can happen after a new imaging modality is introduced into cancer diagnosis. Basically, the increased sensitivity of a new technique (like MBI) can result in a migration of patients from one stage to another that does the same thing for cancer prognosis that Will Rogers' famous quip did for intelligence. For example, patients who would formerly have been classified as stage II cancer (any cancer) have additional disease or metastases detected that wouldn't have been detected in the past, thanks to the new imaging modality. They are now, under the new conditions and using the new test, classified as stage III, even though in the past they would have been classified as stage II. This leads to a paradoxical effect in which the survival of both groups (stage II and III) appears better, even though there has not been any actual change in the overall survival of the group as a whole. This paradox comes about because the patients who "migrate" to stage III tend to have a lower volume of disease or less aggressive disease compared to the average stage III patient and thus a better prognosis. Adding them to the stage III patients from before thus improves the apparent survival of stage III patients as a group. Meanwhile, the patients who have extra disease detected by the new technology tend to be the stage II patients who would have recurred and done more poorly compared to the average patient with stage II disease; i.e., the worst prognosis stage II patients. But now, they have "migrated" to stage III, leaving behind stage II patients who truly do not have as advanced disease and thus in general have a better prognosis. Thus, the prognosis of the stage II group also ends up appearing to be better with no real change in the overall survival from this cancer.

There's another effect, as well, an effect that was first noticed when breast MRI began to be widely used for the preoperative workup of breast cancer. Because of the greater sensitivity of MRI, frequently more disease was discovered than expected, leading to more extensive surgery. The mastectomy rate, which had been falling for decades as a result of the greater understanding among surgeons that breast conserving surgery resulted in the same survival rate as mastectomy, began to rise again. Over the last few years, evidence has been accumulating that the routine use of preoperative MRI does not improve survival rates or increase the rate of lumpectomies with negative surgical margins but does increase the rate of mastectomies (blogged here). In other words, when it comes to screening, as I have described many times before, more sensitivity is not always better. It might be better in the case of MBI because the sensitivity of mammography in women with dense breasts is pretty low, but we won't know until we do the studies.

Which brings me to another part of this talk that irritated me:

Mammography isn't perfect, but it's the only test that's been proven to reduce mortality from breast cancer. But this mortality banner is the very sword which mammography's most ardent advocates use to deter innovation. Some women who develop breast cancer die from it many years later. And most women, thankfully, survive. So it takes 10 or more years for any screening method to demonstrate a reduction in mortality from breast cancer. Mammography's the only one that's been around long enough to have a chance of making that claim.

Elsewhere, on the TED Blog, Dr. Rhodes says that she thinks we should stop debating mammography:

So the problem is whenever a new technology comes around, the mammography mafia, as we call them, says, "Your test is no good, because you can't demonstrate a mortality benefit." Well, of course we can't demonstrate a mortality benefit. Mammography's been around since the 1960s; they're the only ones who have a prayer of demonstrating a mortality benefit, because it takes that long to demonstrate.

First, we need to stop debating mammography and put our resources into developing and evaluating alternative screening techniques for women with dense breasts. MBI is certainly a very promising technique, and there are other promising techniques. Second, we need to accept an endpoint for success that is not strictly mortality-based. Although mortality is the most important outcome, there are intermediate outcomes that can serve as acceptable proxies for mortality. For example, instead of insisting that each technique must demonstrate a reduction in mortality from breast cancer, I believe it is acceptable instead to evaluate whether one technique can find tumors at an earlier stage - in other words, small tumors that have not spread to the lymph nodes.

The "mammography mafia"? Nice. I also wonder if Dr. Rhodes has considered the possibility of lead time bias and length bias in her screening test. Whatever the case, in essence, Dr. Rhodes' argument boils down to a case of special pleading, wherein she insists that a different, more lenient, standard be applied to her favorite technology than was applied to mammography and than is applied to any other screening test. I can't agree. MBI should be subject to the same standards as any other screening test for breast cancer. If there's one thing we've learned over the last 30 years or so of mammographic screening is that it's harder than it seems it should be to save lives with a screening test and that screening tests have unintended costs and produce unintended harms. These have to be balanced against the benefits. We will never know for sure what these risks and benefits are for MBI if we don't do the studies, and if it takes a decade or more to find out what they are, then so be it, particularly given that there is potential for harm as well as benefit. That's how long it will take. Also remember this: MBI subjects the entire body to radiation in order to try to save lives from breast cancer. How do we know that repeated doses of ^99mTc sestamibi won't result in the increased incidence of, for example, colorectal or ovarian cancer that cancel out any decrease in mortality observed that is attributable to better screening for breast cancer?

We don't. And we won't unless we do the studies.

Dr. Rhode's MBI methodology is an example of a test that is evolutionary, not revolutionary. There is nothing whatsoever wrong with that, either. That's how science advances, building incrementally on what has gone before. In fact, Dr. Rhodes and her coworkers are to be commended for taking a test that never caught on widely because of its low sensitivity and specificity and recognizing that the technology had developed to the point where it might be possible to overcome these limitations. I just wish she wouldn't sell it to the general public as though it were some radical new test that the "mammography mafia" don't want you to know about.