Good Math, Bad Math

The Balance of Screening Tests

As you’ve no doubt heard by now, there’s been a new recommendation issues
which proposes changing the breast-cancer screening protocol for women under
50, by eliminating mammograms for women who don’t have significant risk
factos. While Orac has done a terrific job of covering this here and
here, I wanted to throw
in a couple of notes and a personal perspective.

To begin with, there’s a bit of math which has been bandied about, and
I thought I’d just quickly walk through it.

When you look at things like screening programs, what you’re doing is
performing some kind of test on a very large population, in the hopes of
finding a comparatively small number of serious illnesses. Any process
like that is, necessarily a tradeoff.

I’m going to use totally fake numbers to explain this – so don’t
think that these numbers are real.

Suppose that you’ve got a non-contagious disease which will be caught by
one out of every 1,000 individuals. That’s a fairly rare disease. If
it were harmless, you’d completely ignore it – you wouldn’t even bother
spending money on researching a cure for it. It’s just not worth the
trouble.

Now, suppose that the disease is universally fatal. Only one person out of
every 1,000 will catch it – but all of them will die. In this case,
there’s not much point in trying to figure out who has it – there’s nothing
you can do for them. But you would start spending money to figure out how
to cure it.

Now for the tricky case. Suppose that the disease isn’t universally
fatal. It’s almost always fatal if it’s had time to establish itself. But
if you catch it early enough, then you can with high probability, save the
person who has it. This is the case for cancers where we consider
doing screening.

In this last case, there’s a very complicated tradeoff. Should you
spend time and money trying to detect the disease? Or should you spend
the time and money trying to discover better treatments for the disease?

In fact, the tradeoff is ever worse that that, for two reasons. First, the
screening process has a huge false positive rate. The screening process comes
up positive in 5 out of every 1000 cases. So you wind up treating 4 people who
don’t have the disease. And the treatment isn’t always completely
benign. Most of the time, you do an additional test, and that’s it. But that
additional test has some amount of risk to it – some people will end up dying
as a result of the treatment for a disease that they didn’t have. And second,
the test itself isn’t risk free: it’s got a small but real chance of
causing exactly the disease that it’s being used to detect!

So how do you set a balance? You can screen people – and save some number
of lives by detecting disease that would have killed them had it gone
undetected. But by doing that, you’ll give the disease to some number of
people who wouldn’t have gotten it otherwise; and you’ll harm some number of
people who were caught in the screening process, but didn’t have the disease
at all.

It ends up coming down to a mathematical optimization process. You want to
maximize the survival rate of the population as a whole. You do that by
putting together a lot of factors: how many people will get the disease? How
many will miss detection if you don’t do the screening? How many will get the
disease as a result of the screening? How many will be harmed by procedures
done as a result of false positives? And how many could have been saved by
spending money on developing cures instead of screening?

The current situation appears to be that for women under
50 who don’t have other risk factors, it doesn’t make sense for them
to get the screening. The risks and costs of the screening outweigh any
benefits of it.

To shift to the personal side for a moment, I’d like to provide you
with a concrete example of how a false positive can do harm.

Close to 20 years ago now, my father had a muscular cancer in his leg. He
had surgery followed by radiation to have it removed. Everything went
beautifully. It turned out to be a highly aggressive cancer, but the surgery
appeared to have gotten it all. But because it was so aggressive, they wanted
to keep screening, looking for any trace of a recurrence. About two years
after the surgery, they saw something on an x-ray – it was either a
patch of scar tissue right by a vein, or it was the beginning of a new tumor.
They did numerous tests, but nothing was able to determine definitively what
the hell it was. A radiologist from Memorial Sloan-Kettering thought it was
just scar tissue, and recommended waiting. But the radiologist from the
hospital where the surgeon worked was equally sure that it was cancer. So they
decided to do surgery – it was an aggressive cancer with a very low survival
rate; why take a chance on letting it spread? So they went in and removed it.
The second surgery went very badly. It took over a year for the surgical wound
to heal, and there was enough circulation lost during the surgery to kill the
nerves in his leg. After that surgery, he could never feel or move that leg
beneath the knee. For the next 18 years, it caused constant trouble with poor
circulation. A blood clot in the vein affected by the surgery is what
eventually led to his death.

The point of that isn’t to tell you a sad story – but to illustrate the
very real risks of any intervention. The initial surgery – the one for the
confirmed cancer, was a complex procedure, due to the location and type of the
tumor. But the second one was quite routine – removing a one-centimeter,
well-isolated mass from the muscle below the knee. It wasn’t simple, but it
wasn’t by any means a particularly complicated surgery either: it was the sort
of procedure that the surgeon geon did, on average, twice a week! But even a
routine procedure has risks. Even the most routine procedure can, in rare
cases, wind up killing you.

That’s the point of the optimization problem that’s used to figure out
whether or not to screen for a potentially deadly disease: no intervention is
ever free – and I don’t mean that just in terms of money. Every intervention
comes with an associated risk. You have to find out where the balance point is
between the risks that you’re trying to protect people from, and the risks
that you’re going to inflict on people in the process of protecting them.

The best recent evidence, when put into that optimization problem, has
strongly suggested that the mammograms in women under 50 aren’t worth the
risk. The risk of the radiation of a yearly mammogram plus the risks of the
biopsies end up being worse, on average, than not doing the mammogram. The
balance point between risk and benefit doesn’t work out well until you
shift the pool of people being screened to be older – and thus at higher
risk.

The natural response to this is to say: but what about the women under 50
who have cancer that would have been detected early enough to be cured? Isn’t
refusing to give them mammograms harming them?

Yes, it is. But giving the mammograms will harm a different
group of women – and it appears that the group harmed by the earlier
mammograms is larger that the group helped by them.

To give a very different example of screening balance decision, one
which again has some personal resonance for me:

Most men will, at some point in their lives, suffer from gastric reflux.
One of the side-effects of severe reflux is esophageal cancer – which is
almost universally fatal.

Esophageal cancer is actually pretty easy to screen for. The
vast majority of cases start as pre-cancerous lesions in the esophagus
long before the cancer develops. Those lesions can easily
be detected by doing an upper endoscopy. Nearly every case of
reflux-driven esophageal cancer is preventable by
endoscopic screening.

So – if we can save all of those people, why don’t we give every man over
40 a yearly endoscopy? Because the endoscopy has risks. They’re not
particularly common – but when they happen, they’re pretty severe (perforation
of the esophagus and/or stomach, infection). Even though those occur
very rarely, they do occur. And the risk of those injuries
caused by the procedure outweigh the benefits of the procedure for men with no
symptoms or risk factors for severe reflux-related disease. (The personal
connection here is that I’ve had 6 endoscopies, and surgery to try to
eliminate the reflux. I’ve also had several relatives die of esophageal cancer
caused by reflux.)

In contrast, we do routinely do colonoscopies to screen for colon cancer.
It’s another common, dangerous cancer. And the risks of doing a colonoscopy
are comparable (if not a bit greater!) to the risks of an endoscopy. But it’s
got no symptoms that we can use to identify people who are likely to develop
it. So we can’t do what we do with endoscopies – which is to select people at
risk based on symptoms. So we routinely screen people when they get to the
right age – because the number of lives saved is less than the number of lives
lost due to complications.

If we started doing colonoscopies at 30 instead of the current recommendation
of 50, we’d save some people from dying of colon cancer. But we’d also hurt
a whole lot of people without colon cancer. So we don’t do it.

It all comes back to the optimization problem: find the optimal point
where the benefits, costs, and risks balance out.

Comments

  1. #1 W. Kevin Vicklund
    November 19, 2009

    So we routinely screen people when they get to the right age – because the number of lives saved is less than the number of lives lost due to complications.

    Change ‘less’ to ‘more’

  2. #2 Tom
    November 19, 2009

    You didn’t make much mention of costs, and they are a significant consideration in this analysis. And, nobody is saying a 45-year old woman can’t have a mammogram. She just might have to pay for it herself, that’s all.

  3. #3 James Sweet
    November 19, 2009

    It ends up coming down to a mathematical optimization process. You want to maximize the survival rate of the population as a whole.

    It’s worse than that, though, because in addition to objective costs, you also have morbidities, which are purely subjective. Like the loss of feeling and mobility in your father’s leg.

    Also, I would argue that even the cost of deaths are subjective — they shouldn’t be, but we are humans after all, and trying to deny that is going to just result in a bunch of unhappy humans. The perceived “cost” of a we-did-all-we-could death vs. a death caused by an undetected condition might be different. In fact, I suspect the latter is generally greater — which helps explain the uproar over the revised guidelines.

    To what extent this irrationality should be indulged is debatable, but to ignore it completely is folly…

    Oh, and one minor nit: I would not say “the best recent evidence, when put into that optimization problem, has strongly suggested that the mammograms in women under 50 aren’t worth the risk” (emphasis mine). There are a lot of subjective costs in the analysis that drove the recent guidelines, and some people might legitimately look at the same data and reach a different conclusion. I agree that it strongly suggests that once every two years is better than annually — but on the 40-49 thing, the cost/benefit is not tremendously worse than in the 50-59 age group.

    This doesn’t change the points you made about the math behind it, but it is worth emphasizing that part of the reason for the controversy is that mammograms for 40-49 age group are a damn tough call.

  4. #4 spit
    November 19, 2009

    But giving the mammograms will harm a different group of women – and it appears that the group harmed by the earlier mammograms is larger that the group helped by them.

    But the level of harm in the group harmed by screening is going to be far, far less — maybe a biopsy that turns out to have been unnecessary, but there’s no way to know that ahead of time — than the level of harm in the group harmed by not screening — they’re probably going to die or have much more advanced cancer by the time it is found.

  5. #5 NeoDevin
    November 19, 2009

    spit: “But the level of harm in the group harmed by screening is going to be far, far less — maybe a biopsy that turns out to have been unnecessary, but there’s no way to know that ahead of time — than the level of harm in the group harmed by not screening — they’re probably going to die or have much more advanced cancer by the time it is found.”

    Except that there is a non-zero risk of causing breast cancer with the x-rays used in mammograms. And there are risks of infections/complications with the biopsies in false positive cases as well. Each of these risks to an individual is smaller than the risk to an individual with undiagnosed breast cancer; however, these risks need to be multiplied by the entire cancer-free segment of the population to get a proper comparison.

    Screening earlier presents a small harm to a large group of people, to prevent a large harm to a small group of people. The small group of people is small enough in those under 50 that the small harm to the large group is greater.

  6. #6 Gerald
    November 19, 2009

    There was a similar debate recently in Australia about testing for prostate cancers:
    http://www.theage.com.au/opinion/prostate-proposal-risky-20090923-g2l1.html

  7. #7 Chris
    November 19, 2009

    But by doing that, you’ll give the disease to some number of people who wouldn’t have gotten it otherwise;

    This sentence is confusing. You said the disease wasn’t contagious. Do you mean that the disease will be detected in some people, but they may die earlier from other causes?

  8. #8 Michael I
    November 19, 2009

    Chris@6

    In Mark’s scenario, the test itself has a small chance of CAUSING the disease. Therefore by testing lots of people who don’t have the disease, you will inevitably give some of them the disease.

  9. #9 spit
    November 19, 2009

    NeoDevin:

    The small group of people is small enough in those under 50 that the small harm to the large group is greater.

    I’m dubious about, in the real world, applying some sort of multiplication of small risk from mammograms, say, across a large population without having a way to really separate the breast cancer that was brought about by mammograms from the breast cancer that wasn’t, if such a thing could really be teased out of actual data.

    I don’t think your conclusion is obvious to me. I don’t necessarily disagree with the point you’re trying to make, but I think it’s fundamentally a subjective problem — there has to be a cutoff, and people will try to make that cutoff in different places given the data that we’ve got.

    I think the math can inform those decisions, but I don’t think in this case that it can provide an obvious answer. The difference between the 1 true positive in 1300-ish total screenings (for 50 – 59 year olds) and 1 in 1900 (for 40 – 49 year olds) is not, as far as I can tell, an “obvious” line when you consider the cost to that one patient of not screening.

  10. #10 NeoDevin
    November 19, 2009

    spit,

    The increased incidence of breast cancer due to the radiation dose received from mammograms is easily quatifiable. In addition, because remaining life expectancy is lower at 50+ than 40+, there is less time for a patient to develop cancer, and therefore the risk of developing cancer due to the mammogram decreases with age. Basically older patients are more likely to die from something else before they get radiation induced breast cancer.

    If cause fatal breast cancer in ~1 in 1500 patients through regular mammograms, then it’s pretty clear that (neglecting other complications and quality of life arguments for the moment), with the numbers you gave, it’s worth it for 50-59, but not for 40-49. I don’t have the actual numbers handy, if anyone does, feel free to post them.

    All of these factors, and others, are taken into account before a recommendation for screening is made. The goal of the recommendation is to maximise the life expectancy and quality of life of the people. The better you can quatify the various risks and benefits, the better recommendation you can make.

  11. #11 Cathy
    November 19, 2009

    For hormonal reasons, breast tissue in pre-menopausal women is denser than in older women, which makes the mammogram a less effective screening method for those under 50. This is another reason why mass screening for younger women is not recommended.

  12. #12 Gingerbaker
    November 19, 2009

    “The increased incidence of breast cancer due to the radiation dose received from mammograms is easily quatifiable.”

    Easily quantified or easily extrapolated? It is hard to imagine an actual human study being allowed to take place.

    It is easy to extrapolate an increased cancer rate in humans based on animals subjected to concentrations of dioxin, but I don’t think it would be accurate.

    If these recommendations really take into account increased cancers caused by screening mammograms, just how accurate is that number? We are talking of very low frequencies, and a 40-49 strata that is very close to the next decade.

    My concern is that part of the calculus for these new recommendations was actual monetary costs, not just the ‘costs’ of anxiety in women patients or increased cancer rates due to mammograms. I don’t think anyone is privy to the algorithm used and what weighting was given to monetary costs.

    Call me cynical, but when I see anxiety listed as a cost, I’m thinking that what they mean is they can justify not spending the dollars on 40-49 year old women.

  13. #13 Ali
    November 20, 2009

    Gingerbaker, does it matter if monetary costs are included? Unnecessary tests are part of the larger problem of healthcare in the US (increased costs for the insurance companies lead to increased costs for the rest of us, even when we do not participate in the original cost-increasing tests). If the data shows convincingly that more women are hurt by screening under 50 than are helped by it, and there are adequate provisions for women with BRCA genes to still be screened regularly, then why shouldn’t this equation factor in the monetary cost of screening women under 50 as well? I guess I just don’t see how that’s a bad thing.

  14. #14 NeoDevin
    November 20, 2009

    Gingerbaker,
    “Easily quantified or easily extrapolated? It is hard to imagine an actual human study being allowed to take place.”

    Easily quantifiable by comparing rates of incidence in people who have had the screening vs. those who have not.

    “My concern is that part of the calculus for these new recommendations was actual monetary costs, not just the ‘costs’ of anxiety in women patients or increased cancer rates due to mammograms. I don’t think anyone is privy to the algorithm used and what weighting was given to monetary costs.”

    Anxiety caused by false positives needs to be taken into account. Given that being told “Your mammogram was positive”, or even “You need to come back for additional tests” causes a huge amount of stress.

    Monetary costs need to be taken into account as well. Every dollar spent on mammograms is one dollar less that’s available for another procedure. This also has to be weighed into the cost-benefit analysis. If the money to screen women 40-49 for breast cancer comes at the expense of, say, screening men 50-59 for colon cancer, there will be a trade-off. There is not unlimited money to go around, and financial concerns need to be taken into account.

  15. #15 AnyEdge
    November 20, 2009

    @14: It’s not as simple as comparing the incidence rate in those screened v. those not screened. Because we know a lot about other factors for breast cancer than ‘subject has breasts’.

    Those with genetic predispositon will be screened more often. Some with extreme predisposition for extremely agressive cancers will have profylactic radical mastectomy. Thus, never screened, never positively tested. It’s far more complex than you suggest.

  16. #16 gourmet
    November 20, 2009

    What about doing a baseline mammogram at 40, then another at 45? Why does it have to be yearly or nothing?

    As a woman diagnosed with aggressive BC in her 30s, the only solution I see is a cheap, safe screening method. I had never had a mammogram because I was too young. By the time my tumor was discovered it was basically the whole breast. The guidelines wouldn’t change my situation, but what if I had been slightly older when the cancer started? A mammo at 40 might have saved my life.

    One other thing not in the discussion: rates of under 45 breast cancer is skyrocketing. Delaying diagnosis will lead to an increase in BC deaths. Mark my words. Plus, these new guidelines will make it easy for insurance companies to deny mammograms.

    The whole situation is very troubling.

    What I wonder is if the board didn’t take the “lead time bias” argument into account. There is a great essay by Plotkin arguing that we aren’t curing many early BCs, just finding them earlier and therefore making it look like women’s survival is longer. His argument is that the “cured” diseases would never have killed anyone anyway – they would have remained hidden forever. And the aggressive cancers will kill regardless of how early they are found.

    Pretty chilling stuff for a 41 year old with a huge grade III tumor.

  17. #17 gourmet
    November 20, 2009

    And no, I didn’t have any family history, was healthy, slim and a vegetarian.

    So, even though YOU at home probably think you’re safe and that cancer only hits the fat, old, lazy or genetically predisposed, you’re wrong. It can happen to you too.

  18. #18 Mark C. Chu-Carroll
    November 20, 2009

    @12:

    Monetary costs *are* a real and important factor to consider in things like this.

    For many people (in the US, for most people), the amount of money that they can spend on medical care is limited. There’s always a balance between giving people every possible bit of care that they could possibly ever want, and limiting it to what’s necessary.

    Why not do mammograms twice a year? You’ll catch some cancers much earlier than you would have if you did it yearly. Because the cost of it would be huge (both in terms of health, and in terms of money), and the benefit of it would be miniscule.

    If every time one of my kids got sick, we ran a genome extraction on the infectious agent to identify it precisely, and then picked a medication which was best for that specific infection – that would be wonderful. We’d be able to treat them better, they’d get better faster, and we’d have a whole lot more information about how and where antibiotic resistance is developing.

    We don’t do that, because the cost of sequencing the genome of a bacteria is just too expensive, and most of the time, the kid will get over the infection by themselves without help.

    Even being less extreme than that – last spring, I caught the swine flu. Or at least, I *think* I did. I wasn’t tested for it. The area that I live in had a huge infection rate, and the school system knew that several children had been sent home and tested positive for H1N1. Anyone who became ill with flu symptoms during the normal incubation period after exposure was assumed to have swine flu. Would it have been better to know for sure? Maybe. But it would have cost a huge amount of money to do the tests on the thousands of people in the county who were infected, and it wouldn’t have really helped anything.

    In the case of cancer, if you spend more money on screening, you’re leaving less money available for research and treatment. The numbers are fairly interesting to consider: the number that I recall is that for each 1800 mammograms of a woman below 50, 1 cancer is detected that wouldn’t have been detected otherwise.

    A quick web search says that a mammogram costs roughly $100. So the screening costs
    $180,000 to detect a single cancer.

    If that detection saves a life, that’s wonderful. But a fair number of the people detected have cancers that would have been detected anyway; and an additional number have cancers that won’t respond well to treatment – their lifespan won’t be improved by the early detection. (See Orac’s description of lead time bias.)

    It works out to something on the order of $250,000 per treatable case of cancer.

    Now, what you need to consider is: what else could have been done with that $250,000 that was spent to detect a cancer at an early stage? Could it have been used to provide better treatments for women with breast cancer?

    That’s a real question that needs to be balanced. It’s possible that you’ll save more younger women with breast cancer by putting the money into treatment rather than early detection.

    Another possibility is to use other screening methods to identify people at risk. There are lots of women out there with the mutations that make breast cancer more likely, but who don’t know it. What if we spent the screening money on identifying the women at risk?

    There’s a similar tradeoff being debated right now about cervical cancer. Cervical cancer is a terrible disease which kills lots of women. Right now, starting at age 21, most women get screening tests yearly to detect cervical cancer. But the vast majority of cervical cancer cases are caused by a virus – HPV. We can spend money on pap smears, or we can spend money on a rather expensive vaccine. Which one should we do? The balance appears to run in favor of the vaccine.

    Talking about these balances sounds cold. But it’s a real thing that needs to be done. You can’t just spend infinite amounts of money on medical screening. So you need to find out the balance, where the cost justifies the benefit.

  19. #19 James Sweet
    November 20, 2009

    @12:

    Monetary costs *are* a real and important factor to consider in things like this.

    Mark, just ignore it. Gingerbaker, a commenter who is usually somewhat sensible, made a big stink over at Orac’s blog with the same ludicrous argument. At one point, he/she actually asserted that the only reason “[r]esources are limited [is] because of collusion and lack of social justice.” Seriously.

    Normally a very sane commenter, but not on this issue.

  20. #20 Nelson
    November 20, 2009

    Let me preface this by saying these are just questions and first impressions. I don’t actually know much of anything about insurance, health care, or economics.

    How’s the change of money work? So, right now, my insurance pays the $100 for my wife to get the mamogram because that’s a service in our commodity based system.

    How’s that money get diverted away from that detection effort and toward the cure for my wife’s cancer?

    More likely, I would think, our insurance premiums go down by some amount. Then either (a) the insurance companies profit goes up and perhaps they contribute that increased profit to cures? or (b) the gov’t imposes some kind of tax which is used to fund cure research efforts. But often, the government uses money for other purposes… unless legislation is laser precise.

    From a monetary standpoint and in view of the way the system is constructed now: is it a necessary implication that if you don’t spend the money on detection efforts, then the same money is spent on efforts to develop a cure?

    If not, then are we sure that the money is being put to better use? Or is it used to build a bridge, or fund a school, or build some insurance executives house? Does the comittee take the way that the system really works into account?

    All arguments about the detection effort causing what you are trying to detect are perfectly reasonable and valid. But the monetary costs vs. benefits… I don’t know that its our best interests at heart… like whoever is spending the money will take that and do something “better” with it. I don’t know about that since its really us who the money comes from for a service provided directly to us. I’d like to see the accountable mechanism that proves that the money will now be spent on cure research.

    So I see Gingerbaker’s point. Maybe someone can educate me about that concern.

  21. #21 Nelson
    November 20, 2009

    Just as a quick follow up:

    I’d rather spend $100 a year on my wife’s mamogram than on a new tv or electrontrics or to fix a pot hole or whatever. She means more to me than any of that. But I’d also not mind spending $100 to find a cure or do something for her that will benefit her life more than that.

    And really, I’m just saying my wife as a placeholder for any female figure I care about.

    So where’s the money really going?

  22. #22 James Sweet
    November 20, 2009

    Nelson: I see these as two separate problems. Rarely when we save money in one area is it clear “where the money goes” — that might not even be a reasonable question in most circumstances. It’s like saying, if I scoop a bucket of water out of the ocean, from which river is the water coming to replenish it?

    To me, there is the localized problem of, “What is the best bang for the buck on cancer screening?” and there is the global problem of, “How can we identify and eliminate gross abuses of resource allocation?” If we conflate these two separate problems, it can lead to all sorts of absurd conclusions.

    For example, I might argue that we don’t want to take away the money to “build some insurance executive’s house”, because that money will just end up going to finance illegal CIA interrogations. At least when the executive was getting it, the outcome was innocuous, and it maybe provided a handful of jobs, right? Of course this is ridiculous, but you see my point… there is often not a clear point A-to-point B connection of “where the money saved is going”, and if you worry that money saved will be spent on something even more worthless, you can never save any money on anything.

    I have a bad habit when my wife and I are grocery shopping, that if she decides to buy some super high-end candy bar (her vice), I say, “Well, if we’re spending seven bucks on a candy bar, I can surely spend nine bucks and get this 6-pack of microbrewed beer!” (my vice) But the fact that she probably shouldn’t have indulged in the candy bar doesn’t mean that I should indulge in the beer. Or should I just get my beer anyway, because if I don’t, the money will just be squandered on candy bars? :D

    In the specific case of privatized insurance companies, this argument has a touch of legitimacy. The percentage of any given healthcare savings that ultimately just go to increasing the companies’ bottom lines is most likely appalling. (Though it’s hard to make a direct A-to-B connection as I described above) But I still argue that this is a separate problem. The fact that for-profit health insurance is a retarded idea does not give us carte blanche to spend money any way we want to, any more than my wife’s purchase of a seven-dollar candy bar gives me carte blanche to buy as much beer as I want. It’s still necessary to weigh costs and benefits — even monetary costs — or else you are going to end up with all sorts of pathological conclusions.

    I do think, though, that as long as we’re stuck with this broke-ass healthcare system, that maybe slightly deprioritizes monetary costs… but again, isn’t that exactly the problem we are having right now, that are causing healthcare costs to spiral out of control?

    As I argued to Gingerbaker over in Orac’s blog, the problem that healthcare reform is trying to solve is that resources are being allocated inefficiently. To argue that this means we should disregard efficiency altogether until reform is achieved is just silly.

  23. #23 Chuk
    November 20, 2009

    How would it change the numbers if there was a less dangerous version of a mammogram, like maybe an ultrasound based one? (Not that I actually have any idea whether that’s currently practical.) And MRIs are less dangerous but I think also more expensive.

    I do like the comment about finding cancers which would have probably been “mostly harmless” (or at least not fatal), and then treating them so aggressively as to cause more harm than the cancer would have.

  24. #24 James Sweet
    November 20, 2009

    @Chuk: I for one don’t know the answers to those questions, but one problem with MRI as a first-line screening method (in addition to the cost) is that it’s so sensitive that you end up treating even more of the “mostly harmless” tumors, i.e. ones that would never have become symptomatic during the patient’s lifetime.

    It’s really tough, because when you find a tiny tumor, there is no way of knowing if it is slow-growing and will never become a problem, or if it’s super-aggressive and you just got lucky in finding it early (really aggressive tumors can go from “undetectable” to “you ain’t gonna survive” in less than a year, i.e. between mammograms :/ ). Ideally, someday in the future you will be able to tell from a biopsy which type it is, and then make a decision between “aggressively treat” vs. “watch and wait”. But until that day, we’ve got this weird paradox where sometimes it’s better to not know you have a tumor… :(

  25. #25 Nelson
    November 20, 2009

    @22

    I don’t think that we should disregard efficiency, but we should remember how the system really works.

    With respect to the candy bar analogy: In this country you can buy an expensive candy bar or microbrew beer any time you want. It’s bad in that you can get obese, but its good in that you can.

    I remember coming home from serving overseas, and I got really bad food poisoning. I went to a local small town hospital after five days of not being able to keep anything down. The doctors there gave me two CAT scans. They probably did it to bill the insurance company since this hospital was having issues paying their own bills. But did I care then? Hell no, I wanted to get fixed.

    Do I care now? Maybe that money could have been spent more wisely on somebody else in the hospital or on the hospital itself. Maybe instead of two CAT scans I was only justified one. Maybe none. Maybe it was inefficient. Then again, maybe not. Doctors made that call, and even if it was inefficient you can be and do so in America.

    An engineer friend of mine once said: “Be efficient 99% of the time, but when you have to be inefficent don’t worry about efficiency.”

    So the question that this problem really raises is how you keep that freedom, while at the same time making sure that everyone has the ability to obtain affordable health care. Maybe you can’t do both at the same time… maybe you can. I guess we’ll find out in a couple months.

    The reason most people take affront to the 40-49 recommendation is that it is not coupled with anything else in these news articles. Mark and other people like him have to spell it out.

    Maybe the recommendation is good, but it (or more precisely, the news that tracks it) lacks the follow up of what other screening processes or pertinent risk factors justify a screening. Most people see it and think that it will give insurance companies justification to drop coverage for mammograms. Probably an irrational argument, but that’s what people see and think.

    I plan on reading some of the recommendation fully before I come to any decisions. I don’t know enough to support or refute it. I would recommend others do so as well.

    http://www.ahrq.gov/CLINIC/uspstf/uspsbrca.htm

  26. #26 Nelson
    November 20, 2009

    @23

    There is a breakdown of some of the considerations made with regard to the 40-49 argument and the treatment methodologies at the link below.

    http://www.ahrq.gov/clinic/uspstf09/breastcancer/brcanrs.htm#clinical

  27. #27 ScottSM
    November 20, 2009

    Interesting post. Just for closure to the story, who ended up being right about your dad’s second surgery? Was it scar tissue or cancer?

  28. #28 Mark C. Chu-Carroll
    November 20, 2009

    @27:

    Sorry, I thought I said. It was not cancer. He went through the surgery, the horrible recovery, and the 20 years of trouble with circulation in the leg, for nothing. Complications from that are what led to his death – he died of an antibiotic-resistant staph infection acquired in the hospital where he was surgery because of a blood clot in that leg.

  29. #29 ScottSM
    November 20, 2009

    Thanks. I guess you did say it was an example of how a false positive can do harm now that I think about it. In any case, that’s really a shame.

  30. #30 Justin
    November 22, 2009

    Hey guys, for more information about all of this, I would suggest reading Gigerenzer, Gaissmaier, Kuz-Milcke, Schwartz, and Woloshin’s 2008 paper “Helping Doctors and Patients Make Sense of Health Statistics.” It is located in the journal Psychological Sciences in the Public Interest. It is a very well-written paper (an easy read), and it gives a good lit review of much of the current research in the area of statistical illiteracy and the potential health problems that it can pose. Have fun reading, and keep up the pursuit of knowledge guys!

  31. #31 Brian
    November 22, 2009

    While another great analysis, good luck in fighting the good fight. The reason I say this is because of conversations I’ve had with people in a much, much, much simpler example of probability in a real world problem. When should you guess on an SAT question? This was back in the early 90s when I took it. 1 Point for a correct answer. -1/4 for an incorrect answer. Five choices. Guess if you can eliminate one response as dead wrong. Some people could not realize this was not a penalty for guessing. The Princeton Review made a small fortune pointing this out to people.

    Between 1: A lot of people not getting it and

    2: How big a deal it was when The Princeton Review pointed this out to people, good luck seeing a much more sophisticated analysis enter into the popular conscience.

  32. #32 trrll
    November 24, 2009

    The “where the money goes” argument doesn’t have a lot of applicability when it comes to something that almost everybody gets, like breast cancer screening. You can shift the costs around a bit so that men shoulder some of the costs for women, or the more well-to-do carry some of the costs for the poor, but basically the cost of the extra screening, plus overhead, just gets tacked onto your insurance premiums (or taken out of your salary if your employer pays, or–if we were to go to a more socialist health care system–tacked onto your taxes).

    The only real virtue of having routine screening covered by insurance is that it gets people to do things that are good for them, because they’re paying for them anyway, instead of being tempted to spend the money on something else that provides more immediate gratification. So it is worth considering the purely economic issue (even if, for the sake of the argument, you could save a few more people by spending a lot more money). How much would you spend to reduce your risk of dying of cancer by say, 0.1%? $100? $1,000? Would you sell your house? There is a cut-off point at which almost everybody would draw the line. And remember, it’s not like you aren’t allowed to sell your house to pay for the test, if it is really worth that much to you–you just aren’t going to be forced to do so. The problem with covering such things by taxes, or insurance, is that it is easy to loose sight of the fact that under any plausible economic system, unless you are at an extreme end of the economic scale, it’s going to be mostly your money paying for your tests. So people slip into grousing, “Those rich SOB’s would rather spend money on big houses and fancy cars than spend a nickel to save my life” (which is likely true, but largely irrelevant)

  33. #33 trrll
    November 24, 2009

    While another great analysis, good luck in fighting the good fight. The reason I say this is because of conversations I’ve had with people in a much, much, much simpler example of probability in a real world problem. When should you guess on an SAT question? This was back in the early 90s when I took it. 1 Point for a correct answer. -1/4 for an incorrect answer. Five choices. Guess if you can eliminate one response as dead wrong. Some people could not realize this was not a penalty for guessing. The Princeton Review made a small fortune pointing this out to people.

    I’d put it even stronger: always guess if you don’t know the answer. You may know more than you think you do. And if you don’t, you come out even on the average, anyway. Yet to this day, I still hear people saying, “Don’t guess, because they take off points fro wrong answers.” My nephew was given the “don’t guess” advice in a test prep course just a few years ago.

  34. #34 mac
    November 27, 2009

    Regarding the ‘always guess’ attitude – you aren’t answering an infinite amount of questions, and while the expectation value of your result will stay the same (or rise, if you had [b]any[/b] information about the answer), the standard deviation will rise – giving you a better chance of getting a grade well below what you ‘deserve’. (and a better chance of getting one much better than you deserve, but I fear being underrated more than I desire being overrated :))

    Sorry for derailing the discussion. Interesting post, MCC.

  35. #35 Gingerbaker
    December 4, 2009

    “@12:

    Monetary costs *are* a real and important factor to consider in things like this.

    Mark, just ignore it. Gingerbaker, a commenter who is usually somewhat sensible, made a big stink over at Orac’s blog with the same ludicrous argument. At one point, he/she actually asserted that the only reason “[r]esources are limited [is] because of collusion and lack of social justice.” Seriously.

    Normally a very sane commenter, but not on this issue.”

    — Jamse Sweet

    In case anyone happens to come back to this thread, as I just did, let me just say that according to you I caused a “big stink” and I am “not sane” on this issue? Hmmm… I think I made two posts there?

    Seriously, James, go fuck yourself.

The site is undergoing maintenance presently. Commenting has been disabled. Please check back later!