Cognitive Daily

Americans, as any ScienceBlogger will tell you, have a woefully poor understanding of math and science. For the most part, even the most ignorant among us are able to stumble through life, but what happens when we’re confronted with a genuine scientific question with a real impact on our lives?

Consider the typical doctor’s office scenario: the doctor asks a breast cancer patient to decide on a treatment. “There’s a 30 percent chance of recurrence in five years,” she tells the patient, “but with chemotherapy, the chance is reduced to 10 percent.” If the patient doesn’t have a basic understanding of probability, she can’t make an informed decision about whether to undergo treatment.

Doctors are likely to be better educated than the vast majority of their patients, so how does this discrepancy impact the way they share information about medical risk with their patients? Andrea Gurmankin Levy and Jonathan Baron devised a study to try to understand the difference between doctors’ and patients’ concept of medical risk.

They asked paid participants to rate seven different medical conditions on a scale of “badness” from 0 to 127, where 100 corresponds to a 100 percent chance of immediate death (the scale goes up to 127 to account for the possibility that some conditions may be worse than death). The conditions were: a harmless, painless wart on the big toe; amputation of a big toe; deafness in one ear; amputation of one leg below the knee; amputation of both legs above the knee; blindness; and blindness and deafness. Nearly all participants ranked these conditions in the same order I’ve just listed them.

Next, the participants rated the badness of a series of probabilities of developing each condition on the same scale. For example, in one question they were asked to rate the badness of a 10 percent chance of having a toe amputated. Here are the results:

They are as you might expect: as the probability of developing the condition increases, the badness response also increases. The worse the condition, the higher the badness response at any level of probability.

But how do these ratings compare to the ratings of doctors? Perhaps surprisingly, the results weren’t especially different. Doctors tended to be more sensitive to probability, rating the higher probabilities as worse and lower probabilities as better than laypeople — but this effect was primarily observed at older ages. Young laypeople weren’t significantly different from doctors.

What’s most striking is the individual differences between respondents. Take a look at this chart:

The numbers have been transformed to a new scale in this chart, but you can still read it the same way. The dotted lines represent the bottom 20 percent of respondents in terms of sensitivity to probability, while the solid lines represent the top 20 percent. The bottom fifth of respondents essentially made no distinction between a 0.1 percent chance of acquiring a condition and a 100 percent chance — all were seen as equally bad. Meanwhile, the top fifth saw a 32 percent chance of deafness in one ear as worse than a 3.2 percent chance of death.

It’s pretty clear that the bottom twenty percent of this group simply has no grasp of probability, but the top twenty percent may be making a similar mistake by weighting the probabilities too much. The problem for doctors then becomes, how to express probabilities in a form that all their patients can understand. Unfortunately, this study doesn’t provide clear answers to that question.

Gurmankin Levy, A., & Baron, J. (2005). How bad is a 10% chance of losing a toe? Judgments of probabilistic conditions by doctors and laypeople. Memory & Cognition, 33(8), 1399-1406.

Comments

#1 CanuckRob
May 3, 2006

I would question how many physicians truly understand probablity themselves. Doctors, while not stupid peiple, are not (usually) scientists and or mathematicians. Perhaps the AMA could develop a way to help patients(and doctors) understand probability but I can’t see the average doctor being able to do so.
#2 Dave Munger
May 3, 2006

In the doctors’ defense, the researchers reported that “few” doctors were in the bottom 20 percent of sensitivity to probability. Of course, “few” implies some, and I’d submit that it is frightening that some doctors apparently can’t distinguish between a .1 percent probability and a 100 percent probability.
#3 Eric
May 3, 2006

“The problem for doctors then becomes, how to express probabilities in a form that all their patients can understand. Unfortunately, this study doesn’t provide clear answers to that question.”

Great question: how do you express probability concisely and accurately without education? A pamphlet addressing the most common misconceptions about probability? I can’t really think of a better solution.
#4 Ahmet Kutsi Nircan
May 4, 2006

This reminded me an edge.org article “Smart Heuristics” written by Gerd Gigerenzer. He talks about how he worked with “…physicians and physician-patient associations to teach the acceptance of uncertainty and the reasonable way to deal with it.”

Given a question on probability of being breast cancer after a positive test on mammography physicians and radiologists gave answers ranging from 1% to 90%!. Only 1/3s were close to the answer.

You can read it on: http://www.edge.org/3rd_culture/gigerenzer03/gigerenzer_p2.html
#5 Cat
May 4, 2006

I hope (*hope*!) that 20% is an overestimation of the amount of people who don’t have a grasp of probability. Is it possible that a considerable amount of these respondents simply didn’t care about the task, and just whizzed through it as quickly as they could? After all, filling out a survey about medical conditions is not the same as having those conditions apply to yourself.
#6 Dave Munger
May 4, 2006

Cat,

I don’t think so, because they did manage to distinguish between the various medical conditions (e.g. deafness worse than toe amputation). And consider the number of people who regularly squander huge amounts on the lotter (c.f. today’s foxtrot).
#7 That Girl
May 12, 2006

Perhaps they also did not consider the effect that some of these conditions have upon the “lower 20%”. Being deaf or deaf and blind doesnt make a difference – they both mean you will lose your job and be unable to pay your bills.
When studying how people rate injuries/disease versus how they rate risk – being in the lower 20% puts you at risk every day and your only option most times is to hope you beat the odds (thus the lottery).
#8 Joseph j7uy5
May 16, 2006

Thank you. This helps me to understand some patient’s responses to information about adverse effects from medication. It appears that, for some people, a 10% chance of getting headaches seems just as bad as a 100% chance.

Now I just need to figure out how to explain that to the patient, in 30 seconds or less.
#9 SmellyTerror
June 1, 2006

You know the old gag about something being “due”? As in, “there hasn’t been a plane crash for ages, so we’re probably due one”?

Lots of people believe that.

I did a quick survey of people I know on two questions:

1. We get a plane crash in this country – on average – about once a year. We haven’t had one for two years. Are we “due” to have one? Is it more likely than usual we’ll have one soon?

2. You flip a coin and get heads three times in a row. Is it more likely than usual that you’ll get tails on the next flip?

In both cases, more than half said yes.

You can see the same with casual gamblers. We have “two-up” in Australia, a coin-toss betting game that is only legal one day per year – so people who don’t normally gamble do so on this one game, on this one day. If there have been a run of heads, you’ll see a very strong trend as people bet on tails. They think tails are “due to come up”!

I think most gambling can be seen as a stupidity tax. Poker machines have a sign on them saying “this machine will return 87.5% of money over time” – but people will still pump money in, night after night. The really amazing thing is that they’re not embarrassed to do so.