Respectful Insolence

It seems a reasonable question to ask, given my propensity for it.

Unfortunately that’s not what our Seed overlords asked this week. This week, they ask:

If you could shake the public and make them understand one scientific idea, what would it be?

Predictably, some ScienceBloggers answered: evolution and what it really means, not the parody of evolution presented by creationists or the simplistic version of it that is often taught in school or discussed in the mainstream media. I can’t argue with that answer, but I’m a physician; so my answer will be different:

If I could get the public to understand one scientific idea, it would be the concept of clinical trials. In particular, I would want to make them understand why most anecdotal evidence is a very poor guide to determining what treatments are effective, thanks to the placebo effect, confirmation bias, and regression to the mean (all of which I discussed here), among other confounding factors. I would want them to understand the hierarchy of evidence in evidence-based medicine, and why double-blinded, placebo-controlled, randomized clinical trials (RCTs) are the strongest evidence, while retrospective studies are weaker, and case reports are the weakest of all. Finally, I would want them to understand that lesser evidence should only be used to guide treatment choices only in the absence of stronger evidence (RCTs), particularly for questions where doing a true RCT would be unethical.

An understanding of these concepts would go a long way towards decreasing the susceptibility of the public to the sales pitches of quacks and inject a healthy dose of skepticism towards the claims of pharmaceutical companies for their latest, greatest wonder drugs, as well.

Comments

  1. #1 Ahistoricality
    May 24, 2006

    You’d have to add some very elementary statistical principles: bigger trials mean more reliable results; not all positive results are statistically significant; clustering.

    It’s a good answer.

  2. #2 Catherina
    May 24, 2006

    I like the answer, too – unfortunately, alternative “medicine” seems to fill a yearning in the human soul that evidence based medicine cannot…I think if you could shake some “fulfilment” into people to make them more content, they would be much less gullible (be it for quacks, or revisionists, or creationists)

  3. #3 Julia
    May 24, 2006

    Good answer.

    In the short run, that’s a lot more important than evolution; in the long run, if people can get that into their skulls, that might make it easier to get evolution in there later. So that would be a win/win sort of thing. IMO.

  4. #4 tom-d
    May 24, 2006

    1) The scientific method, including the experimental designs described in earlier postings.
    2) How science is a progression. For example from Newton to Einstein, from Darwin to genetic engineering, we build on fundamental concepts. This is different from “scientists always disagree.”
    3) The standard model of physics that explains the evolution of chemical compounds from exploding stars.
    4) The evolution of biological systems and how entropy is not violated.
    5) The emergence of complex systems from basic functions in physics, biology, economics, and psychology.

    You wanted 5, right?

  5. #5 Hank Barnes
    May 24, 2006

    Wow, you are finally talking sense, Orac:

    I would want to make them understand why most anecdotal evidence is a very poor guide to determining what treatments are effective, thanks to the placebo effect, confirmation bias, and regression to the mean (all of which I discussed here), among other confounding factors.

    True

    I would want them to understand the hierarchy of evidence in evidence-based medicine, and why double-blinded, placebo-controlled, randomized clinical trials (RCTs) are the strongest evidence, while retrospective studies are weaker, and case reports are the weakest of all.

    True. One problem, though, with placebo controls. Some drugs have immediate side-effects, most, to my knowledge, placebos don’t. So, blindedness gets thwarted quickly.

    …..and inject a healthy dose of skepticism towards the claims of pharmaceutical companies for their latest, greatest wonder drugs, as well.

    Whoaa! This is almost revolutionary for you:)

    Not if you actually applied these fine principles to ALL of your pet projects, well, we’d be getting somewhere!

    Hank Barnes

  6. #6 Bronze Dog
    May 24, 2006

    Some drugs have immediate side-effects, most, to my knowledge, placebos don’t. So, blindedness gets thwarted quickly.

    I suspect some of us could be surprised about the way people react to placebo suggestion. Also, there’ll probably still be people who don’t get those immediate side effects.

    Of course, you’re welcome to suggest something better.

    Whoaa! This is almost revolutionary for you:)

    That’s nothing new. Wasn’t there also a post on the old site where Orac criticized the pharma industry for not doing enough to research new antibiotics?

    Not if you actually applied these fine principles to ALL of your pet projects, well, we’d be getting somewhere!

    So much innuendo, so little reference.

  7. #7 boletus
    May 24, 2006

    Interesting that you think this would be the number one.

    Intuitive intelligence may be the impetus behind the recent interest in alternatives. The current system in America is not the best health care system on the planet and every American should know at least this:

    http://www.businessweek.com/magazine/content/06_22/b3986001.htm

  8. #8 Hank Barnes
    May 24, 2006

    Boeltus,

    Yeah, good article. I love that quote by the heart surgeon:

    “The problem is that we don’t know what we are doing,” he says. Even today, with a high-tech health-care system that costs the nation $2 trillion a year, there is little or no evidence that many widely used treatments and procedures actually work better than various cheaper alternatives.

    HankB

  9. #9 Panda
    May 24, 2006

    However, anecdotal evidence does have one virtue. While clinicians use base rates with some modification, statistics intuitively gathered by patients from friends and family (people that live near and possibly have similar genetic predispositions to the patient) apply additional filters to base rates. Outliers often come in geographical or familial clumps. Individual and environmental differences are not really explained by statistical analysis of clinical trials. So, anecdotal evidence can have some unexpected weight.

    Of course, the collection of more extensive and detailed statistics could replace the role of anecdotal evidence.

  10. #10 Flex
    May 26, 2006

    Wow!

    Am I missing something or are the last three comments really hinting that alternative medicine has some validity?

    Comment 1 – People are turning to alternative health care because they ‘intitively’ know that modern medicine is not as effective as older medicine.

    Comment 2 – Newer medical practices may not be more effective than older, and by implication, they shouldn’t be used (or tested).

    Comment 3 – Anecdotal evidence can be considered statistical evidence gathered intuitively.

    I think these comments prove Orac’s point. More education needs to be done on the subject of clinical trials.

    Intuition is highly suspect because it is clear that we are very adept at self-deception. Thus blinded-trials, and double-blinded trials, are necessary to tease accurate information apart from information generated by self-deception.

    Anecdotes can suggest directions for research, and it does. However, because of our great capacity for self-deception, anecdotes are not evidence. Further, it seems likely that anecdotal (narrative) evidence is given more weight simply because it has a narrative. Are there people who still believe the anecdotes even though extensive double-blind studies show nothing? Yes.

    Finally, there are many possible reasons for people to embrace alternative medicine. These reasons range from fear through ignorance and includes such disparate reasons as fear of debt to false promises and hope provided by the alternative health industry.

    Can improvements be made to modern medicine? Certainly. But to reject what we already know because we can’t regrow a kidney (yet), is just as silly as thinking that all of evolutionary theory is at risk because we can’t fully explain a developmental pathway.

    Every profession has huge, gaping, holes in it. My own profession as an automotive electical engineer (a much more tightly focused profession than Orac’s, or medicine as a whole) is still finding second and third order interferance affects for example. The only profession which claims it has no holes is theology.

    It’s the holes which makes the job interesting.

    -Flex