Dispatches from the Creation Wars

David Bernstein in a post at the Volokh Conspiracy:

I’m working on a project about phony experts, which I loosely define as purported experts who can’t do what they claim to be able to do, or whose claimed expertise is nonsensical or non-credible. I would welcome reader suggestions regarding categories of experts to look into. Here are some examples I’ve been toying with: (1) Homeopaths; (2) Economic forecasters; (3) Creationists; (4) Psychiatrists who claim to be able to judge “future dangerousness”; and (5) Clinical ecologists.

I’m not looking for Holocaust deniers, Flat Earthers, or any other group that’s not taken seriously by the American public, but categories of experts (or even individuals–Deepak Chopra?) whose views are quoted in the media (Discovery Institute creationists), who testify as experts in court (future dangerousness prognosticators), whose services are paid for by companies (economic forecasters), or whose products and services many individuals buy (homeopathic “medicine”).

Cue the whining from Luskin in 3, 2, 1….

Comments

  1. #1 little peanut
    March 27, 2007

    How ’bout Dr. Phil?

  2. #2 coturnix
    March 27, 2007

    1 – send him to Orac for all sorts of medical quackery of which homeopathy is only one – and many are raking in nice money on their woo.

    2 – All economists?

    3 – A couple of months of 8h/day reading of Panda’s Thumb should do, I guess….

    4 and 5: never heard of those. Where did he get the idea that these are influential, trusted or raking in the money?

    Isn’t the biggest such business astrology? How about millions “earned” in sales of biorrhythms calculators?

  3. #3 Disgusted Beyond Belief
    March 27, 2007

    I posted there – nominating priests and other religious figures as bogus experts on ethics and morality. As if being religious makes you any more an expert on either.

  4. #4 mess
    March 27, 2007

    Coturnix,

    I agree – all economists? That semas a little harsh

  5. #5 Mikado
    March 27, 2007

    Sounds like there might be some crossover with James Randi here.

  6. #6 Raging Bee
    March 27, 2007

    Coturnix: “All economists” is a bit harsh — many of them are honest, and give us a good view of business events to counter the propaganda of advertizers and stock-traders.

    Of course, my mom was an economist, who spent 12 years debunking phony budget numbers from the Reagan and Bush I administrations, so I’m a bit biased here.

    And as for biorythms, I haven’t heard anything from that field since 1975. Would the calculators still work at all, or would they be nothing but rusted metal and crumbled plastic? Even Atari geeks would laugh at them.

  7. #7 MattXIV
    March 27, 2007

    4 is probably a reference to mental health examiner testimony in civil committment proceedings. 5 seems to be term that purveyors of MCS-related woo prefer to go by.

  8. #8 Mustafa Mond, FCD
    March 27, 2007

    Psychics who claim to be able to talk to the dead, solve crimes, and find missing persons.

  9. #9 Stuart Coleman
    March 27, 2007

    I think the question is: what doesn’t cause Luskin to whine? He should add psychics, UFOlogists, and astrologers, as large portions of the American public do believe them.

  10. #10 Dr. X.
    March 27, 2007

    Half the people I meet have a absurd degree of confidence in their understanding of… you name it…

  11. #11 Grouch
    March 27, 2007

    What about nutritionists who promote the latest fad and usually disagree with each other, even when their recommendations are not supported by actual testing?

    The latest is eating blueberries to prevent colorectal cancer. Mice were injected with an extract from blueberries that reduced CRC by 20%. Okay, but who is going to get these injections for the rest of their lives? Note that the experimenters didn’t feed the mice blueberries, they injected them with an extract. How much of this fruit would you have to eat daily to gain a benefit? If the answer is 40 pounds, forget it.

  12. #12 jon
    March 27, 2007

    Why not all psychiatrists and mental health professionals?

  13. #13 Bill Poser
    March 27, 2007

    Psychiatrists’ predictions of future dangerousness also play a role in decisions about the release of mentally ill criminals on parole or probation and in sentencing. And in Canada they play a role in decisions as to whether to designate a convicted criminal a “dangerous offender”, which basically means that he or she is too dangerous to release and is given an indefinite sentence even if the underlying crime is not punishable by a life sentence. It is most frequently used for serial sex offenders.

  14. #14 Coin
    March 27, 2007

    DAVID BROOKS!

  15. #15 Gretchen
    March 27, 2007

    I’m really surprised that nobody yet has said “clergy” or “theologians.”

  16. #16 Flying Fox
    March 27, 2007

    jon,
    Most psychiatrists and mental ealth professionals are honest, professional and genuine. If you want names, I can give em to you. Most therapists are quite willing to admit when they can’t give their patient anymore advice.

  17. #17 tristero
    March 27, 2007

    Any “expert” on the Middle East who:

    1. Has not spent years there.
    2. Cannot speak more than two languages in regular use in the Middle East.
    3. Talks about Islam as if it were one monolithic belief system.
    4. Has trouble locating Baghdad, Najaf, Mecca, or Amman on a map.
    5. Advocated the invasion and conquest of Iraq.
    6. Advocates attacking Iran.

  18. #18 Jon
    March 27, 2007

    Flying Fox:
    I agree with you that most practitioners are sincere. I’d meant to make the point that much of what they do is about as hit or miss as any ‘woo’. The effectiveness of non-pharmaceutical modalities is difficult to confirm.

  19. #19 Dr X
    March 27, 2007

    Jon,

    Why do you say that? There is a great deal of evidence that psychotherapy is effective in the treatment of a variey of conditions.

    This is a self report study, but it is one that’s available online:

    http://www.apa.org/releases/consumer.html

    Is it perfect? No. But, the findings are consistent with a great deal of research on nonpharmacologic treatments and pharmacotherapies. The self report on marriage counseling is consistent with the research in that marriage counseling at the time was not more effective than no treatment, although in the past 10 years several researchers have finally attempted to study relationships in the way that treatment of individuals has been studied. Previously marriage counseling had often been a shoot from the hip, piggy-backing of individual modalities into couples treatment. Efficacy with certain forms of marriage counseling is improving for certain couples. Not all relationships can benefit and there are ways to determine that within a few diagnostic sessions.

  20. #20 Jason
    March 27, 2007

    I’m working on a project about phony experts, which I loosely define as purported experts who can’t do what they claim to be able to do, or whose claimed expertise is nonsensical or non-credible. I would welcome reader suggestions regarding categories of experts to look into.

    Theologians, clerics, prophets and anyone else who claims knowledge or expertise regarding the existence, nature and will of a god or gods.

  21. #21 Jon
    March 27, 2007

    Dr. X:
    It may be that I’m giving too much credence to controversy, but ‘Who–Or What–Can Do Psychotherapy?” was quite an eye opener for me. This famous study/lit review found that practitioners’ credentials, years of experience, education, or training couldn’t be correlated with the success of treatment, and that nonprofessionals were just as effective as professionals. To me, this calls into question the “expertise” of those professionals. (I don’t mean to present myself as particularly knowledgeable on this topic. I’m not a healthcare professional.)

  22. #22 Jon
    March 27, 2007

    Dr.X:
    I forgot to thank you for the link. I appreciate that you used a consumer publication rather than a healthcare org that might be more invested in positive results. However, I’m inclined to discount your survey because it doesn’t address my comparison of psychotherapy to woo: I imagine a survey of astrology consumers will find that those who used astrology the longest had the best results, just as those in pschotherapy self-reported increasingly better results with longer treatment. I’m open to getting slapped down on this, though.

  23. #23 RavenT
    March 27, 2007

    5 seems to be term that purveyors of MCS-related woo prefer to go by.

    Thanks for clearing that up, Matt; I thought it was referring to disease ecology, and wondered what such a specific area of epi/infectious diseases research had done to be targetted like that.

  24. #24 Jon
    March 27, 2007

    Hi Jon,

    I’m going to long, so try to bear with me.

    There are problems with self report, however, you should note that consumers did report problems with their pharmacotherapy and they did not report good results with marriage therapy, which tends to belie your assumption about positive reporting bias in the CR measures of improvement.

    In any case, the findings in the CR research are consistent with findings from academic research. If you discount academic research then essentially you’re saying you won’t accept evidence based upon the use of control groups and comparision groups. Who else is going to conduct this research? It’s going to be university and teaching hospital based.

    I don’t know what you mean by woo, but various psychotherapies have been compared to no treatment (wait list), pharmacotherapy, pharmacotherapy-psychotherapy combination, talk with sympathetic non-professional listeners and participation in self-help groups.

    In the academic research, psychotherapy is more effective than no treatment while the other non-pharmacological treatments have not been found more effective than no treatment. Psychotherapy is at least equal to pharmacological treatment (some individuals do not fare well with drugs but do well with psychotherapy and vice-versa and some don’t improve with either). Improvement is also better sustained with psychotherapy than with pharmacological treatment after termination of treatment.

    Researchers have no basis for using individuals seeing astrologers as a research comparison group. Astrologers don’t claim to treat mental disorders. You might just as well use plumber’s customers or TV repair customers as a comparison group.

    Mental health researchers use non-treated controls and other treatment groups of the type I described above. If someone wants to evaluate the effectiveness of astrology as a treatment for mental disorders, they should use the same kinds of comparison groups to determine efficacy.

    To place the burden on mental health researchers to compare themselves to astrologers (who have no research to support efficacy in the treatment of mental disorders) is arbitrary. The comparison to non-treated controls, sympathetic listeners and self-help groups indicates that psychotherapy is effective. To the best of my knowledge, astrology has not been shown to be more effective than any of these comparison groups. There is no reason for anyone to conduct such research because no one has seriously hypothesized that astrology is an effective treatment for any mental disorder.

    Why not insist that astrology be compared to antibiotics for the treatment of infections? It’s just as arbitrary. That said, if astrology was shown to be more effective than non-treatment, sympathetic listeners, self-help groups, or if it was shown to be as effective as medication, I’d say go for it. In the meantime, it remains an arbitrary demand to insist that astology clients serve as controls or a comparision group in evalutating the treatment of any condition – be it a mental health condition or otherwise.

  25. #25 James
    March 28, 2007

    Labelling all economists as false experts is definetly too harsh, though as an economist I suppose I would say that wouldn’t I? :)

    Proper economic forecasting includes confidence intervals, just like any responsible forecast. The better economists will also tell you that a modern economy is just too complex to model effectively, and any forecast will carry high uncertainty.

    In the economics and finance field I would nominate “technical analysts” i.e. chartists. These are the people who claim to be able to predict future stock performance based on past stock performance. Its about as useful as haruspicy.

  26. #26 Tyler DiPietro
    March 28, 2007

    Following James on “technical analysts” I’d have to nominate “industry analysts” as well. Those are the guys you always see in the media (especially on cable channels like CNBC and Bloomberg TV) “advising” investors about what is and isn’t going to succeed in a given industry. It’s especially prevalent in the IT and electronics industries, and most of the time such people are wrong.

  27. #27 Jason Spaceman
    March 28, 2007

    I nominate Wilson Key, the ‘subliminal message expert’, who did pre-trial testimony for the plaintiffs at the Judas Priest trial back in 1990. According to a Skeptical Inquirer article Key pretty much destroyed his own credibility:

    It is possible that he undermined his own credibility with the court by opining that subliminal messages could be found on Ritz crackers, the Sistine Chapel, Sears catalogues, and the NBC evening news. He also asserted that “science is pretty much what you can get away with at any point in time.”

    There was a documentary made about the Judas Priest trial called Dream Deceivers in case anybody is interested. Even if you’re not a heavy metal fan it’s funny to watch all the 80′s era metalheads hanging outside the courthouse. Reminds me of Heavy Metal Parking Lot.

  28. #28 Jason Spaceman
    March 28, 2007

    Oh, and I almost forgot about Judith Reisman, anti-Kinsey activist who claims that the brain produces ‘erototoxins’ during the viewing of pr0n.

  29. #29 coturnix
    March 28, 2007

    Hey, “all econoomists” was a joke (and even had a question mark) because every few weeks or so someone writes a blog post asking the question “is economics science?”

  30. #30 Jim Lippard
    March 28, 2007

    Has psychotherapy really been demonstrated to be more effective than talking about your problems with a friend? I got the contrary impression from Robyn Dawes’ _House of Cards_, which argues that there’s a huge chasm between what experimental psychology shows evidence for and what is actually done in therapy.

  31. #31 mark
    March 28, 2007

    That’s a pretty momentous task. The sheer volume of people paid to astroturf, sling BS, and sow doubt is incredible. It also happens to be something I work on a bit. His work is cut out for him, and in the end, the people who do this are like whack-a-moles. They pop up with one astro-turfing organization, accomplish their goal, then slink away again, websites and all.

    The only really persistent one is the WSJ editorial page (ha!).

  32. #32 Jon
    March 28, 2007

    Jon:
    I don’t see how your post addresses my question about evidence that professionals perform no better than laypersons. And thanks, Jim Lippard, for your mention of House of Cards.

  33. #33 James
    March 29, 2007

    Tyler: The industry analysts (or “fundamental analysts”) are indeed generally wrong. Their problem is often not so much bad method but that by its very nature investment advice becomes obsolete approximately 5 seconds after it is made public.

    coturnix: I wondered if you were serious, but my irony detector has been on the fritz lately. As for the question of whether economics is a science I would say yes, and no.

  34. #34 Stegve
    March 29, 2007

    Here are some examples I’ve been toying with: (1) Homeopaths; (2) Economic forecasters; (3) Creationists; (4) Psychiatrists who claim to be able to judge “future dangerousness”; and (5) Clinical ecologists.

    (6) Chiropractors
    (7) Polygraphers
    (8) Doctors, pharmacists, and others promoting vitamin and other nutritional therapies, including “nutraceuticals”

The site is undergoing maintenance presently. Commenting has been disabled. Please check back later!