In which David Freedman criticizes health journalism and simultaneously destroys my irony meter

About a year and a half ago, I applied a heapin' helpin' of not-so-Respectful Insolence to a a clueless article about the the "triumph" of New Age medicine. The article channeled the worst fallacies of apologists for alternative medicine. Basically, its whole idea appeared to be that, even if most of "complementary and alternative medicine" (CAM) or "integrative medicine" is quackery (which it most certainly is), it doesn't matter because allegedly it's making patients better because its practitioners take the time to talk to them in a way that most doctors do not. In brief, the article was basically a massive "What's the harm?" argument, combined with an argumentum ad populum, that portrayed CAM as the wave of the future, a wave that’s washing over medicine and teaching us cold, reductionistic doctors to care again about patients and thus make them better through the awesome power of our newfound empathy. Reading the article the first time, I thought that its author, David Fredman, a contributing editor for The Atlantic, had probably been watching treacly feel-good movies about doctors discovering how to become caring (like Patch Adams or The Doctor) a few too many times. He even contrasted CAM to what he called the “failure” of scientific medicine. Of course, he neglected to consider or acknowledge the biggest flaw in his thesis, namely that it is not necessary to embrace quackery in order to practice more empathetic, humanistic medicine. Not surprisingly, CAM advocates want you to think it is. They want you to think that it's impossible to be a caring, empathetic doctor practicing holistic and humanistic medicine unless you embrace the quackery that is at the heart of much of CAM.

Yes, David Freedman grasped the trope with both hands, hugged it tight, and then doubled down on the bad science. The sad thing is that he's capable of better, although he does seem to have a bit of a crush on John Ioannidis (what proponent of science-based medicine doesn't?) but for the wrong reasons. More on that later.

First, it was with a wry smile and a bit of trepidation that I came across Mr. Freedman's latest plunge into the waters of medical science in the form of an article published in the January/February 2013 issue of the Columbia Journalism Review entitled Survival of the Wrongest. it's subtitle? "How personal-health journalism ignores the fundamental pitfalls baked into all scientific research and serves up a daily diet of unreliable information."

Oh, goody. I can feel my irony meter tingling with anticipation.

It gets even better, because Freedman begins by criticizing Tara Parker-Pope, a writer and blogger for the New York Times known for her tendency to fall for the same sort of pseudoscience that Mr. Freedman did a year and a half ago, as she did with her credulous take on a series of very bad (are there any other kind?) acupuncture studies and her acceptance at face value of some equally bad science about a purported link between cell phones and cancer.

As my irony meter needle crept steadily upward, Mr. Freedman began:

In late 2011, in a nearly 6,000-word article in The New York Times Magazine, health writer Tara Parker-Pope laid out the scientific evidence that maintaining weight loss is a nearly impossible task—something that, in the words of one obesity scientist she quotes, only “rare individuals” can accomplish. Parker-Pope cites a number of studies that reveal the various biological mechanisms that align against people who’ve lost weight, ensuring that the weight comes back. These findings, she notes, produce a consistent and compelling picture by “adding to a growing body of evidence that challenges conventional thinking about obesity, weight loss, and willpower. For years, the advice to the overweight and obese has been that we simply need to eat less and exercise more. While there is truth to this guidance, it fails to take into account that the human body continues to fight against weight loss long after dieting has stopped. This translates into a sobering reality: once we become fat, most of us, despite our best efforts, will probably stay fat.”

Mr. Freedman then characterized Ms. Parker-Pope's article thusly:

In short, it’s a well-reported, well-written, highly readable, and convincing piece of personal-health-science journalism that is careful to pin its claims to published research. There’s really just one problem with Parker-Pope’s piece: Many, if not most, researchers and experts who work closely with the overweight and obese would pronounce its main thesis—that sustaining weight loss is nearly impossible—dead wrong, and misleading in a way that could seriously, if indirectly, damage the health of millions of people.

My irony meter has stopped tingling with anticipation and started tingling with the electricity of waves of irony washing over it. After all, leaving aside the inconsistency here (how could Ms. Parker-Pope's article be "well-reported" and "careful to pin its claims on published research" if, as Mr. Freedman claims, it came to such a horrendously incorrect conclusion?) Mr. Freedman's article on the "triumph" of New Age medicine could be described in a very similar way. Well, not exactly. Because Mr. Freedman cherry picked his "experts" at universities with active "integrative" (i.e., quackademic) medicine programs. The irony still remains, however. After all, Mr. Freedman has in essence accused Ms. Parker-Pope of cherry picking her studies and data, which is exactly what he did for The Atlantic a year and a half ago. Moreover, in doing so, he even goes on to quote David Katz, MD, the man who is best known among skeptics as the physician who advocated the concept of a "more fluid form of evidence than many of us have imbibed from our medical educations" and then used anecdotes about The One Quackery to Rule Them All (homeopathy) to suggest that homeopathy works.

In any case, I never thought I'd feel sorry for Ms. Parker-Pope, but I kind of do, because Mr. Freedman uses her as the poster girl for a problem in health journalism that most medicine bloggers, myself included, would also characterize as a problem: The persistence of medical misinformation in the popular press. I'll even give him credit for laying out the problem by listing stories that appeared in the New York Times. These articles were often in conflict with medical consensus, and many of them even contradicted each other. How's a reader to figure out what the consensus even is?

In fact, for a significant part of the article, Freedman sounds almost as though he were auditioning for this blog, minus the Insolence (well mostly—he just doesn't crank it up as high as Orac does). He pontificates on various problems that can interfere with the validity of scientific studies, including confounding factors, publication bias, and how, as John Ioannidis has shown us time and time again, at least half of all scientific studies related to health ultimately turn out to be wrong. I do find it rather telling, however, that apparently Mr. Freedman, a health journalist, doesn't know the proper use of the word "theory" in science. Hint: It is not synonymous with the word "hypothesis," but Mr. Freedman seems to thin it is:

A frequent defense of this startling error rate is that the scientific process is supposed to wend its way through many wrong ideas before finally approaching truth. But that’s a complete mischaracterization of what’s going on here. Scientists might indeed be expected to come up with many mistaken explanations when investigating a disease or anything else. But these “mistakes” are supposed to come in the form of incorrect theories—that a certain drug is safe and effective for most people, that a certain type of diet is better than another for weight loss. The point of scientific studies is to determine whether a theory is right or wrong. A study that accurately finds a theory to be incorrect has arrived at a correct finding. A study that mistakenly concludes an incorrect theory is correct, or vice-versa, has arrived at a wrong finding. If scientists can’t reliably test the correctness of their theories, then science is in trouble—bad testing isn’t supposed to be part of the scientific process. Yet medical journals, as we’ve seen, are full of such unreliable findings.

I respectfully (well, actually, Respectfully Insolently—or even not-so-Respectfully Insolently) suggest to Mr. Freedman that the word he is looking for is "hypothesis," not "theory." You might think I'm nitpicking or doing what drives me crazy when others do it to me: Being pedantic. However, language matters, and if Mr. Freedman can take the time to write such a long, generally well-written article I would suggest that he could take the time not to use the colloquial use of the word "theory" in an article about medical science. End of rant, at least this one point, because I actually agree (mostly) with this next point that Mr. Freedman makes:

Given that published medical findings are, by the field’s own reckoning, more often wrong than right, a serious problem with health journalism is immediately apparent: A reporter who accurately reports findings is probably transmitting wrong findings. And because the media tend to pick the most exciting findings from journals to pass on to the public, they are in essence picking the worst of the worst. Health journalism, then, is largely based on a principle of survival of the wrongest.

Of course, Mr. Freedman's exaggerating here to some extent. Most studies, even ones that are highly publicized, are quickly forgotten. It is, however, true that in health reporting, it seems that all too frequently it is the scum, rather than the cream,that rises to the top. Unfortunately, Mr. Freedman's prescription to fix this problem strikes me as impractical. Dismissing as useless and insufficient the usual caveats that should be, but seldom are, reported with studies, such as subject dropout rates, commenting on modest effects, and the like, Mr. Freedman insists that readers need to be alerted "to the fact that wrongness is embedded in the entire research system, and that few medical research findings ought to be considered completely reliable, regardless of the type of study, who conducted it, where it was published, or who says it’s a good study."

Well, yes and no.

Mr. Freedman's prescription, rather than a useful caveat to warn viewers, strikes me more as a prescription for inaction and not reporting any health studies. However, while it is good to remind readers that the results of any scientific study is tentative, Mr. Freedman seems to be continuing his misapplication of Ioannidis once again to paint all science as completely unreliable, as he did before when he referred to Lies, Damned Lies, and Medical Science. A less inflammatory (and likely more effective) way to communicate the limits of science would be to emphasize that all scientific studies are tentative and could be wrong, rather than saying that "wrongness is embedded in the entire research system." This might strike some readers as a subtle distinction, but it's not, not really. Mr. Freedman's approach is designed to present medical science as a near-fatally flawed enterprise, even through his choice of the word "wrongness," which implies a moral judgment (a better word would be "error"). My approach is designed to communicate the uncertainties inherent in science without painting the entire enterprise is wrong.

Don't believe me? Then check out this snark:

The New York Times’s highly regarded science writer Dennis Overbye wrote in 2009 that scientists’ “values, among others, are honesty, doubt, respect for evidence, openness, accountability and tolerance and indeed hunger for opposing points of view.” But given what we know about the problems with scientific studies, anyone who wants to assert that science is being carried out by an army of Abraham Lincolns has a lot of explaining to do.

Except that no one, not even Dennis Overbye, says that it is. Indeed, even he qualified his statement by pointing out, "Arguably science is the most successful human activity of all time. Which is not to say that life within it is always utopian, as several of my colleagues have pointed out in articles about pharmaceutical industry payments to medical researchers." To me that sounds as though Overbye isn't as Pollyanna-ish as Mr. Freedman tries to paint him.

Then there's this whine:

Of course, journalists who question the general integrity of medical findings risk being branded as science “denialists,” lumped in with crackpots who insist evolution and climate change are nonsense. My own experience is that scientists themselves are generally supportive of journalists who raise these important issues, while science journalists are frequently hostile to the suggestion that research findings are rife with wrongness. Questioning most health-related findings isn’t denying good science—it’s demanding it.

Uh, no, but do I detect a bit of a hint of resentment at the well-deserved criticism Mr. Freedman received for his credulous paean to pseudoscience in The Atlantic back in 2011? Or maybe not. It doesn't matter. Either way, this is a massive straw man argument. Journalists who question the integrity of medical findings are not considered "denialists." Journalists who reject well-established science like evolution and use the tactics of denialism to do so are correctly labeled denialists. In order to make a rhetorical point, Mr. Freedman conflates legitimate criticism of journalists who get it badly wrong, as he did in 2011, with dismissing such journalists as "denialists."

And he keeps using the term "wrongness" instead of the more appropriate "error."\

Mr. Freedman started his little screed with the arguments that (1) health reporting is mostly bad and (2) that bad health reporting endangers public health. Oddly enough, he can't seem to give any specific examples. That's why my irony meter cranks up to eleven, because I can point out a specific example where bad health reporting endangered public health and still does: Credulous reporting on the "concerns" about vaccines voiced by members of the antivaccine movement. As part of that, such reporters all too often fall for the bad science and outright pseudoscientific studies used by antivaccinationists to support their dangerously erroneous views. You can thank me later, Mr. Freedman, for that example, if you ever decide to use it in a followup article. Fortunately, journalists seem to have learned and appear much less solicitous of the antivaccine movement these days.

For the most part, though, Mr. Freedman reserves most of his bile for reporters who dare to report on issues of diet and obesity. It would have been helpful if he had provided a couple of counter examples to all that journalism that he considers to have been so bad, for example. Perhaps, as Paul Raeburn suggested at Tracker at the Knight Science Journalism site at MIT, Mr. Freedman thinks that any journalism that comes to different conclusions than he does about diet and obesity is fatally flawed. More importantly, I tend to agree with Mr. Raeburn as well that Mr. Freedman's critique is amazing in the broadness of its attack and failure to admit exceptions. Particularly devastating is his retort to Mr. Freedman's characterization of Parker-Pope's article as being "well-reported, well-written, highly readable, and convincing piece of personal-health-science journalism that is careful to pin its claims to published research," and then characterizing it as "dead wrong" and in conflict with the medical consensus as exprssed by experts asking:

Again, we find ourselves in a logical knot. If the article was "well reported," how did it miss the "many experts" who would pronounce its thesis dead wrong?

Maybe she misinterpreted or dismissed what the experts told her, the way Mr. Freedman in essence dismissed what Steve Novella and others told him about alternative medicine. (Sorry, couldn't resist.)

I was also amused by Mr. Raeburn's likening of Mr. Freedman's article to the liar's paradox:

The logical problem here is that Freedman's story is a piece of personal-health journalism, and the criticism he applies to others dooms his own piece. His story collapses like a liar's paradox: All health journalists are wrong, he writes. But it can't be true. If they are, then he is right.

Indeed. I can almost picture that scene from the Star Trek series (the original series, of course) I, Mudd, in which Kirk and crew used the liar's paradox to cause an android to fry its logic circuits.

When it comes to medicine, Mr. Freedman's reporting seems to be a litany of "too much is never enough. He makes some good points about health journalism and its tendency to sensationalize. There is no doubt that is a problem. I've written about it myself many times. He also cautions about the tentativeness of scientific findings, something I've done many times. However, he can't resist going too far. To Freedman, science has "wrongness embedded" in it, and journalists, although skillful and able to document their work with studies, get it completely wrong (except, apparently, when they are David Freedman (although in fairness he does make one self-deprecating acknowledgement of the problem).

It reminds me a lot of his reporting on John Ioannidis, in which he played up every criticism that Ioannidis makes about how medical science is done and seemingly uses it to try to demonstrate, in essence, that science is inherently unreliable. He seems to forget (or not to know) that, for all its flaws, science is indeed self-correcting, and it seems to fly right by him that John Ioannidis is an example of that very process of self-correction inherent in science! True, science doesn't correct itself nearly as fast as Mr. Freedman apparently thinks it should. As I've said, the process is messy and takes longer than even scientists would like, but science does eventually correct itself. It is not full of "wrongness," as Mr. Freedman seems to believe. To the extent that he reminds health journalists that no scientific study is the final word or beyond criticism, Mr. Freedman has made a good point. To the extent he portrays science as inherently unreliable, he's distorted the situation at least as much as the journalists whom he attacks.

Oh, and now Mr. Freedman's exploded my latest irony meter. Remember how he started his article by castigating Tara Parker-Pope for having come to conclusions about obesity and diet that, according to him, were at odds with the expert consensus? He seems to have a lot of faith in experts, doesn't he? So what was his most recent book called? Believe it or not, it was entitled WRONG: Why experts keep failing us–and how to know when not to trust them.

You can't make stuff like this up. At least, I can't.