Posted by On Saturday we posted our take on The Atlantic magazine article by Shannon Brownlee and Jeanne Lenzer. It’s a major story in the November issue, a banner across the top of the cover page reading: Swine flu: Does the vaccine really work? We tried to ignore it. People kept asking us to comment on it, but we didn’t want to get entangled in vaccine controversies. As Orac warned me, it’s a game of Whac-A-Mole. But we got fed up and posted our global response, not a point by point refutation, since that wasn’t what the issue was. Our main point was that it was a straw man argument built around the narrative device of the brave, mavericky truth teller who is shunned by colleagues and has to eat alone at conferences. A scientific Enemy of the People. Not unexpectedly it drew a sharp response from the authors here and elsewhere. So like the worker who gets his sleeve caught in the machine, we are being drawn inexorably into the gears of the vaccine controversy. Sigh. We should have known better. But what’s done is done, so we will add this to our initial immediate reply to the authors.
Brownlee/Lenzer title their response to us, “Faith-based science not methodolatry is the problem.” This is apparently a reference to our use of the neologism methodolatry to describe what we see as Dr. Jefferson’s overweaning fealty to the randomized trial as the only reliable knowledge. It was actually a rather minor point, but not to hear the authors tell it:
Blogger Revere describes our article, published in The Atlantic, ?Does the Vaccine Matter?? as an instance of ?methodolatry.? Revere and his followers, are so enamored with their new-found term that they fail to examine, using facts rather than insinuation and emotion, whether this article was worthy of that term – a term that is useful and could be used to describe a very real problem.
Leaving aside the issue of using “blogger” as an epithet (something they also do with Dr. Gorski), they apparently can’t separate their role as journalists from the role of Dr. Jefferson as the person we accused of methodolatry. We did not refer to their article as methodolatry because, as far as we know, the article had no formal study design. It is a magazine article meant for popular consumption and is clearly rhetorical in nature, where we use the word rhetorical in the sense of being an example of the art of persuasion. That’s the problem. It attempted to persuade people there is a real question about the usefulness of influenza vaccine during a pandemic, a time when the public is trying to understand what to do. It confused pretty much everyone who read it on that score. That might be acceptable if there were grounds for being confused. But the article doesn’t present any. It talks about a long standing controversy regarding the efficacy of influenza vaccination among the elderly, which for seasonal flu make up 90% of the fatalities, not the accepted efficacy (accepted even by Dr. Jefferson) of well matched influenza vaccine in the younger age groups that are the main target of a pandemic virus, including this one.
In a response to us over at the Knight Science Journalism Tracker (KSJ Tracker won’t accept our direct response because we use a pseudonym, although they have offered to let a colleague post under his or her own name), Brownlee and Lenzer try to dial back the their argument to reach safer ground:
We?re simply saying that the nation?s seasonal flu strategy, which is to vaccinate the elderly and other vulnerable populations, may not be working as well as has been claimed. If we rely on vaccine during a major, deadly pandemic (which thus far this one does not appear to be, but things could change), the public may not be as well protected as we might hope or believe. We don?t say flu vaccine is worthless, we simply say nobody really knows how much protection it might offer. Better studies would help define what vaccine can and can?t do – and help us prepare for the really nasty pandemic that many experts believe is only a matter of time. (Knight Science Journalism Tracker)
I don’t know many people (in fact I don’t know anyone) who took away from their article that it was a simple question about the nation’s seasonal flu strategy, which targets the elderly. That’s something we’ve done ourselves, for some of the reasons Brownlee and Lenzer cite. Go to the KSJ Tracker site and take a look at the cover of The Atlantic: Swine flu: Does the vaccine really work? That’s not about seasonal flu strategy. And that’s the nub of the issue.
And when it comes to pandemic flu, exactly how do they expect to demonstrate the vaccine works against a novel strain? Let a couple of waves of roll over us while they perform a randomized clinical trial? If you don’t use antibody titers as a surrogate for protection (again, a problematic subject we’ve highlighted here numerous times) then you are stuck. Being stuck doesn’t seem to be a problem for them. It is a real problem for people who have to make decisions in public health. None of Brownlee, Lenzer or Jefferson are in that category.
In their response here, Brownlee and Lenzer accuse us of disregarding “data or even basic facts,” which data and facts they go on to list, numbered 1 to 7. Numbers 1 to 6 are just a rehash of the seasonal flu controversy. Number 7 has this remarkable form:
7. Claims of ?benefit? from vaccine are often based on the ability to generate antibodies – yet the very people who generate antibodies, young healthy people, almost never develop complications from flu – raising the conundrum: Does it work in those for whom it?s necessary and is it necessary for those in whom it works?
This isn’t data or fact. It’s handwaving. Moreover there is a literature of randomized trials in this age group that show the vaccine works. That’s the data and basic facts authors Brownlee and Lenzer ignored. We didn’t discuss the whole controversy in our response but we did provide links to the places we discussed them in the past several years. That’s the beauty of the web versus their magazine. On the web we have these things called hyperlinks that can lead interested readers to the back-up details. But the main reason we didn’t address them — although the authors appear to be surprised we didn’t — is that they were not relevant to the issue at hand, whether vaccination is well-founded as a way to deal with pandemic flu.
What does well founded mean? For Dr. Jefferson and other randomized trial zealots it means only one thing: a double blind randomized clinical trial (RCT). That’s what we referred to as methodolatry. A demand for RCTs for swine flu vaccination before employing it in a pandemic defies the imagination. The existing data indicate that vaccines work for younger people. Using current standards it would be unethical to intentionally withhold them. You’d have to regress to a time before there was this thing called the Nuremberg Code, or if you prefer something closer to home, the Tuskegee syphilis study. But if you insisted, picture this. Those long lines of anxious pregnant women and families with small children lining up for blocks to get the flu vaccine arrive at the head of the line and are handed an informed consent form saying that half of them won’t get a real vaccine but only a dummy shot with buffer and they won’t know which ones they got. I’d nominate the authors or Dr. Jefferson to be in charge of getting the consent forms signed, but I’m not sure they could be there. Most of us live on planet earth but only someone living on a planet orbiting slightly past Pluto would think a randomized double blind trial could be done with swine flu vaccination (I know Pluto is no longer a planet; so sue me).
Here’s more of the Brownlee/Lenzer EM response:
Revere declines to give a data- or fact-based challenge to any of Jefferson?s findings; instead he calls Jefferson a ?martyr,? a ?crank? and a ?contrarian.? OK, now that Revere smeared him – did he offer a challenge to a single one of Jefferson?s findings? Did Revere provide any evidence that Jefferson was wrong in his assessments of the data or the shoddiness of the studies? The answer is no.
Finally, Revere says we ?rely on and romanticize? Jefferson. This is simply not the case. We presented the work of several other researchers – mainstream and highly respected researchers including Lone Simonsen, Lisa Jackson and Sumit Majumdar -each of whom raises similar questions about the validity of the assumptions made about flu vaccine. It is interesting that Revere never once mentions any of these researchers and their findings, instead he attacks the person he appears to see as the weak link, Jefferson. The degree to which we did raise some points about the way Jefferson is treated was not to ?romanticize? him but to point out that emotion rather than clear-headed, sober, discussion of data and facts seem to be the way that purportedly dispassionate scientists are responding to the challenge to widely held assumptions. That is troubling.
I addressed the first issue already: Jefferson’s points were not germane to the swine flu problem. If he were summarizing studies of flu among the elderly and the question is about flu in younger age groups, why bother to discuss it beyond pointing the interested reader to what we’ve said about it in numerous other posts. Which we did. We did not criticize Lone Simonsen’s work on the same topic (work which we’ve posted about here and which we take seriously) because her work wasn’t at issue, Jefferson’s was. What we did do, though, is present the Conclusory statements from three papers Jefferson had his name on that made the case in a form considerably weaker than presented by the authors. Maybe the way the authors presented Jefferson’s views distorted them, although I am giving them the benefit of the doubt (to Jefferson’s disadvantage). To say that describing the way Jefferson alleges he has been treated was purely factual or based only on his scientific views is disingenuous. The rhetorical use of this image in the article is plain to us but we invite any reader to make their own judgment. Here’s a link to The Atlantic article.
More:
Revere writes: ?There is nothing judicious about Jefferson? and ?he is extreme in his views, looking at Jefferson’s own recent papers in the literature show conclusions much weaker than the picture painted in this article:?
B&L respond: It is interesting that when Jefferson writes in a manner that is neither injudicious nor extreme, that Revere, instead of acknowledging his work as nuanced or careful, then calls his writings ?much weaker? – much weaker than what? The most extreme thing Jefferson suggested in interviews with us was that scientists don?t have all the answers to the flu vaccine story and that some RCTs are in order. That hardly seems injudicious or extreme.
His published statements, which we displayed, present weaker views than the authors portrayed. See previous point. We find non credible on its face the idea that the article only intended to convey that Jefferson was merely suggesting “scientists don?t have all the answers to the flu vaccine story and that some RCTs are in order. That hardly seems injudicious or extreme.” Nor would it have made much of a magazine article. “Does the vaccine really work?” based on your reporting of Jefferson’s views goes way beyond that. Way, way beyond that. And mind you the “answers” Jefferson was asking for had to do with the efficacy of the vaccine in the elderly, something not germane to the pandemic setting.
Still more:
B&L respond: Once again, Revere reverts to emotional suasion rather than fact to argue his case. We raised point by point concerns that researchers have had about the flu vaccine narrative. We demonstrated that those who raise questions about vaccine efficacy are sometimes subjected to faith-based, emotional attacks rather than sober discussion. The last part of Revere?s sentence is particularly troubling. He writes, ?…this was a particularly irresponsible time to pull this stunt.? The implication is that during this time of the 2009 H1N1 flu our article will be responsible for the loss of lives if people decline the flu vaccine. We?ve heard that argument before. When we challenged widely held beliefs about various interventions, including cancer screening campaigns, we were told that we were causing the deaths of untold numbers of people because of our articles. Years later, we are now joined by many top experts and professional organizations, such as the American Cancer Society, who now agree that the early voices were right. We could just as easily charge those whose work we criticize are causing the deaths of untold numbers of patients for publishing their work. But such emotional appeals have no place in science. We suggest that for those concerned about this issue, there is no better book to read than Arrowsmith.
Once again, we’ll leave it to readers to decide whether Brownlee and Lenzer rely on appeals to emotion, or if their device of calling our take “faith-based” is fact based. Next they’ll be telling me atheism is a religion. I didn’t accuse them of endangering lives. I accused them of spreading doubt and confusion at a time when there is already too much doubt and confusion. And why drag cancer screening into it? Is the straw man style just too irresistible?
Dr. Gorski has ably defended his statement here and written further here, so I’ll leave it at that.
Finally, we brought up the unsupported claim that hand washing will protect against flu. The authors apparently concede the point but go on to say it’s a sensible thing to do that will have other demonstrated benefits. We couldn’t agree more. Wash your hands. It will help prevent respiratory diseases (maybe even flu, although that is less clear) and intestinal ailments. But my point was not what they claim. I don’t demand high quality randomized trials of handwashing before recommending it. That’s a good example of methodolatry. And to prove that irony isn’t dead, they specifically use the word “ironically” to misunderstand an attempt at irony on our part:
Ironically, Revere’s argument about hand washing is precisely the same as Jefferson’s (and our) argument about vaccine: mortality reduction needs further direct testing in high quality research. The irony appears to be lost on Revere, however, who then goes on to say, “They [physical interventions] are still good things to do, although if Dr. Jefferson decides to review the literature, I wouldn’t count on him finding any support for them.” Actually, Jefferson wrote the Cochrane review on hand washing and other physical interventions, and indeed, he found support.
Not so minor point: they left off the final interrogatory: “So what?” Our point was this: whether there is RCT support or not for handwashing specifically in influenza (and that’s the subject, isn’t it? after all it is indisputable that polio vaccine works and that tetanus vaccine works, etc., but that’s not what we are talking about) or whether Dr. Jefferson finds papers he deems acceptable to that point or not, we wouldn’t or shouldn’t stop recommending people wash their hands unless there was a good reason, e.g., it was counterproductive.
OK. Enough. The gears of the machine are eating me alive.
October 27, 2009
October 27, 2009
October 27, 2009
October 27, 2009
October 27, 2009
October 27, 2009
October 27, 2009
October 27, 2009
October 27, 2009
October 27, 2009
October 27, 2009
October 27, 2009
October 27, 2009
October 27, 2009
October 27, 2009
October 27, 2009
October 27, 2009
October 27, 2009
October 28, 2009
October 28, 2009
October 28, 2009
October 29, 2009
October 29, 2009
October 29, 2009
October 29, 2009
October 30, 2009