I keep getting asked about the Atlantic Magazine article, Does the Vaccine Matter? by Shannon Brownlee and Jeanne Lenzer, two reporters whose particular bias is that we as a nation are “over treated.” As a generalization that’s probably true, and finding examples isn’t hard. Unfortunately by taking as their main example flu vaccine during a pandemic, they have not only picked the wrong example but created more confusion at a time when there’s already too much.
Here’s the gist:
Vaccination is central to the government?s plan for preventing deaths from swine flu. The CDC has recommended that some 159 million adults and children receive either a swine flu shot or a dose of MedImmune?s nasal vaccine this year. Shots are offered in doctors? offices, hospitals, airports, pharmacies, schools, polling places, shopping malls, and big-box stores like Wal-Mart. In August, New York state required all health-care workers to get both seasonal and swine flu shots.
But what if everything we think we know about fighting influenza is wrong? What if flu vaccines do not protect people from dying?particularly the elderly, who account for 90 percent of deaths from seasonal flu?
Brownlee and Lenzer rely upon (and romanticize as a martyr and truth-teller) Dr. Thomas Jefferson, someone who is fast establishing himself as an “Evidence Based Medicine” (EBM) crank who who courts notoriety by being a contrarian. The kind of EBM practiced by the likes of Jefferson and some other randomized trials zealots is far from the judicious weighing of the evidence envisaged by its early proponents. For example, David Sackett, defined EBM as:
“the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research.” (BMJ, 13 Jan 96, Sackett, David L.).
There is nothing judicious about Jefferson, whose problem was described by one of my colleagues as “methodolatry,” the profane worship of the randomized clinical trial as the only valid method of investigation. In this case his evidence base isn’t even relevant, because we aren’t dealing with seasonal flu but pandemic flu. For the record, what he is saying about the uncertainties about flu vaccine efficacy in the over 65 age group isn’t new. In fact we’ve discussed it here, several times (here, here), going back a couple of years. But it is also clear that the vaccine offers protection in the age groups that matter for this pandemic, the people under age 50.
Nor is Jefferson, as claimed in the article, someone who “knows the flu-vaccine literature better than anyone else on the planet.” That’s an absurd claim. The literature is vast and he knows only a tiny part of it. But insofar as there is an acknowledged expert on vaccine efficacy, it would be biostatistician Elizabeth Halloran, who reviewed the clinical trial and experimental challenge literature recently in the American Journal of Epidemiology. We wrote a longish post on the subject here. There is general agreement, even among so-called skeptics, that the vaccine works in the under 60 age group, precisely the group at issue with the swine flu vaccine.
Jefferson’s points about vaccines in the elderly are not new, as I’ve said. But even though he is extreme in his views, looking at Jefferson’s own recent papers in the literature show conclusions much weaker than the picture painted in this article:
In long-term care facilities, where vaccination is most effective against complications, the aims of the vaccination campaign are fulfilled, at least in part. However, according to reliable evidence the usefulness of vaccines in the community is modest. The apparent high effectiveness of the vaccines in preventing death from all causes may reflect a baseline imbalance in health status and other systematic differences in the two groups of participants. (Rivetti D, Jefferson T, Thomas R, Rudin M, Rivetti A, Di Pietrantonj C, Demicheli V, Vaccines for preventing influenza in the elderly, Cochrane Database Syst Rev. 2006 Jul 19;3:CD004876)
Influenza vaccines are effective in reducing cases of influenza, especially when the content predicts accurately circulating types and circulation is high. However, they are less effective in reducing cases of influenza-like illness and have a modest impact on working days lost. There is insufficient evidence to assess their impact on complications. Whole-virion monovalent vaccines may perform best in a pandemic. (Jefferson TO, Rivetti D, Di Pietrantonj C, Rivetti A, Demicheli V., Vaccines for preventing influenza in healthy adults, Cochrane Database Syst Rev. 2004;(3):CD001269.)
We concluded that there is no credible evidence that vaccination of healthy people under the age of 60, who are HCWs [health care workers] caring for the elderly, affects influenza complications in those cared for. However, as vaccinating the elderly in institutions reduces the complications of influenza and vaccinating healthy persons under 60 reduces cases of influenza, those with the responsibility of caring for the elderly in institutions may want to increase vaccine coverage and assess its effects in well-designed studies. (Thomas RE, Jefferson T, Demicheli V, Rivetti D, Influenza vaccination for healthcare workers who work with the elderly, Cochrane Database Syst Rev. 2006 Jul 19;3:CD005187)
When there is evidence for efficacy in a trial but insufficient numbers for meeting Jefferson’s arbitrary standard of precision of the estimate, Jefferson typically interprets the results as showing insufficient evidence. For any particular study that may be fair enough, but EBM means weighing the evidence, not just tabulating. When he talks to reporters he abandons rigor and says stupid things. Celebrity does that to some people, especially scientists unused to being feted as truth-telling mavericks. And like many randomized trial zealots, Jefferson frequently implies lack of statistical significance is affirmation of the null hypothesis, a serious interpretive error that slides by most journalists. And I know from personal experience journalists are always trying to get you to say something in the least nuanced way. Too many scientists succumb.
The maddening thing about this article is that there are valid points to be made here and we have tried to make them until we were blue in the face (random examples here, here, here, here, here, here). We have always felt the way to prepare for and battle a pandemic is to rely on a strengthened and robust public health and social services infrastructure, not to plan for the best (that there will be an effective and timely vaccine available to everyone) while hoping the worst won’t happen. Vaccines and antivirals are a poor second best as a strategy. But that’s what we have now, and while not the optimum, they do work. On the other hand, the nostrums also touted in this article as a substitute for vaccines and antivirals, like washing your hands, have almost no scientific support in the literature for influenza. They are still good things to do, although if Dr. Jefferson decides to review the literature, I wouldn’t count on him finding any support for them. So what?
The bottom line is this. There is excellent and credible evidence in the scientific literature that vaccination against influenza reduces infections in people under 60, evidence that even Dr. Jefferson accepts. For those over 60, there are legitimate questions that were raised by others about the extent of the benefit of seasonal flu vaccine, but they were raised before Jefferson got into the act. The argument put forward in this piece is a straw man argument as far as pandemic influenza is concerned (and in which context it was placed).
I understand the rhetorical value of having a martyr-hero when pitching a story, but this was a particularly irresponsible time to pull this stunt.