The other day the British Medical Journal (BMJ) published a clutch of articles about whether Tamiflu was as useful a drug as some have touted. I read the main article, another one of the Cochrane Collaborative meta-analyses of the studies they deem useful about any particular subject, and it didn’t seem to make much news. It confirmed what their previous review had said about the neuriminidase inhibitor antivirals for influenza (Tamiflu and Relenza): these drugs work but their effect is modest. We’ve been saying the same thing for years here, not because we did a fancy meta-analysis, but because that’s quite clearly what the literature said. They confirmed it. Again. Not very interesting, I guess, so the BMJ, quickly becoming medical tabloid central, fastened on the one scientific aspect of the paper that might remotely have a news hook: the meta-analysis didn’t have enough information to show — according to Cochrane Collaborative standards, that is — that healthy people who got flu and were given a neuriminidase inhibitor avoided more serious complications like pneumonia. It didn’t show the antivirals didn’t work for this. It just alleged there was no Cochrane-required-level-of-evidence they did. The data — in their hands — showed the evidence was compatible with either outcome. Yawn. But yawn was good enough. It was elevated to a different story: that one reason we didn’t know is that the drug companies were hiding the data. That is a news story, I agree, but not a news story about whether the drug works or doesn’t. While just an allegation (because they didn’t get to see the data), the medical journal was doing this in collaboration with television channel 4 in the UK and the Cochrane Collaboration itself. Conflicts of interest?
The paper was co-authored by, among others, Dr. Thomas Jefferson and graduate student Mr. Peter Doshi, both of whom I have criticized here on one occasion or other. I’m not making any serious complaints about this paper (although why, on scientific grounds it should have been published in a high profile journal isn’t clear to me since it didn’t provide new evidence), but I do know enough about this kind of work to know that a great deal of judgment is used in accepting or rejecting papers (indeed that’s why this follow-up was done; someone objected to a paper that was considered in the previous review). However the overwrought point-counter-point between drug maker Roche, the authors, BMJ reporters and the editors over access to the data and who was supported by whom had the effect of entangling use of an important class of drugs for influenza — a class of drugs that everyone seems to agree works to some extent when no other does (and during a pandemic, no less) — and the important but unrelated issue of transparency over data involving pharmaceuticals. Let me be clear that on this issue I am on the BMJ’s side. I think it’s a scandal that we don’t have access to information used to license drugs given to the public with an official sanction of safety and efficacy. But it’s an issue that would likely crop up with almost any drug they sought to examine. Doing this in tandem with a media outlet whose objectives are not science but snagging viewers and the Cochrane Collaborative itself is unseemly at best and borders on the unethical. It makes it look like the BMJ was again engaging in self-promotion (OK, I understand it’s a business enterprise, but let’s recognize what’s involved). It didn’t hurt that the self-promotion seemed to serve everyone’s purpose (except for Roche’s, and frankly I can’t bring myself to feel sorry for them).
Well, maybe not everyone’s. I don’t think it served the public purpose or the public health. Tamiflu and Relenza work. They are in fact they only therapeutic modality we have other than supportive care or in critical cases heroic methods (mechanical ventilation). Vaccines are preventive, not therapeutic. So here we are in the middle of a pandemic and the Cochrane folks, aided and abetted by the BMJ and television producers, are saying, “How do you know for sure that they will prevent pneumonia in an otherwise healthy person who gets flu?” There is evidence about this, even if the Cochrane zealots don’t recognize it:
And observational data on these drugs’ usefulness in patients hospitalized with severe cases of flu – seasonal and H1N1 – points toward benefit, Dr. Tim Uyeki of the U.S. Centers for Disease Control recently reported in the New England Journal of Medicine.
“We don’t want people to stop using the antivirals in the way that we’re recommending they’re used because we believe they are having a beneficial effect on hospitalization, on severity of illness and indeed (preventing) death,” [antiviral expert Charles] Penn said Tuesday from WHO headquarters in Geneva.
“And if people stop using them, then the consequences of that will be an increased burden on the health-care system and worse outcomes.” (Helen Branswell, Canadian Press)
But as long as we’re talking about the need for randomized trials — something I’m not against but know enough about not to think they are automatically better than observational studies — let’s really talk about the need for them. The Cochrane Collaborative exists to do systematic reviews of the medical literature for the purpose of improving medical outcomes (note that they set their own rules for this). To be at all effective they have to be read and understood and used by practicing clinicians. Do we really know that clinicians who follow the recommendations of the Cochrane Collaborative have better outcomes for their patients? Isn’t it possible that some Cochrane recommendations actually do harm by inhibiting the use of treatments that are helpful in certain cases? Remember that their judgments, even if valid, are about average effects. If we wanted to find out if anyone should pay attention to a Cochrane review, shouldn’t we subject them to a randomized clinical trial? It would be easy. We could give a randomized groups of physicians real and sham Cochrane reviews (no one can really understand how they are really done from the descriptions in the papers anyway so the placebo ones would look just like the real ones). Then we could see if they do result in better medical practice.
And I know just who would publish it.