Peter Doshi has a bone to pick with CDC . His particular idée fixe is that CDC is cooking the books on their estimates of excess mortality attributable to influenza and he aims to set the record straight. He’s done it before. Doshi is not the kind of critic CDC is used to. He is a graduate student, not an established public health figure. But he’s no shrinking violet and is getting in CDC’s face again in the latest issue of the American Journal of Public Health. This time Doshi extends his criticism to imply CDC is pandemic fear mongering, perhaps in collusion with Big Pharma. This has been interpreted in the press as saying “a new study” suggests we don’t have much to fear even if a pandemic materializes because a pandemic is no worse than seasonal influenza.
Needless to say I don’t have a lot of sympathy for this position, but Doshi’s points still need to be examined. In my view they have their roots in his misunderstanding of how and why CDC estimates excess influenza mortality. The number of deaths attributed to flu each year is far in excess of deaths attributed to flu on a death certificate. Doshi has never filled out a death certificate, but I have. It is not rocket science. In fact, there may not be much science in it at all. Only about 10% of deaths in the US have autopsies and many pneumonia deaths among the elderly are never diagnosed, either as pneumonia due to influenza or even pneumonia at all. So there is a misspecification of influenza deaths in the mortality data. Some of the deaths due to influenza may not be due to infection with the virus but other respiratory viruses. Many influenza deaths will never be classed as influenza because there is no information upon which to base the diagnosis. Classification as influenza is more likely to be true during an outbreak for purely mathematical reasons (a consequence of Bayes’ Theorem) but the relation between deaths with “influenza” as a principal or underlying cause of death is uncertain and has varied over the hundred years that Doshi surveys in his paper.
Despite the difficulty, the toll attributable to influenza remains is an important public health question and CDC has approached it by estimating excess mortality using the total of deaths classified on death certificates as due to pneumonia or influenza, so-called P&I deaths. Because we know P&I mortality has a seasonal pattern, the estimate is made using historical data on the phase and amplitude of the seasonal variation (the frequency is annual). This provides a seasonally varying baseline for what is expected. Excesses beyond the expected are the excess mortality from influenza.
Do we know all the excess is due to circulating influenza virus? Some of it is since we know a fair amount about the seasonal variation in infection from that virus (although we don’t know what produces the seasonal pattern). But as Doshi and almost everyone else notes (including CDC), other viruses are also involved. Does that mean the excess is an obvious overestimate? It would if we thought the only effect of influenza on mortality were in P&I deaths. But we also know with certainty that mortality registered as from other causes, particularly cardiovascular deaths, may be hastened, if not directly caused, by influenza infection. That is a large pool and if even a small proportion would not have occurred absent influenza infection, the P&I estimate is also an underestimate.
Doshi says in his paper he asked for CDC surveillance data on co-circulating respiratory viruses so he could take them into account in his interpretation. He claims CDC would only give him the data on condition of co-authorship. Since I have never heard of CDC doing this, I am surprised and dismayed — if it is true. I don’t consider this acceptable behavior regarding de-identified surveillance data gathered under government auspices. Doshi has a right to the data and withholding it seems petty, uncollegial and possibly unethical. Perhaps there is another side to the story, but Doshi’s version is disconcerting.
At any rate, the issues raised by Doshi have been discussed and examined many times by others, both within and outside of CDC. Experts in this area are not naive to the possibilities. They are in fact much more expert than Peter Doshi, who keeps referring to subtypes as strains and has no problem taking 100 years of deaths classified on a death certificate as influenza as reliable data (nor does he say if his data are primary cause or underlying cause or even if he knows). He sums up his “concerns” with three points. The first is that recorded influenza deaths are only a small proportion of deaths CDC attributes to influenza. That’s what we have been talking about, above. His second point is that in the 1980s and 1990s the estimate of the excess has gone up but the deaths classified as influenza have declined. He is not the first to notice this. CDC flu experts have looked for the reasons for this discrepancy and have come up with a number of plausible explanations. Whether it is trends in subtypes or reporting differences over time the questions have been asked and possible answers proposed. Some highly expert scientists have been worrying about and discussing these things for years. They are not new or suppressed questions. Doshi’s third concern is that various models come up with different estimates. As he points out, the CDC model is one of the lower, not higher, estimates. Everyone involved in this fairly difficult and somewhat arcane exercise are aware of the problems. Different estimates from different models and methods is not special to influenza. It is the rule rather than the exception in science. I know of no one except Doshi who advocates merely using the cause of death on the death certificate as the best basis for resolving the questions.
In the bird flu world, the headlines were made by his dismissive attitude to the threat of a pandemic. Using his method whereby a death is only caused by influenza if it says so on the death certificate, Doshi compares pandemic years with non-pandemic years, and finds that except for 1918 there is no clear distinction. Even on his terms, it is clear the pandemic years are among the highest but his point is that they are not the only high mortality years. There are other years where “influenza classed” mortality (his definition) is as high or higher, although he doesn’t compare the pandemic years to all years but only selected years. It is not a cogent point in my opinion. CDC’s statement that the hallmark of a pandemic is excess mortality is still true. It is just not exclusively true of pandemic years, even if we agreed on his definition of pandemic mortality.
More important, however, is a point he makes with his own data: not all pandemics are alike. The three pandemics of the last century all had significantly increased mortality but varied in severity. The reason the flu world is concerned about H5N1 is the extraordinary virulence of the virus as expressed by its case fatality ratio (Doshi mistakes pathogenicity for virulence but this is a minor point). If this flu virus becomes transmissible, even with a major reduction in virulence, the results would be horrific. We hope it won’t happen. But we have no way of knowing. Preparing for it is prudent and it would be insanity to look the other way, although that is precisely what Doshi’s hand waving dismissal encourages. Handwaving — and rank speculation that things aren’t as bad as they look — is the hallmark of Doshi’s own claims that a pandemic “may not be” a catastrophic event. True, but trivial. A pandemic may be a catastrophic event as well, and given the extreme brittleness of the medical care system (a brittleness Doshi does not seem to recognize or understand), it wouldn’t take an outlier event to make an already full bucket overflow.
This paper is a pointless exercise. Too bad it isn’t also a harmless one.