One frequently hears claims that the current swine flu pandemic has been exaggerated because there are "only" 1000 or so deaths, while seasonal flu is estimated to contribute to tens of thousands of deaths a year. There are two reasons why this is not an apt comparison. We've discussed both here fairly often. The first is that the epidemiology of a pandemic and seasonal flu are very different. Epidemiology studies the patterns of disease in the population and swine flu is hitting -- and killing -- a very different demographic from seasonal flu. Its victims are young and many are vigorous and healthy. The second is that it compares apples to oranges. The 1000 deaths figure is for laboratory confirmed swine flu deaths (as are the various case counts), while the seasonal flu figure is an estimate, not a count of laboratory confirmed influenza deaths (see our post here if you want to know more about how the sausage is made). CDC and the states stopped counting cases early in the pandemic (here for some commentary from us), so we don't know how many cases there have really been. CDC keeps track of the general trends and patterns through a multi-part surveillance system. But for planning and resource allocation it would still be nice to know how much flu there is. Now a paper has appeared in Emerging Infectious Diseases that provides us with some rough and ready estimates. It also explains why this number is so hard to get.
Here's the set-up:
Human cases of influenza A pandemic (H1N1) 2009 were first identified in the United States in April 2009. By the end of July, >40,000 laboratory-confirmed infections had been reported, representing only a fraction of total cases. Persons with influenza may not be included in reported counts for a variety of reasons, including the following: not all ill persons seek medical care and have a specimen collected, not all specimens are sent to a public health laboratory for confirmatory testing with reverse transcription-PCR (rapid point-of-care testing can not differentiate pandemic (H1N1) 2009 from other strains), and not all specimens will give positive results because of the timing of collection or the quality of the specimen. (Reed et al., Emerging Infections Diseases [Epub ahead of print; DOI: 10.3201/eid1512.091413], cites omitted)
In order to make an estimate, the researchers (from CDC and Harvard School of Public Health) used a very simple multiplier method coupled with a Monte Carlo simulation. We'll explain both. First the multiplier method.
The multiplier method sequentially adjusts for the loss in each of the five steps a case has to climb in order to be counted: seeking care for ILI (A), specimen collection (B), submission of specimens for confirmation (C), laboratory detection of pandemic (H1N1) 2009 (D), and reporting of confirmed cases (E). Take the first step (A), care seeking behavior. What proportion of all cases of influenza-like illness (ILI) have a specimen taken? If you don't go to a health care facility or doctor for your illness you will never have a specimen taken and won't be counted. So in order to figure out how many ILI cases will pass this first barrier to being counted, the authors checked published and unpublished studies where such information could be estimated. For example, in 2007 CDC did a community survey in 10 states on ILI and health--seeking behavior as part of their ongoing Behavioral Risk Factor Surveillance Survey (BRFSS). BRFSS is done in all states but each state can add items to a core set. Ten states added items on ILI and care seeking. In May 2009, at the start of the pandemic, the 10 states and CDC repeated these surveys, supplementing it with special field studies done in Chicago and Delaware. That provided some basic information from a random sample in 10 states of how likely someone was to seek care if they had ILI, both in a "normal" flu season (2007) and during the early stages of the swine flu pandemic (May 2009). Early in the pandemic doctors and health departments were urged to collect clinical specimens from all suspect ILI cases, but the testing burden rapidly overwhelmed the states. On May 12, CDC guidance was revised to concentrate on hospitalized patients. The authors therefore used different estimates for the proportion seeking care and getting tested (step A) after that date. And because they suspected hospitalized patients presented with more serious illness and were more likely to be tested, they estimated prevalences from hospitalized and non-hospitalized lab confirmed cases separately, using higher multipliers for the non-hospitalized patients.
To give you some idea of the multiplier data and their source, here are steps (A) and (B) as given in Table 1:
(A) Proportion of persons with influenza who seek medical care: 42%, from 2007 BRFSS; 52 - 55% from 2009 BRFSS, 49 - 58% Delaware University survey; 52%, Chicago community survey.
(B) Proportion of persons seeking care with a specimen collected: 25% (2007 BRFSS), 22- 28% (2009 BRFSS) 19 - 34% (Delaware survey).
And so on for the rest of the steps. The multipliers are 1.0 divided by the proportions. For example, if the proportion seeking care was 50%, the multiplier would be 2 (1/0.5).
Although I illustrated the process at steps A and B, the authors had to work backwards from step E, the number of reported laboratory confirmed cases. That's the number CDC gives out to the public. On July 23, 2009 there were a total of 43,677 lab confirmed cases reported to CDC by the states, including 5009 hospitalizations and 302 deaths. These are the "hard data" they have for this. But if you have a multiplier for that step you can get a number for step D, and using step D's multiplier you can get steps, C, B and A. Finally step A's multiplier gives you the estimated number of swine flu over the period covered. This is the estimated prevalence for swine flu in the US from April to July 2009 (it is thus a period prevalence, not a point prevalence).
Now these data give ranges for the proportions and are not precise, so there are uncertainties in the underlying number. That's where the Monte Carlo method comes in. As the name suggests, this is a probabilistic method, but it isn't difficult to explain. If you are familiar with spread sheets, you can easily see how you might use the proportions in formulas to "back out" from the lab confirmed flu number to the number you want. Of course you could easily do this by hand, but the Monte Carlo method does this over and over again, thousands of times, so a computer is needed. The Monte Carlo method actually does use a spread sheet to do this, but it randomly picks a proportion for each step from a range suggested by the data. For example, in step B (proportion of specimens taken for non hospitalized patients) the proportion was randomly picked (uniform distribution) from the range 19 - 34% and for step C (proportion of specimens collected sent for confirmatory PCR) it was randomly selected from the interval 20 - 30% before May 12 and 5 - 15% after May 12. If you do this over and over again you will get a different set of 5 multipliers each time (because they are randomly selected) and you can run the spread sheet to get a final number (multiplier times result of step A). The authors did this 10,000 times and got a range of numbers for the final result of backing out all the multiplier steps. These are the ranges given by the paper. The same multiplier method was done for specific age groups but there wasn't enough information to use the Monte Carlo method.
So what were the results?
We demonstrate that the reported cases of laboratory confirmed pandemic (H1N1) 2009 are likely a substantial underestimation of the total number of actual illnesses that occurred in the community during the spring of 2009. We estimate that through July 23, 2009, from 1.8 million to 5.7 million symptomatic cases of pandemic (H1N1) 2009 occurred in the United States, resulting in 9,000-21,000 hospitalizations. We did not estimate the number of deaths directly from our model, but among reports of laboratory-confirmed cases though July 23, the ratio of deaths to hospitalizations was 6%. When applying this fraction to the number of hospitalizations calculated from the model—that is, by assuming that deaths and hospitalizations are underreported to the same extent—we obtain a median estimate of 800 deaths (90% range 550 - 1,300) during this same period. (Reed et al., EID [cites omitted])
The median multiplier for reported to estimated cases was 79. In other words, for every reported, lab confirmed case there were 79 total cases. That median estimate gives about 3 million cases. The 90% range was 47 - 148, meaning that each reported case could represent somewhere between 47 to 148 unreported cases. That's gives the 1.8 million - 5.7 million figure.
There are also age-specific incidence estimates, using the multiplier method and census data on underlying populations for each age group. These give a median of 107 cases per 100,000 persons in the 65+ age group, 2196/100,000 in the 5 - 24 year old age group. Hospitalization rates were highest in young children (median 13/100,000 children under age 5). Once again, the extraordinary susceptibility of the younger age groups is evident.
There are soft spots in this back-of-the-spreadsheet type of analysis, of course, which the authors are explicit about. For example, the multipliers are derived from people with ILI (fever with cough or sore throat without any other known cause). Not everyone with flu has fever or respiratory symptoms like that, so that would underestimate the true number. But it's a reasonable way to get a handle on something that at first seems impossibly elusive.
The tool is up on CDC's website at http://www.cdc.gov/h1n1flu/tools
- Log in to post comments
Excellent summary, important paper.
People often complain about bootstrapping, Monte Carlo, simulation in general, but that is not the real limitation of this sort of study. The methods used to get the estimates, ranges, confidence limits use sampling to allow the data itself to control how the confidence limits are estimate, in contrast to what are often thought of as more traditional methods which ultimately are the same but use data that have nothing to do with the current problem generated by hand a century or so ago to come up with p-values that half a century ago were routinely looked up on a paper table and more recently looked up (or regenerated) inside a computer.
The real difficulty with this sort of estimation is that it is extrapolation but there is not much else you can do.
Since it is Friday, I'd like to ask an Ira Flatow question: If you had unlimited funds (or at least a lot of funds) what new things would you implement to improve surveillance? Sample every tenth person who walks into any clinic? A huge national test population that you get weekly or monthly data on? Cameras in all the drug stores?
Second question: Was that "first wave" a "wave" or an arbitrary time unit.
Greg: I think my first wish would be a "lab on a chip" that could do 30 minute diagnoses for the main flu subtypes and other respiratory viruses (RSV, metapneumovirus, adenovirus, etc.). I'd deploy them in the sentinnel physicians' offices now part of the NRVESS system. This may not be too far in the future, so it wouldn't take unlimited funds. The aim would be to get rid of ILI and substitute the true cause agents as diagnoses.
The "wave" business is arbitrary, but intuitively it corresponds to modes in the epidemic curve. Some flu epidemiologists don't believe in waves as a dynamic entity and consider it just a descriptive term. I don't have an opinion about that.
"...and consider it just a descriptive term."
Why, so it is. How astute! And how pedantic. Just as "disease" is a "term" that describes a condition in the body. I doubt that there is really much chance of the usage of "wave" as a descriptor being displaced, here, any time soon. Lots of "terms" out there are arbitrary, but that alone doesn't necessarily undermine their utility.
The number of confirmed swine flu cases continues to rise as 73,000 cases were reported in the UK last week. Swine Flu UK
Dylan: No, I think it's a bit more substantive. So here's an example. Consider a weight hanging from a spring. If I pull down on it the weight and let go it will bounce up and down at a frequency (wave) that is determined by the force exerted by the spring as a function of how far from the resting point the weight is. It's something inherent to the weight spring system. But suppose instead I just have a weight that my hand moves up and down according to some rule that I determine (e.g., whether it is spring or fall). Then I and the season are driving the weight's waviness, not something inherent to the system. We know that certain kinds of infectious disease dynamics will generate waviness, so that is like the weight spring system. However it might also be some forcing function determined by the environment or social relations (e.g., the return to school). We would approach these two possibilities differently, so it isn't just a terminology question.
Quick question about the proportion of patients with specimen collection (step B). This comes from the BRFSS, but isn't that dependent on telephone interviews? If a patient goes to the clinic, he or she may not know whether a specimen was collected, only that the doctor said to open wide and say "Aaah." If the doctor did a swab for a rapid strep test rather than for an influenza typing, will the patient be certain what kind of specimen was taken?
For a look at the number of confirmed swine flu cases (reported by CDC & WHO) in the US & the world, check out:
http://www.peterdolph.com/2009/10/how-many-swine-flu-cases-are-there.ht…
Revere: I think that I detect some level of "hand waving," here. If you guys don't have a lock on this stuff, yet (and no epidemiological consensus has emerged), then I suggest that you get busy and dredge one up. You've only had a couple of decades to work on it, it would seem.
Dylan: This is an active area of research for mathematical modelers, but it isn't easy. The systems of equations are intractable and the mathematics very difficult (these are non-linear systems). The handwaving was used to illustrate there is a real problem here, not for giving an answer.
It's a really interesting paper. I wonder if it would be possible to further estimate the number of cases to date by comparing the values of some routine surveillance measure(s) during that period with the cumulative values since the end of July. For example, can you take the percentage of outpatient ILI visits to sentinel providers during that period and compare the sum of those weekly values with the sum of weekly values since the end of July and use that ratio to get some approximation of how much of the population has become ill from H1N1 since April 17. I noticed that at least one of the paper's authors reads this blog, so maybe they'd be willing to comment.
> estimate
Google Scholar led to, e.g.,
Statistics in Medicine
Volume 28 Issue 14, Pages 1882 - 1895
Published Online: 22 Apr 2009
Digital Object Identifier (DOI) 10.1002/sim.3592
Research Article
Estimation of incidences of infectious diseases based on antibody measurements
Abstract
Owing to underascertainment it is difficult if not impossible to determine the incidence of a given disease based on cases notified to routine public health surveillance. This is especially true for diseases that are often present in mild forms as for example diarrhoea caused by foodborne bacterial infections. This study presents a Bayesian approach for obtaining incidence estimates by use of measurements of serum antibodies against Salmonella from a cross-sectional study. By comparing these measurements with antibody measurements from a follow-up study of infected individuals it was possible to estimate the time since last infection for each individual in the cross-sectional study. These time estimates were then converted into incidence estimates. Information about the incidence of Salmonella infections in Denmark was obtained by using blood samples from 1780 persons. The estimated incidence was about 0.094 infections per person year. This number corresponds to 325 infections per culture-confirmed case captured in the Danish national surveillance system. We present a novel approach, termed as seroincidence, that has potentials to compare the sensitivity of public health surveillance between different populations, countries and over time. Copyright © 2009 John Wiley & Sons, Ltd.
Revere, re Monte Carlo method, you note "it randomly picks a proportion for each step from a range suggested by the data. . . [and each time gets] a different set of 5 multipliers . . . The authors did this 10,000 times and got a range of numbers. . ." Presumably a good program can do this quickly, and I admit my math has gone rusty since my math 190s days, but couldn't one get vaguely the same range of results if--assuming the five curves are fairly smooth and not bunched somewhere (or even then, if one can adjust for this)--one picks the upper and lower bounds (minus some intuitively sensible portion of end-points) and multiplies these in one's head? I mean, if one needs a quick estimate at a given moment?
Off-topic again, but, with Obama this afternoon "expressing frustration" over the rate of vaccine delivery, the Canadians are expecting 1/6 to 1/5 of their population to be vaccinated by the end of next week. Maybe they (too) are being overoptimistic, but if only we had used the adjuvants.
Paula: No, that wouldn't work. We are dealing with joint probabilities. If you picked the highest and lowest in each range you would have the extremes, but they would be so unlikely (that all the extremes are relevant simultaneously) you wouldn't know how far off you would be. In this case the extremes are the 90% marks, that is, the values in which 90% of the results fell, a much better idea of what is likely to happen. The cost of doing the Monte Carlo on a spread sheet is negligible. You can buy programs like Crystal Ball or @Risk that sit on top of your Excel spreadsheet and carry out all the calculations in seconds or a minute or so. Or you can use the tool on the CDC website, whose link is at the end of the post.
Thanks, Revere--I stand corrected. (Of course, what I *really meant* to say was that if one looks at the curve and then. . . . Or whatever.) OTOH, I'm quite sure those Canadian expected-percentage-vaccinated a week from now are quite ahead of the U.S.'s (if they happen).
You have stated that "you have no opinion;" yet you appear to articulate one, here; clarify, if you do not mind. I want to know what you think about this. Not what the unresolved argument suggests. I want to look at your reasoning.
Dylan: No, I truthfully don't have an opinion because at this point I don't have the basis to have one. The information to decide this just isn't available -- to anyone, ASFAIK. Some people who work on this have made educated guesses, although IMO most people who talk about "waves", including me, do it very imprecisely. At the level it is normally used, I don't think that's much of a problem, so you and I probably don't disagree. But there is a real scientific question involved that at the moment doesn't have an answer, like a lot of scientific questions. I haven't articulated an opinion, just tried to articulate the question.
If I had to guess I'd say both were involved, some inherent in the dynamics, and in some places, an external forcing function. Unfortunately those two things can interact in ways that are very difficult to predict. In some circumstances they might produce a "wave-like" appearance, in others not. At the moment I don't know anyone who can predict this.
The latest CDC graph of the weekly and cumulative case counts (seen in the penultimate graph on this page: http://www.cdc.gov/flu/weekly/ which is an updated version of the one you posted 2 weeks ago â the one you complained about the resolution) *seems* to show a drop off in cases in the most recent reporting week.
Is this real (even if potentially spurious)? Or does it merely reflect incomplete reporting that will be added to next week? I noticed that GAâs hospitalization numbers for each of the last two weeks are about 1/2 the level they were at from early Sept to mid Oct, and so wonder if this might be the start of an new trend.
Revere
I am an italian pediatrician. Excuse me for my bad english but I try to explain my point of view.
Analizing MMWR data, I counted (by hand) the number of deaths, age-specific, both total and for P&I in the 2009,2008,2007,2006,2005,2004,2003 years.
I compared 20-39 weeks, 35-39 weeks, 1-19 weeks of each year.
I found that mortality for P&I in 20-39 w is about 1000 higher than we would expect in 2009 compared to previous years , and that 45-64 total mortality is about 2000 higher compared to previous years (others group is normal or lower). I observed that 45-64 group are 2000 higher also in 1-19 weeks compared to previous years ( with the exception of 2008, but in 2008 there was a high P&I mortality).Elderly (>65) mortality seems to have a normal trend.
Also, it is useful to look at P&I graphic. As you previously observed, 2009 trend after 20 weeks is higher than we would expect by a year in which P&I mortality was low in the peak period ( 2008 was very different ).
122 cities mortality represents about 25% of total american mortality, but, I cannot explain why, 122 cities P&I mortality represents 66% of total mortality, as I found reading national vital statistic report and comparing total P&I mortality of 2006 and 122 cities mortality of the same year.
P&I mortality represents only a part of flu related deaths. There are many deaths masked by others causes, expecially cardiovascular diseases (up to 50% according to recent english study)
In conclusion my conjecture is that in 20-39 weeks real mortality was higher than official reports and regards expecially 45-64 age group. My surprise is that this grooup have higher mortality since from first weeks of 2009 ( not in the last weeks of 2008). I know that virus was circulating (unknown) since from the beginning of 2009 and perhaps the end of 2008. 122 cities represents only a quarter of total national mortality, so the excess of mortality could be much higher.
stefano: P&I mortality is an imperfect but useful indicator of influenza-related mortality. As you observe, not all influenza-related mortality is in P&I, but also not all (or even most) of P&I mortality is influenza. P&I goes up and down throughout the year, and what is modeled as excess mortality from flu are excursions above one std deviation of that modeled wave pattern. Thus one can have zero excess mortality but still a lot of flu related mortality if the variation caused by flu season doesn't push the usual variation (which includes influenza mortality as a component) past one std deviation. Because of the seasonal variation in P&I it can be difficult to see details in these patterns when comparing year to year, where lots of other things are also different each year (the co-circulating subtypes of seasonal flu differ each year and the timing of which one is dominant also differs; remember influenza B is also in the mix). But your main conclusion is certainly correct: something different is happening this year. Whether we can see it in the first 20 weeks of 2009 or not is more difficult to say. You may be actually seeing a signal there, but there are also other explanations. But keep thinking about this. There is no reason why you won't come up with a good explanation. You have the same data as everyone else, and one thing I have learned over the years is that there is a huge reservoir of raw brain power "out there" that is not utilized by gov't agencies and public health in general. The internet allows many more smart people to participate.
And your English -- which is fine -- is much better than my Italian! Certo!
Revere when I read this post this bit struck me "the proportion was randomly picked (uniform distribution) from the range 19 - 34% and for step C". It is the assumption of a uniform distribution that I found odd. I see you and Paula discussed it a little above and that in this instance there were so few cities collecting the data using any other distribution may not have been possible. Intuitively I would of thought a normal distribution would have been more likely and in other surveys (with a bigger data set) has this not been found to be the case and would it not therefore be a more logical default to assume?
http://www.americanscientist.org/science/pub/-179
"H1N1 Cases Vastly Underreported, CDC Says
from the Wall Street Journal
WASHINGTON -- The number of confirmed cases of H1N1 flu from April to July represents just 2% of the actual people who were infected with the virus, according to a report by the Center for Disease Control and Prevention.... posted to the agency's Web site ....
(I'd guess this is the same information that inspired the original post here; just noting the appearance in the press)
JJ: The software allows you to pick the distribution. Uniform is often the default. If you pick a normal distribution you are assuming that the ends of the intervals are unlikely and that is an assumption that is not warranted in this case. I assume that is the reasoning, but you are correct that this is a strong assumption, although probably as good as any.
Thanks - as always.
Revere
thank' you for your answer. I know that P&I mortality is only an indicator and doesn' t represent absolutely the real flu mortality. But I know that in the years in which P&I mortality is high, elderly mortality is high too and vice versa.
I looked at weeks from 20 (the beginning of pandemics) to 39 ( the last week).
In 2009 , from 20 to 39, we count 12986 deaths for influenza and pneumonia, in 2008 13431, in 2007 12176,...
But I searched for other data. I looked how many deaths in people over 65 there were in the same period and we find 134596 (2009), 139835 (2008), 134015 (2007),...
Moreover I looked at 31-39 weeks, the last 2 months, and I found for influenza and pneumonia 5436, 5517,5248,.....deaths
These data seems a little blundering, but if we analyze them, we can do some remarks.
First, in 2008 there were 4500 more deaths than 2009 in over 65 (138928 versus 134596). But if we look at deaths for P&I in 20-39 weeks we find 13431 (2008) versus 12986 (2009). it isn' t a big difference. Moreover if we look at mortality in 31-39 weeks mortality of 2009 is near 2008 (5436 versus 5517). It means that in the last months the mortality for H1N1 is increasing compared to 2008
If we looks at others years we find that in other 2 years elder mortality was similar at 2009. In 2007 it was 134015 and in 2006 134852, What happened in these years? In 2007 P&I deaths were 12176 (5248 in 31-39 weks) and in 2006 12030 (4879 in 31-39). Apparently it is strange , becouse 2007 in 31-39 w have about the same deaths as 2009, while 2006 much less . I had the idea to look the graphic of P&I mortality and here is the solution
If you consider the weeks from 35 to 40, there was an excess of mortality, probably owing to some virus circulating in that period. Effectively in 31-39 weeks we have 5248 deaths and this explains why in 2007 we have a similar mortality as 2009.
However in 2007 and in 2006 we have about 1000 less deaths compared to 2009 in spite of the same elderly mortality .
2005, 2004 and 2003 data are similar to 2008.
In conclusion in 2009 we have about 1000 excess of mortality compared at similar years, and about the same mortality compared with years with 4-5000 more elderly death.
I searched for mortality in the different class age, and the only one in which mortality is increasing in 20-39 w of 2009 is 45-64, and so it happens in 1-19 w.
in this forum I published a table with my data:
http://www.swineflu.org/forum_posts.asp?TID=30495&PN=1
At some point, (I assume) we will have a serologic test that can give us a good count of how many have been infected. At that point, we should be able to tighten up the confidence limits. Until then, anything is going to be in the nature of a SWAG, and this seems about as well thought out as any. Thanks for the explanation.
The real difficulty is that it is extrapolation..