Transmission, pathogenicity, virulence and vaccines, part II.

CDC wants us to get vaccinated for flu every year. Always for seasonal flu, and this year, if there is a vaccine available, for swine flu. They want us to get vaccinated because they think the vaccine works and they want to prevent people from getting influenza, always a dangerous and unpredictable disease, even if most of us usually escape with just a flesh wound. CDC backs up its recommendations by a quite a few scientific studies demonstrating the vaccine is effective, citing figures that the vaccine is 58% effective or 91% or effective or some other number, depending on what group is being talked about. This post is not about contradicting CDC, since I mostly agree that flu vaccination programs are sound public health. It is about clarifying some things that are glossed over when CDC talks about vaccine efficacy and explaining why this is not such an easy thing to figure out.

There are various ways to evaluate whether a vaccine works or not. If it is a vaccine for a communicable disease that hasn't yet circulated in the community, like bird flu, the best you can do is rely on laboratory values as surrogates. The ability to raise antibodies in human volunteers is an example. When new vaccines are tested for safety and efficacy in the licensing process, this is the end point. But the real bottom line is whether it works to prevent disease in people living in the real world. Epidemiology and biostatistics are the public health specialties that try to figure this out.

As an epidemiologist I know many non-epidemiologist colleagues think that all we do is count things. We do count things, it's true (although we also do things that don't depend on counting), but the trick is not in counting them but in figuring out who and what to count, where to find the things to count once you figured what to count, when to count them, how to count them, how to organize what you've counted in a way to reveal the patterns you are interested in, and finally trying to figure out what the counts mean. Let's take a look at vaccine efficacy as an example.

First question: what does it mean for a vaccine to work? That seems pretty straightforward, although you might have to ponder a bit about the question of "works for whom?" Children, the elderly, pregnant women? You get the idea. But it turns out that's just for starters. Because even for flu, there is more than one vaccine. In the US, for example, there is an attenuated live virus vaccine that is sprayed in the nose and causes a very mild infection that in turn causes an immune response. Most people are familiar with another kind of vaccine that is injected into a muscle. This is an inactivated (no live virus) vaccine that is composed of only the parts of the flu virus that are good for provoking an immune response. We also know that vaccines work differently in different age groups. Now we are talking about additives called adjuvants that boost the immune response and allows less viral product and hence makes the vaccine go further. There's more than one kind of adjuvant and each may be used at several doses. Right away we are starting to see an impressively large number of different combinations.

Then some people are already immune to flu (maybe they were vaccinated or had a case of the flu recently), so you should check to see if any of your subjects have signs of immunity before they even get the vaccine. Once you've got all this sorted out (I didn't give you the full list of things you need to think about; just enough to give you the general idea), then you are ready to see if the vaccine "works" in the particular population you've selected. Usually you want to restrict your test subjects to some extent because otherwise there are too many variables to keep track of.

So let's ask again, what does it mean for a vaccine to work? If you have a bunch of people you give the vaccine to and another group you don't, you might think you are done. You simply observe them to see if there is a difference. Observe them for what? When? How? A paper last year by Basta, Halloran et al. reviewing and summarizing results from many different kinds of flu vaccine studies will give you some idea of the problems involved. Elizabeth Halloran is a biostatistician whose academic specialty is evaluating vaccine efficacy. There aren't many who know more about it than she does. She points out at least four different outcomes you could use to evaluate a vaccine, and they correspond roughly to the terms we talked about in yesterday's post: transmissibility, pathogenicity and virulence. Not exactly, but close enough that what we said there is applicable to what Halloran did.

Here are the four ways Halloran notes that a vaccine could be beneficial. It could make people less susceptible to the flu virus in the sense that they'd be exposed to it but wouldn't get infected. It could make people who do get infected not get sick (decrease pathogenicity). We know that many people are infected but asymptomatic and a vaccine could increase that proportion. The vaccine could increase the chance that if you get sick it's not so bad (decrease virulence). You are able to fight off the virus and get back on your feet more quickly. And independently of all that, the vaccine might make you less infectious to others, regardless of how sick you yourself might be (decrease transmissibility). When people talk about vaccine efficacy they should also tell you: effective for what end point. Usually you have to examine the study to see, and most often it's the second category, prevention of people being simultaneously infected and sick with flu symptoms or sometimes serious flu complications, like hospitalization for pneumonia. You also want to know what the comparison is. It could be comparing live virus vaccine to inactivated vaccines (this is called relative efficacy) or either kind of vaccine to no vaccine at all (absolute efficacy).

How would you express efficacy, whatever outcome you used? The most natural way is through some kind of relative risk measure. In this case a relative risk is the risk of getting flu in the vaccinated group compared to the unvaccinated group (NB: in epidemiology a relative risk is a kind of effect measure; other effect measures are risk differences and odds ratios. This blog is named Effect Measure after this term of art in epidemiology). If the vaccine were completely effective, i.e., no who was vaccinated got the flu, then the risk in the vaccinated group would be zero and some other, positive risk in the unvaccinated group (assuming both groups were exposed to flu virus, either deliberately or naturally). The relative risk (RR) would be zero (zero divided by some other number). Vaccine efficacy is (1 - RR) times 100%, so if the RR was zero (a completely effective vaccine), the vaccine efficacy would be 100%. If there was no effect, the risk in each group would be the same, the RR=1, and the vaccine efficacy would be 0%. Of course there is always some random variation, and being able to come up with a good estimate with some notion of how much uncertainty there is is how biostatisticians like Halloran make a living.

More important, though, vaccines are hardly ever 100% effective, and this is especially true of flu vaccine. You can get vaccinated and still get the flu, or get infected, or be contagious, or whatever the endpoint is. It is not a guarantee for an individual. But what we know from the public health standpoint, where the measure is the population, the flu vaccine has a major effect on influenza in the community. We know this from numerous studies that have been done demonstrating it. I've hardly scratched the surface of this business. If you want to pursue it you could start with the Basta, Halloran et al. paper in the American Journal of Epidemiology (Am J Epidemiol. 2008 Dec 15;168(12):1343-52) and move on to look at individual studies. There are all kinds of them: double blind randomized trials where people are deliberately infected ("challenged" is the euphemism) with flu virus, randomized trials of vaccines in the community during flu season, sometimes where it turns out one or more vaccine component is mismatched to the circulating virus, etc. There are all sorts of pitfalls and problems in doing these studies, including how accurately endpoints are determined (which may depend on illness definitions or laboratory methods that vary from study to study), people initially enrolled in the study dropping out and lost to follow-up and much more. So there are holes in what we know, but to give you a flavor of the kinds of vaccine efficacies being seen, here is the abstract of the Basta, Halloran et al. paper:

In this paper, the authors provide estimates of 4 measures of vaccine efficacy for live, attenuated and inactivated influenza vaccine based on secondary analysis of 5 experimental influenza challenge studies in seronegative adults and community-based vaccine trials. The 4 vaccine efficacy measures are for susceptibility (VE(S)), symptomatic illness given infection (VE(P)), infection and illness (VE(SP)), and infectiousness (VE(I)). The authors also propose a combined (VE(C)) measure of the reduction in transmission in the entire population based on all of the above efficacy measures. Live influenza vaccine and inactivated vaccine provided similar protection against laboratory-confirmed infection (for live vaccine: VE(S) = 41%, 95% confidence interval (CI): 15, 66; for inactivated vaccine: VE(S) = 43%, 95% CI: 8, 79). Live vaccine had a higher efficacy for illness given infection (VE(P) = 67%, 95% CI: 24, 100) than inactivated vaccine (VE(P) = 29%, 95% CI: -19, 76), although the difference was not statistically significant. VE(SP) for the live vaccine was higher than for the inactivated vaccine. VE(I) estimates were particularly low for these influenza vaccines. VE(SP) and VE(C) can remain high for both vaccines, even when VE(I) is relatively low, as long as the other 2 measures of vaccine efficacy are relatively high. (Basta et al., AJE [2008])

Let's see what some of this means. If you pick your way through this abstract (even better, read the paper if you have access to a medical library), you will see that using laboratory-confirmed infection, either live or inactivated flu virus has an efficacy of 40% - 45%. This means that the vaccinated group had less than half the amount of lab confirmed flu than the unvaccinated group. This is clearly a Big Win from the public health perspective -- half as much flu -- but for an individual, it means your chance of a win is about 50-50. That's like flipping a fair coin with heads, a win, tails, a loss. That doesn't sound so wonderful, but remember what the comparison is. The unvaccinated group's coin has tails on both sides. It's pretty clear to me which coin I'd rather play with, so I get a flu shot every year. But also remember what the end point is: infection with flu virus, not getting sick from the infection. Once you start parsing the outcomes (you have to dig into the paper for this) you get studies where the vaccine is well matched or mismatched and the outcome is infected plus sick or something else. Things get more complicated. You start to see differences in age categories and differences related to how the outcome was determined. For example, for children, the live virus vaccine seemed to be much more effective, but the reverse is true for adults. For adults the live vaccine had efficacy of about 30%, the inactivated (injected) one, 67%. Moreover this figure was for a study done in 2004-2005 where the vaccine was mismatched, so even in this case vaccination was doing pretty well. For some combinations the efficacy is 90% (for example, well matched live virus vaccine in a challenge study with illness as an endpoint). And some of the biggest uncertainties has to do with people in my age category, over 65, where the data are sparse and there is some evidence we don't respond to vaccines as well as younger folks. Still, I get a flu shot every year. I'm playing the odds. If you are under 60, there's no question. You should get vaccinated against influenza. You'll come out ahead even if the vaccine is mismatched, which it sometimes is, although not often.

The Basta, Halloran et al. paper has lots of other summaries for different combinations which we won't give you because it would get too confusing and isn't the main point of this post -- which is: estimating whether a vaccine works or not is a surprisingly difficult business, even during a pandemic.

More like this

by revere, cross-posted from Effect Measure CDC wants us to get vaccinated for flu every year. Always for seasonal flu, and this year, if there is a vaccine available, for swine flu. They want us to get vaccinated because they think the vaccine works and they want to prevent people from getting…
by revere, cross-posted from Effect Measure I just got my seasonal flu shot. It was free and my medical center is encouraging everyone to get one. I wouldn't be telling the truth if I said I didn't feel it at all, but in all honesty, I hardly felt it. They must be using smaller needles these days…
I just got my seasonal flu shot. It was free and my medical center is encouraging everyone to get one. I wouldn't be telling the truth if I said I didn't feel it at all, but in all honesty, I hardly felt it. They must be using smaller needles these days. Anyway, given that most circulating flu…
It looks like this year's going to be a bad flu season. Hard as it is for me to believe, it was only five weeks ago when I discussed an announcement by the CDC that this year's flu vaccine would likely be less effective because it isn't a good match for the influenza strains in circulation this…

Thanks, Revere. I'd like to go stick my head back in the sand now, but I've been scared of sand ever since you made sand sound dangerous in your "Viruses and Sandcastles at the Beach" post!

Revere,
In addition to effectiveness is the issue of safety, especially with an adjuvanted novel H1N1 vaccine. One would expect HHS and FDA to have 1976 in mind as they try to instill confidence in the public that a 2009 swine flu vaccine is safe. Yet, over the weekend, WHO's Keiji Fukuda felt compelled to caution against cutting corners on safety testing, and, today, Helen Branswell wrote a piece, "Boosting compounds called adjuvants complicate licensing of pandemic vaccines," that ended as follows: "In the U.S., the Food and Drug Administration will use a mechanism called an emergency use authorization or EUA if the U.S. administration decides adjuvants must be used there. That decision, FDA officials signalled at a vaccine advisory committee meeting last week, will be a political one, not one that rests with the regulatory agency." That FDA disclaimer does not fill one with confidence.

SoCal: Well, yes, it's politics, but of a special sort. There is a good interview on the subject with Laurie Garrett over at ScienceInsider where she raises some provocative points of a political nature regarding adjuvants that bear some consideration. This is a very difficult situation. There can never be enough pre-use testing to prevent events too rare to be detected that way but too frequent to be a big problem when millions are vaccinated. The Guillain Barre problem would not have been picked up by safety testing. It was too rare. It showed up when 44 million people were vaccinated. There is no easy answer.

revere,

Sorry this is going to sound so sappy and sycophantic, but I just can't not express how reassuring and enlightening it is to read your posts - this one being a remarkable example. Your ability to *teach* (explain, make meaningful, pass on wisdom) is just exemplary! Knowing your age, seeing so many demonstrations of how well read you are, how sharp, such undiminished ardency, even at this stage in your career, when others might be winding down or just coasting - is not only inspirational, but a privilege having you share your acquired body of knowledge so clearly, uncondescendingly, freely, almost lovingly. If only there had been professors such as yourself in my medical school.

I realize I'm addressing a group named revere (maybe; I truly wonder because of the consistent quality of this blog's product), but I just couldn't, in all honesty, let this opportunity pass, to express my profound gratitude for the time and effort y'all sacrifice to help the rest of us prepare, mentally and emotionally, for what may be threatening us and our loved ones.

And I know you and this crowd won't appreciate my sharing what I'm about to say, but too bad: I thank G-d for you and your commitment to this blog, and that I came across it. (Had to go and ruin it all, didnât I ?) ;-)

Paul: I blush easily. But thank you. Anonymous bloggers don't get much reward since no one knows who we are, but it's nice to know what we write is useful. We want it to be.

I'm just going to toss some points out here FWIW.

1. The swine flu might well crowd out the seasonal flu. Ecology and what happened in Argentina.

2. An article I read claimed that Guillaim Barre was never scientificly linked to the old vaccine. If you vaccinate 44 million people, some will die within a week. If you give 44 million people a glass of water, some will die within a week.

3. AFAIK, the only licensed adjuvant in the USA is alum. The human papilloma vaccine of GSK has a Ribi Detox adjuvant but IIRC that is only licensed in Europe. Don't know about the other licensed HPV vaccine.

It would be a bad idea to toss in an unlicensed, unmass tested adjuvant. This is gambling and stuff like pandemics and vaccines aren't the place to gamble. If something does go wrong in a mass immunization campaign, I wouldn't want to be anywhere near it. If this was a real 1918 style pandemic or worse, maybe that would fly, Anything else, forget it.

I have no doubt that it is going to be crazy in NA this next flu season. Wait till it hits China and India.

Curious: Asthma seems by far the largest risk factor in swine flu complications. In addition, CDC stats show infection is highest with the under 24 group, but death rate is highest with 25-49 group. Is social distancing a good idea for asthmatics (especially in the 25-49 age group) even during the first wave of this outbreak, let alone the second, third waves? Does it matter how severe the asthma? I've read cases of people with only mild asthma dying from this virus. Is there any evidence of steroids and lowered immune system improving outcomes or worsening? Are there any asthmatics who have had this flu without severity?

Revere, what Paul and others say is so true. You are simply the best. I am a nurse and you have inspired me so much that I am thinking about getting a certificate in Epidemiology from my local (but famous) School of Public Health. Who knows? Maybe there's even an MPH in my future...

Thank you for all you do.

By Catharine (not verified) on 29 Jul 2009 #permalink

Keep blushing, Revere.

By D. C. Sessions (not verified) on 29 Jul 2009 #permalink

With regard to you post Revere, the CDCâs New Novel H1N1 Vaccine Rationing Plan was announced today and IMO has hit the nail on the head.

The committee has deliberated and has come up with an entirely responsible and appropriate recommendation. Frankly I am very relieved. They have chosen to give pregnant women and children aged 6 months through 18 years with the highest priority. Health care workers, first responders and the families of children less than 6 months of age are also included in the priority group.

It is easy for me to support the vaccination of children aged 6 months to 18 years of age. This is because they are those who are most responsible for spreading influenza within the population as a whole. Vaccinating them first above all else is clearly the most effective way to blunt the transmission rate of the pandemic strain.

The use of adjutants was not decided during this committee meeting. The CDC sees the addition of adjutants like scalene as a safety concern as has been discussed here.

What is clear though is that the use of a live attenuated influenza virus (LAIV) as produced by MedImmune which is administered by intranasal spray, does not require adjutants and provides those vaccinated with significantly better immunity than that obtained using killed influenza virus with or without an adjuvant. This company has been producing LAIV specific for the present novel strain within the US in very high quantities. A recent article on the firm reveals that the company has made so much vaccine that they have run out of nasal applicators used to administer the vaccine.

IMO, the LAIV vaccine administered intranasal is the most efficacious as well as most likely the safest way to be vaccinated against this pandemic. If you have a choice, my advice is to use the LAIV.

Grattan Woodson, MD

By The Doctor (not verified) on 29 Jul 2009 #permalink

Gratt: I pretty much agree with your assessment. I'm guessing live vaccine will be an important element, although there will be some public apprehension about it. A lot will depend on the actual situation.

Just adding to the thank-you. Your posts have been extremely helpful and informative as I try to learn as much as possible about the flu virus (and its implications for my family and for all of humanity). I find that I count on checking in with your expertise and perspective. Thank you.

I too thank God for you and your great service to all of us.

Just adding to the thank-you. Your posts have been extremely helpful and informative as I try to learn as much as possible about the flu virus (and its implications for my family and for all of humanity). I find that I count on checking in with your expertise and perspective. Thank you.

I too thank God for you and your great service to all of us.

you will see that using laboratory-confirmed infection, either live or inactivated flu virus has an efficacy of 40% - 45%

Not by my understanding. I feel that the 40-45% is way too precise sounding for the data.

I see it as an efficacy of 15-66% (the narrower of the confidence intervals). Perhaps one standard deviation is more fair - at a guess 28% to 54% - still a big range to me - we don't know if it stopped 1/4 or 1/2 of infections.

I'd say this shows how hard it is to measure vaccine effectiveness against wild influenza infections, even after you've made all the choices about who what and how to count (I think you wanted us to learn this). It seems influenza's variability makes quantifying a vaccine's effectiveness much harder than for less variable diseases. Lab challenge tests can give much more precise figures (if you can recruit enough subjects) but only against a selected strain, so only the hardest to get data is really relevant. I've learnt that "How effective is the flu vaccine" is much harder to answer than it looks.

davidp: It is customary or at least common to report the most likely value of the estimate (which in this case is the maximum likelihood estimat, technically speaking). Not all values in the confidence interval are of equal likelihood (in terms of coverage probability) so the ones at the end are less likely in those terms than those more central and the most likely is the point estimate (given the probability model and the observations). But you are correct that it is more informative to give a confidence interval, but for comparisons like this the point estimate is commonly used.

Hi Revere:

I'm late to the party on this one. But I'll throw out a parting point anyway.

One thing neither Halloran nor you mentioned is vaccine effects on severe outcomes (hospitalizations and deaths). The CDC strategy of aiming seasonal vaccination campaigns at high risk groups is to limit these outcomes, not to cut transmission by herd immunity (although by steadily adding to the recommended groups, that is changing).

Because these outcomes are rare, no trial in the world can tell you much about how a vaccine affects them. So epidemiologists fall back on various observational and ecological studies, with all the pitfalls inherent.

Those pitfalls have badly tripped up many. Example: Many observational studies have come to the wholly incredible conclusion that for your average person >65 years living in a community setting, getting a flushot cuts the risk of death from ANY CAUSE over the flu season by 50%. Very strange, given that the best estimates puts flu-related deaths among that population at ~5% of all winter deaths, maybe 10% in a bad year.

But that wild conclusion get repeated over and over--sometimes by authors in The Best Medical Journals just after lamenting the need for "more effective vaccines for the elderly." Better than a magic shot that cuts your average senior person's chance chance of dying between October and March by half?

Anyway, please keep up the good work. I second all that lavish praise above.

PS: the Halloran paper is available free at http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pubmed&pubmedi…

PPS: For an example of a commentator who hasn't thought things through, see October 4, 2007 NEJM, page 1429; http://content.nejm.org/cgi/content/extract/357/14/1439