Effect Measure

We only just got to the surgical/N95 mask article in the Journal of the American Medical Association (JAMA). We’ve been traveling and haven’t been able to keep up with what others were saying, but we’re sure it’s been well covered by the usual suspects. So we’ll just add our take here, for what it’s worth.

As most readers here know, what kind of mask (if any) will best protect a health care worker or anyone else at high risk of exposure to people infected with influenza virus is a difficult question. We still remain unsure whether flu is transmitted mainly by large or small droplets. If most transmission is by very large droplets produced by coughs and sneezes then a surgical mask might prove effective, but if it is mainly very small particles that remain suspended for long periods and have no difficulty finding their way around a loose fitting surgical mask it’s another matter. For that you want a respirator with a good seal. This means having a proper fit and knowing how to don it to make the seal. If you have facial hair (beard or mustache) you won’t be able to fit one of these respirators. They are also more expensive and not in great supply. So the question is great practical importance.

So how would you find out? You could try to conduct an experiment with two groups of health care workers, each wearing one kind of mask during the flu season and check to see if there is a difference in the incidence of flu infections. This sounds straightforward, but in fact such studies are quite difficult to do. The study reported in JAMA is a version of this kind of experiment involving 446 nurses in emergency departments, medical units, and pediatric units in 8 large hospitals in Ontario, Canada. This is a good place to do this kind of study because it was where many health care workers caught SARS in 2003 so there is a high sensitivity to the problem. SARS is a disease most contagious in the latter part of the illness, so health care workers were at special risk. Influenza is most contagious in the period shortly after symptom onset, so many hospitalized patients may not be at maximum contagion. The epidemiology is different.

So how was the study done? Nurses who worked full time in specified units likely to see flu patients were enrolled and randomly assigned to either wear a surgical mask or an N95 respirator whenever they cared for a patient with a respiratory disease with fever at any time during the 2008 – 2009 flu season (enrollments occurred between September and early December and the nurses were followed until the end of April when all switched to N95 respirators because of the onset of the swine flu pandemic). Audits of a small sample showed extremely good compliance. Post-SARS era nurses in the Toronto area take this seriously. Gloves and gowns were worn by both groups, a routine practice in these hospitals.

The type of mask was the variable being assessed against its ability to protect against influenza infection. The nurses were queried for symptoms of influenza-like illness twice weekly via a web-based system. When indicated, the nurse self-swabbed her or his nasopharynx via a specific protocol and these were tested for a variety of respiratory viruses via PCR. Pre-study blood sera was also collected and evaluated for a 4-fold rise in antibody titer against pandemic and seasona flu types and subtypes at the end of the study to verify clinical or occult infection. So what were the results?

One of the first things you do in a study of this type is to check to see if the two groups are roughly comparable. Randomization can fail in a variety of ways (including differential drop out of participants) but all of the usual measures showed an excellent comparability of the two groups (mean age of about 36 years old and 94% female). About 30% of each group had received seasonal flu vaccine. Obviously this is an important thing to check and if it turned out that there was a significant difference in proportion vaccinated, this would have to be taken into account. But there was very little difference (28.1% for the N95 group, 30.2% for the surgical mask group). What about lab confirmed influenza?

The figure for the surgical mask group was 23.6% and the N95 group 22.9%. These numbers are about as close as you can get, indicating no difference in outcome, at least by this measure. While there wasn’t any more infection in the surgical mask group, they did experience infection with fever slightly more often (5.66% versus .9%), although the numbers are quite small. In the nurses diagnosed by serology, 65.9% of the surgical mask group were asymptomatic while the number was 70.5% for the N95 group. One wonders, though, whether these findings might be a reflection of size of the viral inoculation, with N95s being more protective in that sense.

The design does not permit differentiating workplace and community acquisition of infection, nor is this study specific to swine flu, although it includes swine flu. It does strongly suggest but not prove that for this setting small aerosols weren’t dominating transmission. If it turns out to be true, that’s good news on several fronts. It makes it easier to protect health care workers from patients, patients from health care workers and all the rest of us can concentrate on the sneezers and coughers in our vicinity rather than the ones that were across the room the day before. And it adds emphasis to cough and sneeze hygiene.

This is not the end of the story, I am sure. But these are some of the best data points we have, so far.

Reference: Surgical Mask vs N95 Respirator for Preventing Influenza Among Health Care Workers
A Randomized Trial

Mark Loeb, MD, MSc; Nancy Dafoe, RN; James Mahony, PhD; Michael John, MD; Alicia Sarabia, MD; Verne Glavin, MD; Richard Webby, PhD; Marek Smieja, MD; David J. D. Earn, PhD; Sylvia Chong, BSc; Ashley Webb, BS; Stephen D. Walter, PhD
JAMA. 2009;302(17):1865-1871. Published online October 1, 2009 (doi:10.1001/jama.2009.1466).

Comments

  1. #1 LeeH
    November 9, 2009

    One small correction, you seem to be confusing Ottawa with Toronto when you say that it was in Ottawa that many health professionals contracted SARS. The two cities are further apart than New York and Washington.

  2. #2 revere
    November 9, 2009

    LeeH: Yikes. I meant to write Ontario. I corrected it. Thanks for pointing it out. Dumb Americans.

  3. #3 Catharine
    November 9, 2009

    I have to say I have some ethical concerns about the fact that nurses are at once the health care workers with the greatest risk of exposure AND used as guinea pigs. (Especially given that many nurses fall into high risk categories due to pregnancy, diabetes, or obesity.) In my unit, the H1N1 pts are almost always ventilated, which, at least, means we are dealing with more of a ‘closed system.’ Nevertheless, something tells me if the nature of physicians’ work was such that they were repeatedly exposed (in very up close and personal ways) to infectious diseases (like nurses are), we would somehow discover a previously unknown source of N95s.

  4. #4 Locasta
    November 9, 2009

    Any idea what the approximate rate of transmission would have been if no masks were used?

  5. #5 revere
    November 9, 2009

    Locasta: Not from these data. That would not be an experiment that would easily pass an institutional review board. But using serology in community based studies, the infection rate for this age group is about the same for the viruses involved (flu A and flu B), suggsting masks may not make much difference for infection although they may for severity of infection (by reducing viral load).

  6. #6 GAC
    November 9, 2009

    How does this compare with rates among health professionals not wearing protection. I realize there might be ethical problems with adding an unprotected control, but there must be some number out there for this.

  7. #7 Stan Polanski
    November 9, 2009

    In response to the ethical concerns expressed in comment #3: Respiratory droplet precautions (surgical mask for direct patient care)have been the accepted standard for influenza for many years, so I don’t think the nurses in the surgical mask arm of this study were being unreasonably subjected to substandard protection. Airborne precautions (N 95 respirator) are generally reserved for diseases like measles and tuberculosis where very small airborne particles are known to be a significant route of transmission. In the spring of 2009 the US CDC published “enhanced precautions” for H1N1 influenza, which called for expanded use of N 95’s. This decision was made despite a lack of data demonstrating efficacy. Public health departments in some states, including my home state of North Carolina, did not go along with the CDC recommendation. The JAMA study sheds some badly needed light on this subject.

  8. #8 Frank Mirer
    November 9, 2009

    You give this study too much credit.
    First, 22+% seroconversion indicates failure of both infection control measures.
    Second, the relative risk of ILI and fever was hugely in favor of the N95 being more protective.
    “Nine nurses (4.2%) in the surgical mask group and 2 nurses (1.0%) in the N95 respirator group met our criteria for influenza-like illness (absolute risk difference, –3.29%; 95% CI, –6.31% to 0.28%; P = .06) (Table 4). All 11 had laboratory-confirmed influenza. A significantly greater number of nurses in the surgical mask group (12, or 5.66%) reported fever compared with the N95 respirator group (2, or 0.9%; P = .007).” Do you call this “slight?”
    Why the reviewers failed to insist on this being carried into the abstract is beyond me, unless they were surgical mask-symps.
    Third, there was no robust measure of compliance with either.
    Also, it would have been nice had the authors referenced the state of compliance with ASHRAE or AIA ventilation standards.

  9. #9 revere
    November 9, 2009

    Frank: Well, the seroconversion rate isn’t much different than the community. That may well be where the infections were contracted, as we noted. We differ on emphasis but both identified the same issues. You seem to have made up your mind. I haven’t made up mine. There will be other studies. We’ll see what happens. Meanwhile we have some data points and you are free to weight them as you see fit.

  10. #10 Catharine
    November 9, 2009

    Stan, thanks but I know all that stuff already – and I am also in NC. Frank, just plain thanks. Let me say that in my institution, we were using surgical masks *before* the JAMA study, *despite* what the CDC was recommending at the time, because of the *potential* threat of running out of N95s (and despite knowing the CDC recommendations months and months *before* we started to worry about not having enough N95s), all of which adds up to either poor planning or reluctance to spend the extra money on N95s or a potential pool of data (using N95s all along would tell us nothing about the efficacy of surgical face mask protection). In ANY case, it seems to me to be an odd order of operations to use a lower level of protection, THEN study the efficacy of that lower level of protection, THEN conclude that it’s a-ok to use the lower level of protection (assuming – and this is a big one – that the study is absolutely definitive). Is this evidence based practice or practice based evidence? If it is practice based evidence, it seems like a few things are missing – like informed consent, controls, and the five dollar coffee shop cards for which we nurses will do just about anything. (That was a joke. If one thing is evident it is that we are far too disorganized and sloppy to do anything sinister.) As far as disasters go, we’re pretty darn lucky that this one is going as smoothly as it is. It doesn’t take an advanced degree to see that the shortages that we do have (of masks, of vaccines) are enough to consider our overall preparedness a major FAIL. We’ve been lucky so far this time. But the next time….?

  11. #11 Don S
    November 9, 2009

    For the little this is worth … I must admit a little roll-eyes when it comes to masks/respirators to protect me from exposure in the hospital. I spend hours a day being coughed on and sneezed at in the office by hundreds of kids with presumptive H1N1 and God knows what else, walk through the streets surrounded by other potential infectives, and then wear a mask or respirator for the several minutes I am in an identified patient’s room? And I am worried about which one is marginally more effective?

    And that is the limitation of these studies in the real pandemic world. It’s kind of like asking which works better – a 65% alcohol hand gel rub or a 15 second hand wash – when the subject always picks his nose right after either one.

  12. #12 George
    November 9, 2009

    I too have concerns with the level of adherence, only 18 audits (7 among users of N95) among the 400 participants over a 4 month study and still they found 1 non-compliant n95 user.

    My main concern is with the sample size. Perhaps it’s my lack of understanding with an RCT (intention to treat) analysis but it seems to me that 1/3 of their sample size is potentially not at risk for their main end point. I thought that intention to treat accounts for what happens after randomization. Since vaccination happended prior to randomization, shouldn’t they have factored in the historical rate of vaccination and appropriately oversampled to compensate for the participants who were not at risk? The serological results seem to indicate that there was a good match between the vaccine and the circulating virus.

  13. #13 Epifreak
    November 9, 2009

    I agree with Dr. Mirer’s comments and have a few additional comments about the Loeb study:

    #1 There was no control group.
    #2 This non-inferiority study did not show that surgical masks were effective protection for influenza – it only demonstrated that in this study surgical masks and respirators were equally ineffective.
    #3 Although it is stated on page 1867 that vaccinated HCWs were removed from the influenza serology analysis in Table 2, it does not appear that they were. For seasonal influenza serology the n for surgical masks should be 157 (225-68) and the n for respirators it should be 159 (221-62).
    #4 The study states on page 1869 that serology was performed at the end of the study period (April 23, 2009) for A/TN/1560/09(H1N1), the “pandemic swine influenza strain.” This is incorrect. There was no serological test for 2009 H1N1 pandemic influenza in April and in any case, it was not circulating in Canada at that time. The reference strain for the currently circulating pandemic influenza H1N1 virus is A/California/07/2009 (H1N1).

    This study should not be used to create public policy for HCW respiratory protection for H1N1, which is the single most contentious issue in infection control today.

    IDSA, SHEA, and APIC are very entrenched in their position that influenza is transmitted by “large droplets” and that surgical masks are adequate protection. They referenced the Loeb study in a letter to President Obama last week asking him to change the CDC recommendation from respirators to surgical masks for H1N1.

    http://www.apic.org/AM/Template.cfm?Section=Featured_News_and_Events&TEMPLATE=/CM/ContentDisplay.cfm&CONTENTID=14342

    I don’t think anyone who reviews the literature can conclude anything except that influenza may be spread by small particle aerosols and that surgical masks do not protect against inhalation of these small particles.

    The issue that is not addressed in current infection control recommendations for any infectious disease is the risk of the pathogen.

  14. #14 BostonERDoc
    November 9, 2009

    This is a political not a scientific issue. The govt has flip flopped so many times on this issue our heads are spinning. One day its surgical, the other its N95–no wait it surgical. Good grief! I personally reviewed the data presented at the August institute of medicine meeting regarding the proper way to protect health care workers. It clearly demonstrated that N95 masks were necessary. This is why they put out their position statement stating such–much to the disgust of NIOSH and CDC. At my shop, we are following the guidelines of the infectious disease society–use N95 for aerosol-generating procedures, surgical for anything else. In this paper you are reviewing they let the RNs use surgical masks for aerosol generating procedures which I personally think was unethical and reckless. With this said, I believe the best way to control this beast is through administrative and engineering measures and vaccination should vaccine ever be widely available. PPE is the bottom of the pyramid.

  15. #15 revere
    November 9, 2009

    Epifreek, Frank: With all due respect, criticizing a study like this is duck soup, doing one is something else. The objective here was to compare two treatments. No mask was not an option. As for the audit, remember where this was done: the epicenter of SARS in North America. they take HCW protection seriously there, especially at the outset of a pandemic. There is some indication that compliance might have been less with N95s, which would be expected since they are far less comfortable.

    This was properly randomized and the outcome was appropriate. The logistics of a study of this type is daunting. I agree that ISDA, APIC and SHEA have established positions and are spinning this to advance their own preferred policies. But so are the two of you. These are data points and subject to confirmation and peer review (by this I am invoking Charlie Poole’s point that real peer review happens after publication). Having said that, I don’t think your critiques are completely fair or unbiased in themselves (mask-symps?).

    As for concluding that anyone who has reviewed the literature is forced to the conclusion that small particles are a major mode of transmission is obviously not true, as there have been a number of reviews done and the usual conclusion is that the data aren’t there to decide. The question remains open, but the epi literature (e.g., the China tour group paper) certainly is suggestive of droplet transmission. Droplets are clearly involved some of the time. What we don’t know is the relative proportion of droplets, small particles and fomites. We need some data and we don’t need to reject out of hand data that doesn’t agree with our preconceptions or bargaining positions.

  16. #16 Epifreak
    November 9, 2009

    Revere,
    I hate to correct you, but you misquoted my comments. I did not say that small particles were a major mode of transmission of influenza, I said the data are clear that they MAY be a mode of transmission. On this, I don’t think (many) people disagree.

    If the Loeb study weren’t being used to advance important infection control policies, I wouldn’t feel so strongly that it be reviewed thoroughly, but it is and I stand by my criticisms.

    The conclusion of the China tour group paper that droplet transmission was probable because it appeared that close contact was necessary for transmission to occur is problematic.

    Although aerosols can be carried over long distances and cause infection in susceptible people at some distance from the infectious person, the occurrence of long-range transmission is affected by dilution, infectious dose, the number of infectious particles, and the persistence of the agent in the environment.

    Inferring the absence of aerosol transmission because long-range transmission of infection is not observed is incorrect. Respiratory aerosols can be inhaled and cause infection at both short-range and long-range.

    The association of droplet exposure with infection is confounded by inhalation exposure because close contact with infectious people permits droplet spray exposure (particles >100 µm), but also maximizes inhalation exposure.

  17. #17 stefano
    November 10, 2009

    Revere

    what’ s about my comment on 122 cities P&I mortality?

    thanks

  18. #18 Steve
    November 14, 2009

    > About 30% of each group had received
    > seasonal flu vaccine.

    This really surprised me. I would have expected a much higher rate of immunization among this population.

    Should I have been surprised. If not, why not?

    Thanks… Steve

  19. #19 Epifreak
    November 17, 2009

    No Steve, you shouldn’t have been surprised. Uptake of influenza vaccine by healthcare workers is pretty abysmal – usually around 40%. HCWs often believe the same myths about vaccines that the general public believes.

Current ye@r *