If there’s a topic that I don’t write about much, it’s obstetrics. The reason is that it’s not a major area of my interest, and it’s not an area where I have as much expertise as I do in, say, cancer or even vaccines. My expertise in cancer comes from my career, of course, and my expertise in vaccines is self-taught through my 9+ years of blogging about it. Ditto my expertise in “complementary and alternative medicine” (CAM), also known as “integrative medicine,” about which I know quite a lot now. That’s probably why I didn’t pay much attention to a study that came out in late January from the The Midwives Alliance of North America Statistics Project, 2004 to 2009 (MANA). However, this study came to my attention due to a little blog kerfuffle going on between Dr. Amy Tuteur, who goes by the blog moniker The Skeptical OB (or Dr. Amy) and Jamie Bernstein over at Grounded Parents. Fortunately, the paper is available online; so we can all follow along, which allows me to do what I think best, discuss the paper first and then the blog kerfuffle, which, to be honest, I’m having a hard time figuring out why it’s become so vitriolic. It’s actually rather disturbing to see, given that the differences in position seem rather small. Going through it all, as painful as it is, might be educational. Besides, I’ve already pissed off one big name skeptic a couple of weeks ago. If I end up pissing off another one, it’s no big deal at this point, right?
Besides, one of the parties involved in this little dustup (which is, sadly, kicking up a fair amount of dust), Jamie Bernstein, who’s a friend of mine, asked me to look into the issue. At first I didn’t want to, but I changed my mind.
Before I go on, let me just say also that I tend to agree that most of the time home births are probably bad idea. There’s a lot of evidence out there that the risk to both mother and baby from home birth is significantly higher than it is in hospital. While it might be possible in highly selected cases to justify doing a home birth for low risk singleton pregnancies, it strikes me as way too risky to consider a home birth for higher risk, complicated pregnancies. Actually, this study suggests the same thing. Finally, I have noted that there is very much a cultish aspect that has grown up around home births, where the risks are downplayed and the “experience” is all. These are the areas where Dr. Amy, Jamie, and I generally appear (mostly) to agree, all the more reason why I’m puzzled by this whole thing.
The MANA study
First, let’s take a look at the paper. No, wait. First, let’s take a look at what MANA said about the paper in its press releases and fact sheets:
Safe Outcomes with Positive Benefits
- High rate of completed home birth (89.1%)
- High rate of vaginal birth (93.6%)
- High rate of completed vaginal birth after cesarean (VBAC; 87.0%)
- Low intrapartum and neonatal fetal death rate overall:
- o 2.06 per 1000 intended home births (includes all births)
- o 1.61 per 1000 intended home births excluding breech, vbac, twins, gestational diabetes, and preeclampsia.
- Low rate of low APGAR scores
- Extremely high rate of breastfeeding (97.7%) at 6 weeks
Few Emergency Transfers to Hospital Care
- Primary reason for transport was “failure to progress.” Transfer for urgent reasons, such as “fetal distress” was rare.
Low Rates of Intervention
- Cesarean section rate of 5.2%
- Less than 5% used pitocin or epidural anesthesia
Sounds fantastic, doesn’t it? There’s just one problem, which has been pointed out by Dr. Amy, Jamie Bernstein, and Steve Novella. That problem is that, if you compare the death rates in the MANA study to publicly available death rates from other sources, the death rates in the MANA cohort are higher—and not just by a little. The issue behind the fracas is how much higher and in emphasis.
Actually, the better way to describe the problem with this study is that there is no control group. This is a single arm, prospective study. It’s good that it’s prospective, rather than retrospective, because it means that the data were all collected as patients were enrolled. However, it’s also important to recognize that this is not a typical group of pregnant women giving birth. As the authors themselves mention in the introduction that 1.18% of births in the US occur outside of the hospital, and, of those, approximately 66% are homebirths, for a total of 31,500 home births a year. This study examines 16,924 pregnancies, for a total of 16,984 neonates including 60 sets of twins, over birth years ranging from 2004 to 2009, which breaks down to around 3,385 births a year, a bit more than 10% of the home births estimated to have occurred in the US during this time period. That’s a significant sample, but it’s nonetheless a small fraction of an even smaller fraction of all the births that occurred in the US during that time frame.
That this is an unrepresentative sample is obvious just perusing the numbers, which are overwhelmingly white (92.3%) and college-educated (58.4% having completed a college degree, compared to 33.5% recently estimated for the general population). Basically, the numbers were what the press release/fact sheet listed above. Other important numbers were summarized here:
The overall death rate from labor through six weeks was 2.06 per 1000 when higher risk women (i.e., those with breech babies or twins, those attempting VBAC, or those with preeclampsia or gestational diabetes) are included in the sample, and 1.61 per 1000 when only low risk women are included. This rate is consistent with some published reports of both hospital and home birth outcomes, but is slightly higher than others. Because only 0.45 per 1000 separates these samples, further work is needed. These findings should, however, help to inform the process of shared decision-making as women talk with their providers about their own specific risk profiles, value systems and priorities for birth.
This leads the authors to conclude in their paper:
For this large cohort of women who planned midwife-led home births in the United States, outcomes are congruent with the best available data from population-based, observational studies that evaluated outcomes by intended place of birth and perinatal risk factors. Low-risk women in this cohort experienced high rates of physiologic birth and low rates of intervention without an increase in adverse outcomes.
Except, as Steve Novella pointed out, based on Dr. Amy’s calculations, the equivalent number for the overall death rate from labor through six weeks for planned hospital births is 0.38 per 1000. Now, I fiddled around with the CDC Wonder Database a while trying to see if I could replicate Dr. Amy’s number. I couldn’t. There are lots of settings in the Wonder Database, and I tried to match the cohort in the MANA study as well as I could. No matter how I tried to do it, my numbers came out around 0.5 per 1000 or I got a lot of “suppressed” data, which apparently means that there were too few to be considered valid. As far as I’m concerned, that’s close enough, as I didn’t have time to fiddle with the settings all night last night and unless Dr. Amy reveals the exact settings she used to get her number, there’s no way for me to easily replicate it easily. So, because it’s simpler, I’ll just assume she knew what she was doing and go with it. I also note that it’s not possible to get death rates for birth and the six weeks after birth using the MANA database, because the choices do not include the first six weeks. The ranges are (1) under one hour; (2) 1-23 hours; (3) 1-6 days; (4) 7-27 days; and (5) 28 to 364 days.
Be that as it may, for low risk pregnancies, the MANA death rate of 1.61 per 1000 is 4.2 times higher than the estimate provided by Dr. Amy from the CDC database of 0.38 per 1000. If we use my quick and dirty estimate of 0.5 per 1000, then that’s 3.2 times higher. Both of these numbers are within the range of what previous data have shown. For example, a meta-analysis from 2010 concluded that planned home births were associated with a tripling of the neonatal mortality rate. That’s, as I would put it, plenty bad, man.
Of course, there are problems with this comparison. We’re comparing apples and oranges in that there was no control group for the MANA study, and it’s always problematic to compare what is in essence such a highly self-selected group with population-based statistics. We have no idea if the MANA group was representative of women who choose home births, because it’s a group of women who not only chose home birth (less than 1% of pregnancies) but were also willing to sign the informed consent form to be in the study. Moreover, the error bars were large. Even when two groups are not easily directly comparable, as is the case here, a more than four-fold relative risk of death is definitely something to be concerned about, and it’s concerning that MANA does its best to de-emphasize it.
In its Consumer Considerations, MANA tries to downplay the risk, and, to be fair, it does have (somewhat) of a point in that it points out that what we are talking about are relative risks. As Steve Novella points out, when you look at a comparison of absolute risk, the difference between home birth and hospital birth is 0.123% (0.161% -0.038%), or 1.2 per 1,000, estimating that in the study population this amounts to 20 extra perinatal deaths. I liken this to the issue of mammography and an example I used before:
Essentially, mammography reduces the odds of a 60-year-old woman dying of breast cancer in the next decade by 30%. Sounds impressive, until you look at her absolute risk: by getting her annual mammogram, her chances of dying from breast cancer are whittled from 0.9% to 0.6%. Overall, for every 1,000 women in their 60s screened for breast cancer in the next 10 years, mammograms will save the lives of 3 people but 6 others will still die. (The numbers edge up or down in lockstep with a woman’s age.)
The point is not to try to directly compare the numbers but to choose an example to illustrate a similar point, namely that when you look at absolute risks instead of relative risks, the perspective changes. I think it’s important in these studies always to include both numbers, the relative risk and absolute risk. None of this excuses MANA from using the graphs they did to try to downplay the even more elevated risk of infant death in infants presenting in breech position or women undergoing a vaginal birth after Caesarean section (VBAC), which were 5 out of 222 and 5 out of 1052, respectively. What is being weighed is whether the value of the experience of home birth is worth the roughly three- or four-fold increased risk of neonatal death, which translates to approximately an extra 1 in 1000 chance of the baby dying. I’m not going to touch that one with the proverbial ten foot cattle prod except to say I’d conclude: No.
Unfortunately, there are not a lot of high quality comparative data to use. A recent Cochrane review concluded that there is “no strong evidence from randomized trials to favor either planned hospital birth or planned home birth for low-risk pregnant women.” Of course, this is not surprising. Such a trial would be fraught with ethical difficulties, not the least of which because of data like the MANA data suggesting that home birth is almost certainly riskier than hospital birth. Moreover, from a strictly practical standpoint, I don’t see too many women being willing to be randomized either to hospital or home birth. Observational evidence is what we are going to have to use, because it’s highly unlikely that we’ll ever have decent evidence from a randomized trial. For some questions in medicine, it has to suffice.
The entire blog kerfuffle that’s been intermittently flaring up boils down to a single graph and a single passage in Dr. Amy’s post about this paper. Given how much I harped on certain skeptics to provvide full disclosure, I will state right here, right now, that Jamie asked me to look at these posts. So I did. Now here’s the single graph:
And here’s the passage:
As the chart above demonstrates, the MANA death rate for the same years was 5.5X HIGHER. In other words, the MANA death rate was 450% higher than the hospital death rate.
On what planet is a death rate 450% higher than expected a safe outcome? Not on this planet.
MANA and homebirth midwives have been lobbying extensively for a scope of practice that includes breech, twins, VBAC, etc. Now they want to exclude those same births from their statistics. Even then, the MANA death rate is 4.2X higher than hospital birth. So even when homebirth midwives stick to low risk patients, homebirth has a death rate 320% higher than comparable risk hospital birth.
One thing that’s clear is that the figures of 2.06 per 1000 and 1.61 per 1000 were not in the MANA paper, at least not in that form. They were in that form in the Consumer Considerations page that I cited above. This clearly confused Jamie at the time because she couldn’t find the equivalent numbers in the paper. I must admit that at first it rather confused me a bit, too, as to how those numbers were calculated based on the figures reported in the paper. I had to figure it out by wading through the data and determining which numbers had been added together to come up with these figures. Be that as it may, Jamie was critical of Dr. Amy for the above passage, for these reasons:
Let’s leave this study for a bit and go back to the numbers Dr. Tuteur cites in her post. According to Dr. Tuteur, 1.6 per 1000 low-risk planned homebirths from the recent study resulted in neonatal death within 6 weeks of birth. Using CDC data, she also determined that the risk of neonatal death for low-risk white women in the US during the same years was 0.4 per 1000 births. She then points out that OMGZ YOU GUYS THAT’S A 5.5X INCREASE IN BABY DEATHS!
First of all, can I first point out that it’s a 4x increase, not 5.5x (1.6/0.4 = 4). Even comparing the homebirth cohort that includes high-risk births (2.1 in 1000) to the low-risk only CDC cohort (which is not a fair comparison for obvious reasons) would only result in a 5.25x increase in mortality. I seriously have no idea how Dr. Tuteur came up with 5.5x or 450% increase in mortality from the numbers that she cited.
I can find no flaw in this reasoning, and, had I noticed this first, I would have been far more snarky about it, because, well, that’s how Orac rolls. Dr. Amy should appreciate that, because she herself is often as sarcastic or more so than Orac is, only nowhere near as funny about it. Dr. Amy clearly did not do the right comparison, which would have been to compare low risk with low risk—by her own numbers. If she was so convinced that the figure of 2.06 per 1000 was the correct figure, then why did she include the figure of 1.61 on the graph and label it “low risk”? Moreover, her rationale makes no sense. After all, the statistics are right there, in the paper. True, they’re not in a nice table format, like most of the other statistics presented in the MANA paper. But they are there, so much so that Dr. Amy was able to come up with 2.06 and 1.61 out of 1000 numbers by adding together the appropriate figures. In any case, Dr. Amy attacked Jamie’s analysis as a “hatchet job.” Some of what she says makes sense, but I’m going to cite one thing that is a steaming pile of fetid dingo’s kidneys:
Clearly she never bothered to look at the CDC Wonder database, which contains an complete description of contents. I specifically noted that I looked at white women, 37+ weeks, 2500 gm babies.
No, Dr. Amy did not mention any such thing, at least not in her original post discussing the MANA paper. In any case, unlike Jamie, I did look at the CDC Wonder Database. I wasted quite a bit of my time fiddling around with it, trying to replicate Dr. Amy’s number. I couldn’t do it, because I was largely guessing based on what Dr. Amy said in her original post, “According to the CDC Wonder database, the neonatal death rate for low risk white women at term from the years 2004-2009 is 0.38/1000.” Nope, there’s nothing there about 37+ weeks or 2500 g babies. Armed with this new knowledge from Dr. Amy, I went back into the CDC Wonder Database, and plugged her figures in. That resulted in a mortality rate of 0.57 per 1000. At this point I was tempted to appeal to Dr. Amy for all of her other settings she used to search the CDC Wonder Database because I’m not getting the same number she is, and it’s not because I can’t figure out how to use the database. I use databases like this one all the time. But then I came across her a later analysis, I realized that the reason for the anomaly is probably because I didn’t exclude the ICD-10 code for major congenital malformations from the causes of death. So I did it again and came up with an estimate of 0.34 per 1000. That’s better, but still not identical to what Dr. Amy came up with. At least it’s now close enough that I’m not wondering as much as I was before what the heck was going on. Still, it would be nice to know the full settings used.
I don’t really want to go into the next phase of this little kerfuffle, which involves a statistics professor guest posting about the MANA study and a critique by Jamie, because this post has already grown too long, even by Oracian standards. Maybe if there’s interest on your part (and still on my part) I might take it on another time. Whatever mistakes Jamie might have made (and it’s clear that she didn’t find the figures used to come up with the aggregate estimated mortality rates of 2.06 and 1.61 per 1000), she didn’t deserve what came about a week ago, when this whole kerfuffle culminated in a rather amazing post by Dr. Amy that asked Can women be skeptics? In it, she says:
I’m beginning to wonder if there is a germ of truth to the claim that there are not more women in skepticism, because women are so anxious to avoid confrontation.
You cannot be a skeptic and censor debate. Yes, you can remove racism and other evidence of hatred or discrimination. Yes, you can remove comments that are not on point. But you can’t censor comments that you don’t like and still call yourself a skeptic.
Remember that there was a time when Dr. Amy was on Science-Based Medicine. It didn’t work out. You want to know the reason it didn’t work out? A certain guy that you all know and (hopefully) love described what happened in a comment:
Of course, Amy is free to present her side of the story as much as she likes on her blog. For now, we are going to take the high road and not get into a “he-said-she-said” sort of exchange. However, since Amy has cited our moderation policy as a reason for her departure, I no longer see a reason not to confirm that one unresolvable issue was that Amy strongly favored a more heavily moderated commenting system, and Steve and I were very reluctant to change our commenting policy. But the disagreement over the commenting system was certainly not only the bone of contention that we could not resolve.
Many of our regular readers will know who wrote that, but in the interests of full disclosure I’ll just say: Here’s another clue for you all. The walrus was Paul. No, wait, that won’t do. Besides many of my readers being too young to get that reference, it’s just another bad Orac joke. So instead let’s just say that everybody’s box of blinky lights wasn’t too far at all from that particular comment, lo, those four years ago. (Has it really been that long?)
In any case, Dr. Amy clearly left SBM in part because she didn’t like its free-wheeling, hands-off moderation policy (which is only marginally tighter than almost non-existent moderation policy here). I remember it well and could elaborate further, but I won’t, at least not without a very compelling reason. What was written then says enough for my purposes.
The bottom line according to Orac
Here’s what I don’t understand. A four-fold increase in mortality in planned home births attended by midwives, if accurate, is plenty bad enough. It’s more than enough to have grave concerns about the safety of even low risk births in the US. I say “if accurate,” because comparisons between two such different sources are always fraught with issues because we don’t know how comparable the groups are. However, a four-fold or five-fold increase in risk is enough that it starts to raise red flags, even accounting for the problems in comparing two different data sets.
So why does Dr. Amy have to make a questionable comparison—and, contrary to her defense of it, it is dubious—to get the risk up only to a 5.5-fold increase? I don’t know. Compare low risk to low risk or all births to all births, but don’t compare low risk to all births. I could understand the temptation if it meant the difference between a four-fold and a twenty-fold increase, but making a big deal out of the difference between a 4.2-fold and a 5.5-fold increase in the risk of neonatal death strikes me as pointless, given the size of the confidence intervals involved. We’re all agreeing that there appears to be a significantly elevated risk of neonatal death due to home births. We’re all agreeing that the mortality rate for breech presentation and other high risk pregnancies appears to be elevated in the MANA data to a very worrisome degree, making MANA’s lobbying for a greater scope of practice for midwives to include these sorts of high risk pregnancies a policy to be opposed until there is evidence showing that their outcomes are equivalent. If this argument isn’t about how many angels can dance on the head of a pin, it’s not too far off from that.
So why all the vitriol from Dr. Amy? (Yes, it’s mostly coming from her.) It doesn’t make sense to me, particularly her bringing gender, rather than data, into the issue with her unnecessary insinuation that some women can’t be skeptics because they’re too touchy-feely and concerned about not offending—after her demands for an apology. Being civil and being a skeptic are not mutually exclusive. In other words, you don’t have to be a jerk to be a skeptic, although clearly some skeptics are jerks at times, this box of blinky lights included, and some are jerks much of the time. Moreover, one can’t help but note that Steve Novella came to the same conclusion that Jamie did long before I looked into this matter; yet I haven’t seen him criticized the way Jamie was.
I suppose Dr. Amy won’t be happy with me for writing this. I rather expect that I’ll be accused of standing up for a friend (as if this were a bad thing), that somehow I don’t “understand” the data; that I’m too stupid to figure out how to replicate Dr. Amy’s search of the CDC Wonder Database, and the like. So be it. It’s not as though I haven’t endured far worse criticism over the years, sometimes even from people on “my” side. Who knows? She might even convince me that she’s right and I’m wrong. Data can do that.