Time to get back to business after yesterday’s festivities.
One of the items of Gospel Truth among the “autism biomed” movement, which consists of people who fervently believe that autism is caused by some sort of external “toxin,” infection, or vaccines and that subjecting children to various forms of quackery designed either to “detoxify” or reverse whatever physiological derangement believed to be at the root of autism will “recover” these children from autism. Of course, there are a lot of antivaccine believers in the autism biomed movement, and arguably the vast majority of “autism biomeddlers” are fervently antivaccine. That’s the reason why, whenever you see a claim that “environmental influences” are a major cause of autism are trotted out, it’s almost always joined at the hip with antivaccinationists. Antivaccinationists and autism biomeddlers are also very quick to pounce on any study that seems to support an environmental cause for autism. Unfortunately, many legitimate scientists are all too eager to provide such studies. Of course, sometimes true believers produce studies, such as studies claiming that mercury emissions from nearby power plants are associated with autism, but by and large it’s real scientists producing studies that either aren’t very good or are so preliminary as to be poor evidence.
No one denies that making connections between environmental influences and conditions like autism is very difficult. Particularly difficult are studies seeking to find environmental causes through looking for geographic clustering of cases. It’s hard enough to do such studies for diseases like various cancers, and all too frequently when such geographic and environmental associations are claimed to be identified for cancer (often otherwise known as finding “cancer clusters”), media attention and emotions often get the better of science. One wonders if a study about “autism clusters” that I just became aware of will result in the same sort of reaction. I found out about it in Dr. Sanjay Gupta’s column yesterday for MedPage Today, entitled Environmental Factors Tied to Autism Clusters:
An analysis of an insurance claims database of 100 million patients has found clustering of autism spectrum disorder (ASD) and intellectual disability (ID) in counties across the U.S.
The clustering appears to be linked to environmental factors and, to a lesser extent, economic incentives at the state level that affect diagnosis, the researchers reported.
The researchers used male congenital malformations as a surrogate for parental exposures to environmental insults — including pesticides, lead, sex hormone analogs, medication, and plasticizers, among others — which are believed to play a role in the causation of ASD and ID.
“Adjusted for gender, ethnic, socioeconomic, and geopolitical factors, the ASD incidence rates were strongly linked to population-normalized rates of congenital malformations of the reproductive system in males — an increase in ASD incidence of 283% for every percent increase in incidence of malformations (95% CI 91-576, P<6 x 10-5),” wrote Andrey Rzhetsky, PhD, of the University of Chicago, and his co-authors March 13th in PLOS Computational Biology.
Now seems like a good time for a preemptive science-based analysis of the study, as I haven’t seen any antivaccine websites taking it on yet.The study was published yesterday in PLoS Computational Biology and entitled, Environmental and State-Level Regulatory Factors Affect the Incidence of Autism and Intellectual Disability. The investigators come from the University of Chicago at Illinois, Stanford University, and the University of Chicago. Basically, the authors used de-identified patient data from the Truven Health Analytics MarketScan Commercial Claims and Encounters Database to provide geocoded diagnosis counts by gender. This particular database spans the years 2003 to 2010 and contains approved commercial health insurance claims for between 17.5 and 45.2 million people annually. It’s linked across years, so that it yields a total of approximately 105 million patient records contributed by well over 100 insurance carriers and large self-insuring companies. The authors looked at approximately 4.6 billion inpatient and outpatient service claims and identified 6 billion diagnostic codes. Once duplicates were identified and removed, there were left almost 1.3 billion diagnostic codes associated with 99.1 million individuals. All patient-level personal information was redacted and geocoded by county level.
These data were then augmented with US census data consisting of county-level measurements for various socioeconomic and demographic factors, including: Gender, average per capita income, income, percent ethnicity (separately for American Indians, AmInd, Asians, Asian, White Hispanics, WHisp, White non-Hispanics, W, Black Hispanics, BHisp, Black non-Hispanics, B, and Pacific Islanders, Pacific), and the proportions of various socioeconomic groups (poor, Poor, urban, Urban, insured, Insured). County-level environmental indicators were used as fixed-effect covariates (normalized by county population size) and included congenital malformations excluding malformations of the genitals (separately for females and males), congenital malformations of the genitals (separately for females and males), viral infections, ectopic pregnancy, abnormal conception, spontaneous abortion, and multiple gestations.
So right off the bat, we see that this is basically an ecological study. True, it does use patient-level data, but that data is aggregated at the county level and correlated with county-level data from census data to look for correlations and clustering. In other words, the group, not the individual, is the unit of analysis, here county-level aggregated data. In particular, the authors used genetic abnormalities leading to malformations of the genitals as a surrogate for exposure to “toxins” and then tried to correlate the incidence of autism and autism spectrum disorders (ASDs) and intellectual disability (ID) to this surrogate marker. Now, there is an inherent problem with ecological studies in that they have a very high tendency to find spurious correlations and to exaggerate real correlations. This is known as the ecological fallacy or ecological bias. In any case, it’s a reason to be skeptical right off the bat. Add to that the underlying assumption that the birth defects examined are reliable surrogates for exposure to “toxins,” and there’s even more reason to be skeptical. I’ll show you what I mean.
Let’s look at what the authors found:
Adjusted for gender, ethnic, socioeconomic, and geopolitical factors, the ASD incidence rates were strongly linked to population-normalized rates of congenital malformations of the reproductive system in males (an increase in ASD incidence by 283% for every percent increase in incidence of malformations, 95% CI: [91%, 576%], p<6×10−5). Such congenital malformations were barely significant for ID (94% increase, 95% CI: [1%, 250%], p = 0.0384). Other congenital malformations in males (excluding those affecting the reproductive system) appeared to significantly affect both phenotypes: 31.8% ASD rate increase (CI: [12%, 52%], p<6×10−5), and 43% ID rate increase (CI: [23%, 67%], p<6×10−5)
Of this, the lead author Andrey Rzhetsky said in Dr. Gupta’s article:
“I suspected that connection between environmental status to rate of autism might exist, but the signal is much stronger than I expected,” Rzhetsky told MedPage Today in an email.
Well, maybe. It’s true that the authors did analyses to control for several putative confounding variables, specifically county-specific median mother’s age at childbirth, and the proportion of county population in the childbearing age. Several other socioeconomic factors were adjusted for as well, as shown in Table I. However, it’s already known that birth defects are associated with autism. For instance, an Australian study found a 2-fold increase in birth defects in children with autism autism, but not Asperger’s syndrome, while this study also found an association, specifically between urogenital defects and autism. Moreover, it’s already known that there’s an correlation between autism and exposure to teratogens, specifically at least maternal rubella infection, thalidomide, valproic acid, and misoprostol. The authors (sort of) acknowledge this association:
It is known that some birth malformations are caused by de novo genetic events, such as large copy number variants that have been found to increase the risk for ASD by approximately 400% . Single-gene deletions, for example, involving CHD7 are known to cause CHARGE syndrome ,  associated with genital abnormalities and putatively associated with ASD . However, these genetic events may have currently poorly identified environmental triggers, and 70 to 80% of male congenital malformations of the reproductive system have no clear genetic causes . Instead, they appear to be driven by specific environmental insults that were not serious enough to lead to more serious adverse events during pregnancy, such as spontaneous abortion. Therefore, in this study, we used the rate of birth malformations as a surrogate measure for environmental burden.
In other words, the implicit assumption behind this study is that some “toxins” associated with environment cause genetic changes that cause both congenital anomalies and autism. While this is possible, we have no way of knowing how likely this hypothesis is compared to the alternate hypothesis, namely that there is simply an association between congenital anomalies and autism, no environmental “toxins” necessary to explain it. It’s also clear that there are confounders not accounted for in the analysis, as the authors themselves acknowledge. If you look at Figure 2, you’ll find rather clear state boundaries in many instances, and the authors point out that their estimates of random effects suggest unidentified confounders at both county and state levels.
Of course, one major issue with any study of this type is what’s known as the Texas Sharpshooter Fallacy. Basically, that’s a fallacy named after the concept often used to illustrate it: A “sharpshooter” shoots at the side of a barn randomly, and then draws a bullseye around the bullet holes. In the case of geographic clusters, a number of cases might be noticed, and investigators could draw its population from the smallest area possible, without considering that the cases come from a large population. (A good lay person-accessible explanation of the perils and pitfalls of geographic clustering being used as evidence for an environmental cause of a condition—in this case cancer, but it could just as well apply to autism—can be found here.)
The bottom line is that it’s very difficult to know what the best geographic level of grouping is that will minimize the noise in the signal while still not missing potential associations. The smaller the geographic area, the more variation will be seen, and county-level information is generally not very large. In the case of autism, clustering could easily be due to better resources and more intensive screening, which would identify more cases and, given the known association between ASDs and congenital anomalies, there would be a correlation found, no need to invoke environmental “toxins” necessary. For example, look at South Dakota in Figure 2. ASD incidence is very low in some of the counties there. (An annoying thing about this graph: Dark colors mean less autism.) It’s highly unlikely that ASD prevalence would be that low (less than 10-4)anywhere where good pediatric screening programs exist. Even taking into account that this is prevalence and not incidence, that seems way low. Had I reviewed this paper, I would have asked what the heck was going on with these counties and what was going on in South Dakota.
Another thing that annoys me about this study, and it’s the way that the authors throw around the term “toxin,” not being particularly precise in their definition. In the world of woo, “toxins” are the magical miasmas that cause all disease but are never defined, although they must always be removed and the patient be “detoxified.” I know the authors don’t know this and didn’t intend it, but their article comes across to me as custom-made to be quote-mined. It doesn’t help that the authors write things like this:
Following similar logic, in addition to causing birth defects, environmental toxins, such as pesticides ,  can substantially weaken the human immune system, especially in men, which results in more frequent infections.
Clearly, they are not immunologists, and clearly they aren’t familiar with woo-speak.
Another thing it’s important to remember is that this dataset is not a random sampling of the US population. As the authors note in the methods, the compilation of the dataset required agreements between Truven and numerous individual insurance providers to share data, and the providers inherently had uneven and nonrandom coverage of geographic areas, which could lead to what the authors refer to as “traces of hidden correlations imposed by the data collection method.” On the plus side, it does include Medicare and Medicaid (presumably Medicare and Medicaid HMOs, given that it wasn’t clear to me whether Medicare and Medicaid claims themselves were included, but I don’t know that for sure and couldn’t clarify this point before posting this).
Finally, there’s one other interesting finding:
Furthermore, the state-mandated rigor of diagnosis of ASD by a pediatrician or clinician for consideration in the special education system was predictive of a considerable decrease in ASD and ID incidence rates (98.6%, CI: [28%, 99.99%], p = 0.02475 and 99% CI: [68%, 99.99%], p = 0.00637 respectively). Thus, the observed spatial variability of both ID and ASD rates is associated with environmental and state-level regulatory factors; the magnitude of influence of compound environmental predictors was approximately three times greater than that of state-level incentives.
Of course, this is not unexpected. The more diagnostic rigor there is when it comes to diagnosis, the lower the reported incidence will be, virtually no matter what the condition or disease is. What would have been interesting to know, and what the authors didn’t investigate, is what effect screening programs have on the incidence of autism and ASDs. I again liken it to mammography. The more intensely you screen for a condition or disease, the more of it you will find, almost always. It’s not hard to imagine a confounding factor in which counties and states with more intensive developmental screening programs might tend to screen more aggressively for congenital anomalies, meaning a higher incidence of both, due both to screening and the known association between autism and congenital anomalies. I would suggest this as another area of investigation.
The bottom line is that this study reports some intriguing results, but it is not really strong evidence for an environmental toxin as being a major cause of autism. Geographic clusters of conditions and diseases (like cancer clusters) do occur randomly, and ecological studies like this tend to produce a lot of false positives. Also, the authors, after establishing a baseline incidence of autism and ID across the country, interpreted deviations from this baseline as due to local effects. That might not be an unreasonable interpretation, but “local effects” do not equal “toxins”; yet the inference is very strong in the paper that this must be true. Personally, I’d wonder about deviations below the baseline rate as well. In counties with lower than baseline incidences of autism, is there something that’s protective against autism? Yet this question is never even mentioned, much less considered. I realize that the authors weren’t looking for this, but considering the question addresses plausibility. If the authors consider it plausible that local “toxins” cause autism and congenital anomalies, then why are there a number of counties with very low autism and ID incidence? It’s the flip side to asking why there are counties with high incidences.
I’ll finish with a random provocative question that popped up in my mind as I was reading this paper: Where are the superfund sites? These are chemical disposal sites for which the federal government created a fund to clean up. They’re frequently used as a surrogate for known exposures to potentially cancer-causing and teratogenic chemicals. It wouldn’t have taken that much more to look at the counties in which there are superfund sites to see if there is a correlation between autism incidence and being in the same county as (or an adjacent county to) a superfund site.
And if the investigators do this analysis and haven’t already thought of it, please at least give me an acknowledgment in the paper.
Oh, and this:
While the effect of vaccines was not analyzed as part of this study, Rzhetsky notes that the geographic clustering of autism and ID rates is evidence that if vaccines have a role, it’s a very weak one as vaccinations are given uniformly across the US.
Hmmm. Maybe antivaccine activists won’t be citing this paper much, after all.