More than a week has passed, and I thought that this cup had passed from me, and I was glad. After all, if I analyzed every crap study done by anti-vaccine zealots to try to demonstrate that vaccines cause autism, I would have time for little else in terms of other kinds of that Insolence you all know and love. This particular study was released in late May and, at the time, I wasn’t really in the mood to take it on; so I ignored it. But then wouldn’t you know that the Autism Action Network would have to go and send out a press release yesterday entitled New Study Links Vaccines and Autism: Let your Representatives know about this study:
We frequently hear in the corporate media about studies that claim to show no association between autism and vaccines. But when do you ever hear about the studies that do show an association? Well, here’s one that was just published.
A study in the Journal of Toxicology and Environmental Health finds a relationship between the proportion of children who received the recommended vaccines by age 2 years and the prevalence of autism or speech or language impairment. The higher the proportion of children receiving recommended vaccinations, the higher was the prevalence of AUT or SLI. The results suggest that although mercury has been removed from many vaccines, the remaining mercury as well as other culprits such as aluminum and live viruses may link vaccines to autism. Further study into the relationship between vaccines and autism is warranted.
Damn. Just when I thought I was out (at least for a while, on vacation, so to speak), they pull me back in. After all, this study, not surprisingly, is showing up in all the usual anti-vaccine locations, including the home of Olmsted’s band of merry antivaxers known as Age of Autism, where the study author herself, Gayle DeLong, actually promotes her own study. Not a good sign. Not a good sign at all about the scientific credibility of the author. Word to Gayle: If you want to keep even a semblance of scientific credibility, showing up on AoA to pimp your latest study is not a particularly good way to go about it. From one academic to another. Of course, not surprisingly also, DeLong is not a scientist; she is, rather, a faculty member in the Department of Economics and Finance in the Zicklin School of Business, Baruch College/City University of New York. As always, the fact that DeLong is clearly not a scientist doesn’t necessarily mean that she is wrong (although it does increase the probability). Rather, her poor study design and biased presentation are far more likely to show that she is wrong. That’s something this study has in spades, and somehow she managed to get her paper accepted to the Journal of Toxicology and Environmental Health, a journal I only vaguely remember having heard of:
A Positive Association found between Autism Prevalence and Childhood Vaccination uptake across the U.S. Population
Journal of Toxicology and Environmental Health, Part A: Current Issues
Volume 74, Issue 14, 2011, Pages 903 – 916
Author: Gayle DeLong
The reason for the rapid rise of autism in the United States that began in the 1990s is a mystery. Although individuals probably have a genetic predisposition to develop autism, researchers suspect that one or more environmental triggers are also needed. One of those triggers might be the battery of vaccinations that young children receive. Using regression analysis and controlling for family income and ethnicity, the relationship between the proportion of children who received the recommended vaccines by age 2 years and the prevalence of autism (AUT) or speech or language impairment (SLI) in each U.S. state from 2001 and 2007 was determined. A positive and statistically significant relationship was found: The higher the proportion of children receiving recommended vaccinations, the higher was the prevalence of AUT or SLI. A 1% increase in vaccination was associated with an additional 680 children having AUT or SLI. Neither parental behavior nor access to care affected the results, since vaccination proportions were not significantly related (statistically) to any other disability or to the number of pediatricians in a U.S. state. The results suggest that although mercury has been removed from many vaccines, other culprits may link vaccines to autism. Further study into the relationship between vaccines and autism is warranted.
When the fallacy count in just the abstract alone is so high, this is another very bad sign. In fact, I could probably dismiss this paper as an utter waste of time just by deconstructing the abstract alone. Of course, as I almost always do I’ll go farther than that, but I can’t resist pointing out a bit of misinformation right in the abstract. For example, the reason for the rapid rise of autism in the U.S. is not really much of a “mystery.” It’s very likely the result of diagnostic substitution in the wake of the broadening of the diagnostic criteria for autism and autism spectrum disorders that occurred in the early to mid-1990s, as Paul Shattuck has shown. Oh, there may be a genuine increase in autism prevalence over the last 20 years (although even that is debatable), but, if it exists, it’s so small that it’s not even clear that there is one.
DeLong carries on this sort of misinformation right in the text of her paper, which an anti-vaccine commenter on AoA has kindly provided a link to, at least until the publisher finds out that she’s flouting copyright laws. Here’s one thing you should know about reading scientific papers. The introduction is where the authors try to “frame” the issue that led them to do the research and the hypothesis that derives from that issue in the most favorable way possible. To the knowledgeable reader or reviewer, a botched up introduction section that misrepresents the scientific consensus and the issues is almost always a sure sign that the science that follows will either (1) not support the authors’ hypothesis or (2) be of such poor quality that it doesn’t really support any hypothesis or even (3) cast doubt upon the authors’ hypothesis, even though the authors spin it otherwise. In this paper, for instance, DeLong argues that there “are several reasons why vaccines may trigger autism,” after which she lists a veritable laundry list of long-discredited anti-vaccine notions as to how vaccines can trigger autism, bringing up (naturally!) old anti-vaccine bogeymen like mercury and aluminum. Nowhere is it mentioned that this is not the scientific consensus; only one side is presented, the anti-vaccine side.
One way you can recognize a truly bad introduction to a research paper is by the quality of the research that is cited in it. In DeLong’s case, the research cited is awful indeed, with citations to papers by the incredibly logorrheic (even more so than Orac) Mark “Not A Doctor, Not A Scientist” Blaxill, Mark and David Geier (otherwise known as the doctor with a suspended license and his son busted for practicing medicine without a license), all purpose crank Russell Blaylock (who counts HIV/AIDS denialism, antivax, and many other forms of quackery are all part of his repertoire), and Laura Hewitson’s execrable “monkey business” research, which, I note, was also published in the very same journal that DeLong’s study appears in. There’s more, but these are just some of the examples, perhaps the most egregious of which is a reference by the anti-vaccine homeopath James Compton Burnett writing in 1884.
Then there’s the design of the study itself. Jumpin’ Jesus on a pogo stick, there’s the design of the study itself! If this is the sort of research design that is considered acceptable and routine in economics and business, no wonder our economy’s in such a mess. First (and most egregious), there’s the issue of why DeLong combined SLIs (see abstract above) with autism diagnoses to do her analysis. DeLong appears to have used statistics that states are required to maintain under federal legislation, the Individuals with Disabilities Education Act (IDEA). Under IDEA, every school is required to provide data on children who have an Individual Education Plan (IEP), including the students’ primary classification. As Liz Ditz pointed out, IDEA classifications are not medical diagnoses. A child with a diagnosis of autism under IDEA may or may not really have autism. Also, children with an IDEA classification of SLI are most commonly children with problems in fluency, articulation, or voice, not autism. Examples include apraxia and aphasias, voice disorders, stuttering, and language-based learning disabilities. It’s not for nothing that James Laidler characterized IDEA data as not being a reliable measure that can be used to track autism prevalence accurately.
Naturally, DeLong cites papers to justify lumping together SLIs and autism for purposes of her analysis. Also, naturally, they do not support her hypothesis. These are the three papers cited:
- Conti-Ramsden et al. (2006). The prevalence of autistic spectrum disorders in adolescents with a history of specific language impairment (SLI). J Child Psychol Psychiatry. Jun;47(6):621-8.. This paper concluded that “the prevalence of autism spectrum disorders in young people with SLI was found to be 3.9%, about 10 times what would be expected from the general population. In addition, a much larger number of young people with a history of SLI showed only some autism spectrum symptoms or showed them in a mild form.” Let’s put it this way. This paper still does not justify lumping SLIs in with autism because the vast majority of children with SLIs do not have autism. At most, the authors found that a quarter of children with SLI demonstrate behaviors that might indicate an ASD.
- De Fosse et al (2004). Language-association cortex asymmetry in autism and specific language impairment. Ann. Neurol. 56 , pp. 757-766. This paper deals with specific language impairment. This is not the same thing as the broad IDEA category of speech/language impairment. Admittedly, they do have the same abbreviation, though; so maybe DeLong thought they were the same thing. It’s obvious that none of the reviewers noticed this, or, if they did bother to look at the references, were as clueless as DeLong was. You would think that reviewers would look at the abstracts of key references used to justify lumping together data from different conditions the way that DeLong did, particularly when the conditions are not obviously related. Apparently, you would be wrong.
- Herbert et al (2007). Altered brain wave activity in persons with chronic spinal cord injury. Int J Neurosci. 2007 Dec;117(12):1731-46. This paper has nothing to do with autism or even much to do with SLI. Did DeLong even read her own references? Or even read the abstracts from her own references?
I’m left with the not-so-sneaking suspicion that the only reason that SLIs were lumped together with autism and ASDs for purposes of correlation with the percentage of children in each state receiving their full vaccine schedule is because the numbers somehow worked out. Otherwise, DeLong’s looking at mostly unrelated phenomena that have some degree of overlap. Certainly there appears to be no valid scientific or medical justification for combining the data from the IDEA classifications of SLI and autism.
Then there’s the methodology chosen for trying to find correlations, described here:
Children who are vaccinated at age 2 years may not develop autism until they are older. To determine the prevalence of autism for a specific cohort of children, the vaccination data from when the children were 2 years old is compared with autism prevalence when they are 8 years old. The relevant vaccination data for children who were 8 years old in 2001 are those from 1995, when the children were 2 years old. For children who turned 8 years old in 2002, the relevant vaccination data are from 1996, and so on. The earliest available data–vaccination data from 1995–were matched with autism prevalence up to 2007.
Besides DeLong’s having fallen for the ecological fallacy (group level comparisons rather than individual-level comparisons), she doesn’t provide much in the way of a good justification for why she chose ages 2 and 8 as their vaccine time point and prevalence time point. Then there’s the issue of confounders. DeLong tried to control for ethnicity, but in explicably she used the CDC’s National Immunization Survey rather than, say, U.S. Census data to derive ethnicity figures. Other potential confounders examined included family income, other disabilities, and the number of pediatricians in each state. Of course, states range in size from small to very large, and it can easily be argued that state level data are not “fine” enough to be used for this purpose. After all, many states are quite large, with huge differences in urbanicity. Think, for instance, California, with several large cities separated by huge swaths of rural and mountainous land. Or think Pennsylvania, which is in essence a 360 mile wide state with two very large cities, one east and one west, and several medium-sized cities clustered mostly in the east, all separated by miles upon miles of farm land or mountains. Urbanicity, as you might recall, can have a huge effect on the number of autism diagnoses, as I discussed three years ago. Naturally, DeLong made no attempt to control for urbanicity.
In other words, there’s no reason to put any real credence in this study, especially given how small the observed effect appears to be.
After reading this study, I’m left wondering why on earth DeLong did it. After all, most of her papers appear to have to do with banking, the FDIC, and financial risk taking. Why did she embarrass herself so by moving out of her specialty and producing such a craptacular study? (This leaves aside why the Journal of Toxicology and Environmental Health journal has such craptacular peer review that it actually accepted this study for publication.) After all, I would never think of trying to do a paper on economics or business and expect it to be accepted to a journal in the relevant academic discipline. (As Dirty Harry Callahan once said, “A man’s got to know his limitations,” and I do, for the most part, know my limitations.) One clue comes from the fact that it is DeLong herself who signed her name to the AoA post pimping her study. The second clue comes from her saying about Jake Crosby in a comment at AoA:
You are indeed an inspiration. We’re delighted you are putting your many talents to very good use.
OK, so DeLong’s not a very good judge of character or talent.
The third and final clue comes from the observation that DeLong is on the Executive Board of the anti-vaccine group SafeMinds and is described there thusly:
Dr. Gayle DeLong is a parent of two girls with autism. Starting in May 2005, her family began biomedical interventions to treat the girls’ illness. Both girls have benefited greatly from supplements, diet, chelation, and hyperbaric oxygen therapy. Gayle holds a Ph.D. in international business and finance from New York University as well as an International Master’s in Business Administration from the University of South Carolina. She teaches international finance at Baruch College, City University of New York. She serves on SafeMind’s research committee. She has attended rallies in Washington, DC to promote safer vaccines and spoken against adding vaccines to New Jersey’s mandated schedule at a public hearing in Trenton, NJ. She lives with her husband and two daughters in Morristown, NJ.
And there you go. Like Laura Hewitson, an academic with an autistic child whose belief in biomedical woo led her to destroy her scientific career, DeLong has tarnished her own reputation and publication record by letting her belief in the scientifically discredited idea that vaccines cause autism lead her “down the rabbit hole” of pseudoscience. All she’s managed to do for all her effort is to produce another poor quality paper to which anti-vaccine zealots will point as “evidence” that vaccines cause autism. While she is lauded by pseudoscience supporters, DeLong’s article will, in the scientific community, fade into the oblivion it so richly deserves.
Delong G (2011). A Positive Association found between Autism Prevalence and Childhood Vaccination uptake across the U.S. Population. Journal of toxicology and environmental health. Part A, 74 (14), 903-16 PMID: 21623535