I usually don’t comment on this type of study, but this time the hype is just too much for me: New Scientist describes the study as “a milestone in the quest to understand” the placebo effect; an article in ScienceNow quotes a psychiatrist saying that “the findings could have major implications for research design”. The article itself certainly doesn’t talk down its results, with the first sentence of the discussion stating:
The present study demonstrates that the magnitude of the placebo response [...] is tied to attenuated amygdala excitability, which in turn is linked to serotonergic genetic variation.
The problem? The study examined just 25 subjects, and if there’s one clear lesson from the history of candidate gene asociation studies it’s that such tiny studies are essentially worthless: systematic reviews of the field (e.g. here, here and here) have consistently found that the majority of such associations are never replicated, suggesting that positive results in small studies are substantially more likely to arise through a combination of chance, error and publication bias than through a genuine causal link.
It’s only relatively recently that genetic association studies have come of age, with the advent of agnostic genome-wide association studies, massive sample sizes, rigorous statistical frameworks and the use of independent replication cohorts. Unfortunately, it appears that such novelties haven’t yet permeated Uppsala University’s Department of Psychology – but that hasn’t stopped their study from generating media attention, in publications that should really have known better.
So don’t believe the hype: as a good rule of thumb, if a genetic association study contains fewer than 100 subjects, it’s not a “milestone” with “major implications” – in fact, you might as well simply pretend it doesn’t exist at all. (Many studies with more than 100 subjects are also crap, but at least there’s a chance they’re capturing a genuine causal variant.) I’m deadly serious about this. The field is so littered with the stinking carcasses of unreplicated candidate gene associations that it’s a reasonable default to simply assume that any small, unreplicated study is false.
Now, if only there was a way to get the science journalists responsible to internalise that little rule of thumb…