I just wanted to draw attention to two fantastic blog posts that describe a new paper by Edward Vul, a grad student at MIT, and colleagues at UCSD. The first post comes from Vaughan over at MindHacks:
I’ve just come across a bombshell of a paper that looked at numerous headline studies on the cognitive neuroscience of social interaction and found that many contained statistically impossible or spurious correlations between behaviour and brain activity.
Social cognitive neuroscience is a hot new area and many of the headline studies use fMRI brain imaging to look at how activity in the brain is correlated with social decision-making or perception.
This new analysis, led by neuroscientist Edward Vul, was inspired by the fact that some of these correlations seem to good to be true, and so the research team investigated. The abstract of their study is below, and it’s powerful stuff – indicating that many of the results are due to flawed analyses.
If you’re not familiar with neuroimaging research it might be useful to know that what a ‘voxel’ is before reading the abstract.
Essentially, brain scanners digitally divide the scanned area into a block of tiny boxes and each one of these is called a voxel (think 3D pixel).
This allows the scans to be analysed by comparing the activity or tissue density in each voxel to another measure – which could be the same voxel during another scan, or it could be something entirely different, such as a measure of emotion or social decision-making.
The Neurocritic has much more. One noteworthy aspect of this paper is that it calls out a few dozen brain scanning studies for very public criticism. (Vul is a brave grad student.) After reading his analysis, I’m rather amazed that these many of these studies got through the peer-review process, and ended up getting published at such eminent journals!
This latest paper continues what might be called the “fMRI backlash,” although the backlash is really being driven by people within the brain scanning field who are simply trying to improve the quality of the average fMRI paper. Here’s how I described the backlash in a Globe article earlier this year:
The scanners, these critics say, excel at measuring certain types of brain activity, but are also effectively blind when it comes to the detection of more subtle aspects of cognition. As a result, the pictures that seem so precise are often deeply skewed snapshots of mental activity. Furthermore, one of the most common uses of brain scanners – taking a complex psychological phenomenon and pinning it to a particular bit of cortex – is now being criticized as a potentially serious oversimplification of how the brain works. These critics stress the interconnectivity of the brain, noting that virtually every thought and feeling emerges from the crosstalk of different areas spread across the cortex. If fMRI is a window into the soul, these scientists say, then the glass is very, very dirty.