The Science of Mind-Reading: SVMs Extract Intentions from Neural Activity (video)

For the basics about multivariate fMRI "mind-reading" techniques, see the video below. Some of it is based on this 2007 Haynes et al paper from Current Biology, described in more detail following the video.

What Haynes et al have done is to ask 8 subjects to freely decide either to add or subtract two numbers, and to select among 4 options an answer corresponding to the task they chose. After repeating this process many times, the authors ran a pattern classifier on the metabolic activity recorded in the brain.

This pattern classifier was run on the unsmoothed fMRI data - smoothing is normally applied because fMRI is thought to be a relatively noisy recording technique. Critically, the use of a pattern classifier allows the use of unsmoothed data (and in fact requires it) because buried within the noise is a distributed signal reflecting the distributed neural patterns encoding the subject's intention. Such data is presumably lost in averaging/smoothing operations.

Haynes et al trained their pattern classifier (a linear support vector machine) using a "multivariate searchlight" (described here) approach. This means that for every recorded voxel, they fed the classifier information about both that voxel and those surrounding it. The classifier was trained on 87.5% of the data (using 8-fold validation), and maps were produced of the classifier's accuracy at each voxel in the brain. These "accuracy maps" were averaged across subjects to produce the following figure:

As you can see above, the results showed that intention is decodable both prior to and during the intended response in numerous regions in the prefrontal cortex. In particular, the anterior & posterior medial prefrontal cortices as well as lateral frontopolar cortex, right middle frontal gyrus, and left operculum contained information that allowed the decoding of intentions at a level significantly above chance. Intentions prior to responses were also decoded based on activity in the temporo-parietal junction, although it is not illustrated in the above figure (see the supporting online material here). Much debate focuses on the precise roles of these regions, but their involvement here would be predicted by the majority of cognitive neuroscientists.

Conspicuously absent from these maps is the intraparietal sulcus (IPS), which has been argued to reflect numerical processing. An interesting possibility is that the numerical processing accomplished by this region cannot be distinguished based on the numerical operation (addition vs. subtraction), which would support a process-independent representation of quantity. Note that this conflicts with some theories of numerical processing in the IPS.

What's fairly amazing about this work is that they used a pretty standard scanner (only 3 tesla) with a reasonable sampling time (a TR of just over 2.7s). Peter Bandettini has suggested that this unsmoothed multivariate approach would benefit from higher resolution MRI, but Haynes et al have demonstrated surprising success with much more widely-available technology.

Related Posts:
Soon we'll be reading your mind!
Attention vs. Intention: Dissociations in Parietal Cortex
Action without Intention: Parietal Damage Alters Intention Awareness