Must Read Paper on fMRI -and- The Worst fMRI Science Journalism Ever

There is a must-read paper in Nature about the limits of functional MRI as an experimental tool by one of its pioneers, Nikos Logothetis. (Also discussed by Jonah and Vaughan.)

This paper is pretty technical, but Logothetis hits the important points of what it is we think we are actually measuring using the fMRI. Also, he notes that the difficulty in interpreting fMRI data lies in the fact that you have to make assumptions about network architecture that may or may not be true. Other experiments are required to confirm the validity of these assumptions.

Here is his good summary of the limits of fMRI:

The limitations of fMRI are not related to physics or poor engineering, and are unlikely to be resolved by increasing the sophistication and power of the scanners; they are instead due to the circuitry and functional organization of the brain, as well as to inappropriate experimental protocols that ignore this organization. The fMRI signal cannot easily differentiate between function-specific processing and neuromodulation, between bottom-up and top-down signals, and it may potentially confuse excitation and inhibition. The magnitude of the fMRI signal cannot be quantified to reflect accurately differences between brain regions, or between tasks within the same region. The origin of the latter problem is not due to our current inability to estimate accurately cerebral metabolic rate of oxygen (CMRO2) from the BOLD signal, but to the fact that haemodynamic responses are sensitive to the size of the activated population, which may change as the sparsity of neural representations varies spatially and temporally. In cortical regions in which stimulus- or task-related perceptual or cognitive capacities are sparsely represented (for example, instantiated in the activity of a very small number of neurons), volume transmission...-- which probably underlies the altered states of motivation, attention, learning and memory -- may dominate haemodynamic responses and make it impossible to deduce the exact role of the area in the task at hand.

I also like this quote that captures the social controversy that has developed over fMRI:

In humans, fMRI is used routinely not just to study sensory processing or control of action, but also to draw provocative conclusions about the neural mechanisms of cognitive capacities, ranging from recognition and memory to pondering ethical dilemmas. Its popular fascination is reflected in countless articles in the press speculating on potential applications, and seeming to indicate that with fMRI we can read minds better than direct tests of behaviour itself. Unsurprisingly, criticism has been just as vigorous, both among scientists and the public. In fact, fMRI is not and will never be a mind reader, as some of the proponents of decoding-based methods suggest, nor is it a worthless and non-informative 'neophrenology' that is condemned to fail, as has been occasionally argued. (Emphasis mine.)

I really get this statement. Whenever I read an fMRI study, I have to admit that I feel an ambivalence that places me closer to its habitual critics. This is a feeling that I am a little ashamed of and often have to work actively against, but I almost have a presumption against these studies.

Partly it is because in my experience fMRI studies have a major "wheat and chaff" issue. For every brilliantly designed, well-performed study with all of the proper controls, there are at least 10 that are total trash. This is probably true of a lot of disciplines, but I notice it with fMRI studies. In a lot of cases, the behavior hasn't been well-characterized before the person is put into the scanner, or the results of lesion studies have not been incorporated into the conclusions.

Partly it is because of the radical and wrong interpretations that often appear in the press about these studies. I have heard other bloggers call these OMGFMRI stories.

As a brief digression, I can give you a great example of an OMGFMRI. This article by Jeffrey Goldberg in the Atlantic is some of the worst science journalism I have ever read.

It features him talking with UCLA neuroscientist Marco Iacoboni while he is being put in the scanner. They are showing him pictures of political figures like Jimmy Carter, Barack Obama, and Hillary Clinton and trying to interpret what he "really" thinks about them by the different brain activations.

For example, when he shows amygdala activation after seeing Jimmy Carter:

The preliminary findings began to arrive a few days later, in a series of e-mails from Iacoboni. "Carter: big amygdala response on both sides! Jeff, do you fear this guy?" Fear might not be the most accurate term, but I worry about him a great deal. I'd recently given his book on Israel a negative review in The Washington Post. Score one for the fMRI.

The suggestion that amygdala activation means fear without caveats is horse shit of the worst order. One, the amygdala activates in response to salient stimuli regardless of whether they are good or bad. (Scientists call this the valence of the stimuli.) One would think that Dr. Iacoboni would have made that caveat clear. Two, the amygdala activates in comparison to what? What was the control condition? They don't even bother to say.

The story proceeds with interpretations of Barack Obama's activation -- "Your medial orbital cortex lights up. You must love him!" -- and Hillary Clinton's -- "Your dorsolateral prefrontal lights up. You must be conflicted about her!". These interpretations are equally nonsense. Medial orbital activation is associated with pairing reward and response under conditions of uncertainty; it is not just generally activated when you are happy about something. Dorsolateral prefrontal cortex is responsible for inhibition of inappropriate responses under cognitive control models of the prefrontal cortex, but it does other things under other models. This makes the interpretation of why Goldberg activated his dlPFC in response to Hillary nearly impossible to make.

My favorite is when they showed the author a picture of Edie Falco and interpreted it thusly:

Iacoboni told me that there was one other troubling finding, the one concerning Edie Falco. "Your medial orbito-frontal cortex, anterior cingulate cortex, and ventral striatum are all activated. Wow! What will your wife say?"

Apparently my brain -- or the part of it I now refer to as my prefrontal Spitzer lobe -- finds the sight of Edie Falco somewhat exciting. "Both the medial orbito-frontal cortex and the ventral striatum process reward -- sex is, with food, the primary reward -- and the anterior cingulate cortex often activates for internal conflict." He went on, "People watching erotic pictures -- and you watching Falco -- are aroused but kind of feel guilty for being aroused, or simply feel guilty by being aroused in a brain scanner, with other people looking at their brain responses."

All of this leaves the reader with the impression that the brain is nothing but a collection of emotional mood rings whose activation can read by an appropriately trained analyst to figure out what you are really feeling. This is flagrantly untrue -- both for the particular regions and for the brain on the whole.

Frankly, I would like an explanation of how this article got written. I am willing to give Goldberg the benefit of the doubt. He is not a science journalist by trade, and it may be that this is what he was told these experiments meant. But Dr. Iacoboni is a neuroscientist. He is supposed to know better than this. Did he not think it appropriate to include the relevant caveats? Did he state them, and Golberg just ignored them to make a more glib article? Tripe like this doesn't just appear. It requires outright lies or a severe mis-communication, and I would kind of like to know which one it was.

(Iacoboni has gotten into interpretational hot-water with fellow cognitive neuroscientists before for piece's like this one. 17 of his colleagues with that particular article for similar reasons.)

This is the slop that people publish about fMRI. fMRI is not a mind-reading gadget. It can only tell us useful things about brain function in the context of well-designed experiments with appropriate controls, taking methodological considerations into account.

And I shudder to think that what Goldberg alludes to as the "great future for vanity scanning" will actually come to pass. If you are willing to pay for this, let me save you some money. Buy a Scientology e-meter. The results will be just as accurate.

Getting back to the issue of what to think about fMRI studies, however, I think that I may be being unfair to them because of the unfortunate interpretations in the media rather than their actual merits. To be sure, it is a powerful modality, and when used in concert with other techniques can tell us a great deal about the brain. I just wish that more people would take Logothetis' caveats to heart. The science would be much better if they did.

Anyway, read his piece. It is very illuminating to anyone interested in the details of this field.