Must Read Paper on fMRI -and- The Worst fMRI Science Journalism Ever

There is a must-read paper in Nature about the limits of functional MRI as an experimental tool by one of its pioneers, Nikos Logothetis. (Also discussed by Jonah and Vaughan.)

This paper is pretty technical, but Logothetis hits the important points of what it is we think we are actually measuring using the fMRI. Also, he notes that the difficulty in interpreting fMRI data lies in the fact that you have to make assumptions about network architecture that may or may not be true. Other experiments are required to confirm the validity of these assumptions.

Here is his good summary of the limits of fMRI:

The limitations of fMRI are not related to physics or poor engineering, and are unlikely to be resolved by increasing the sophistication and power of the scanners; they are instead due to the circuitry and functional organization of the brain, as well as to inappropriate experimental protocols that ignore this organization. The fMRI signal cannot easily differentiate between function-specific processing and neuromodulation, between bottom-up and top-down signals, and it may potentially confuse excitation and inhibition. The magnitude of the fMRI signal cannot be quantified to reflect accurately differences between brain regions, or between tasks within the same region. The origin of the latter problem is not due to our current inability to estimate accurately cerebral metabolic rate of oxygen (CMRO2) from the BOLD signal, but to the fact that haemodynamic responses are sensitive to the size of the activated population, which may change as the sparsity of neural representations varies spatially and temporally. In cortical regions in which stimulus- or task-related perceptual or cognitive capacities are sparsely represented (for example, instantiated in the activity of a very small number of neurons), volume transmission...-- which probably underlies the altered states of motivation, attention, learning and memory -- may dominate haemodynamic responses and make it impossible to deduce the exact role of the area in the task at hand.

I also like this quote that captures the social controversy that has developed over fMRI:

In humans, fMRI is used routinely not just to study sensory processing or control of action, but also to draw provocative conclusions about the neural mechanisms of cognitive capacities, ranging from recognition and memory to pondering ethical dilemmas. Its popular fascination is reflected in countless articles in the press speculating on potential applications, and seeming to indicate that with fMRI we can read minds better than direct tests of behaviour itself. Unsurprisingly, criticism has been just as vigorous, both among scientists and the public. In fact, fMRI is not and will never be a mind reader, as some of the proponents of decoding-based methods suggest, nor is it a worthless and non-informative 'neophrenology' that is condemned to fail, as has been occasionally argued. (Emphasis mine.)

I really get this statement. Whenever I read an fMRI study, I have to admit that I feel an ambivalence that places me closer to its habitual critics. This is a feeling that I am a little ashamed of and often have to work actively against, but I almost have a presumption against these studies.

Partly it is because in my experience fMRI studies have a major "wheat and chaff" issue. For every brilliantly designed, well-performed study with all of the proper controls, there are at least 10 that are total trash. This is probably true of a lot of disciplines, but I notice it with fMRI studies. In a lot of cases, the behavior hasn't been well-characterized before the person is put into the scanner, or the results of lesion studies have not been incorporated into the conclusions.

Partly it is because of the radical and wrong interpretations that often appear in the press about these studies. I have heard other bloggers call these OMGFMRI stories.


As a brief digression, I can give you a great example of an OMGFMRI. This article by Jeffrey Goldberg in the Atlantic is some of the worst science journalism I have ever read.

It features him talking with UCLA neuroscientist Marco Iacoboni while he is being put in the scanner. They are showing him pictures of political figures like Jimmy Carter, Barack Obama, and Hillary Clinton and trying to interpret what he "really" thinks about them by the different brain activations.

For example, when he shows amygdala activation after seeing Jimmy Carter:

The preliminary findings began to arrive a few days later, in a series of e-mails from Iacoboni. "Carter: big amygdala response on both sides! Jeff, do you fear this guy?" Fear might not be the most accurate term, but I worry about him a great deal. I'd recently given his book on Israel a negative review in The Washington Post. Score one for the fMRI.

The suggestion that amygdala activation means fear without caveats is horse shit of the worst order. One, the amygdala activates in response to salient stimuli regardless of whether they are good or bad. (Scientists call this the valence of the stimuli.) One would think that Dr. Iacoboni would have made that caveat clear. Two, the amygdala activates in comparison to what? What was the control condition? They don't even bother to say.

The story proceeds with interpretations of Barack Obama's activation -- "Your medial orbital cortex lights up. You must love him!" -- and Hillary Clinton's -- "Your dorsolateral prefrontal lights up. You must be conflicted about her!". These interpretations are equally nonsense. Medial orbital activation is associated with pairing reward and response under conditions of uncertainty; it is not just generally activated when you are happy about something. Dorsolateral prefrontal cortex is responsible for inhibition of inappropriate responses under cognitive control models of the prefrontal cortex, but it does other things under other models. This makes the interpretation of why Goldberg activated his dlPFC in response to Hillary nearly impossible to make.

My favorite is when they showed the author a picture of Edie Falco and interpreted it thusly:

Iacoboni told me that there was one other troubling finding, the one concerning Edie Falco. "Your medial orbito-frontal cortex, anterior cingulate cortex, and ventral striatum are all activated. Wow! What will your wife say?"

Apparently my brain -- or the part of it I now refer to as my prefrontal Spitzer lobe -- finds the sight of Edie Falco somewhat exciting. "Both the medial orbito-frontal cortex and the ventral striatum process reward -- sex is, with food, the primary reward -- and the anterior cingulate cortex often activates for internal conflict." He went on, "People watching erotic pictures -- and you watching Falco -- are aroused but kind of feel guilty for being aroused, or simply feel guilty by being aroused in a brain scanner, with other people looking at their brain responses."

All of this leaves the reader with the impression that the brain is nothing but a collection of emotional mood rings whose activation can read by an appropriately trained analyst to figure out what you are really feeling. This is flagrantly untrue -- both for the particular regions and for the brain on the whole.

Frankly, I would like an explanation of how this article got written. I am willing to give Goldberg the benefit of the doubt. He is not a science journalist by trade, and it may be that this is what he was told these experiments meant. But Dr. Iacoboni is a neuroscientist. He is supposed to know better than this. Did he not think it appropriate to include the relevant caveats? Did he state them, and Golberg just ignored them to make a more glib article? Tripe like this doesn't just appear. It requires outright lies or a severe mis-communication, and I would kind of like to know which one it was.

(Iacoboni has gotten into interpretational hot-water with fellow cognitive neuroscientists before for piece's like this one. 17 of his colleagues with that particular article for similar reasons.)

This is the slop that people publish about fMRI. fMRI is not a mind-reading gadget. It can only tell us useful things about brain function in the context of well-designed experiments with appropriate controls, taking methodological considerations into account.

And I shudder to think that what Goldberg alludes to as the "great future for vanity scanning" will actually come to pass. If you are willing to pay for this, let me save you some money. Buy a Scientology e-meter. The results will be just as accurate.


Getting back to the issue of what to think about fMRI studies, however, I think that I may be being unfair to them because of the unfortunate interpretations in the media rather than their actual merits. To be sure, it is a powerful modality, and when used in concert with other techniques can tell us a great deal about the brain. I just wish that more people would take Logothetis' caveats to heart. The science would be much better if they did.

Anyway, read his piece. It is very illuminating to anyone interested in the details of this field.

Categories

More like this

There are 48 new articles published this week in PLoS ONE. It's hard to choose just a couple to highlight, so look around for what interests you (avian flu, the Plague?). How about these titles that piqued my interest: Neural Substrates of Spontaneous Musical Performance: An fMRI Study of Jazz…
There are 26 new articles in PLoS ONE today. As always, you should rate the articles, post notes and comments and send trackbacks when you blog about the papers. You can now also easily place articles on various social services (CiteULike, Mendeley, Connotea, Stumbleupon, Facebook and Digg) with…
There were 15 new articles in PLoS ONE published last night but I was too busy watching the election returns. As always, you should rate the articles, post notes and comments and send trackbacks when you blog about the papers. Here are my own picks for the week - you go and look for your own…
Today, we unveil a brand new PLoS ONE Collection - the Prokaryotic Genome Collection. The Collection was edited by Niyaz Ahmed, who wrote an introductory Overview. In other news, there are 17 new articles published last night and another 17 new articles published tonight in PLoS ONE. As always,…

Science journalism is much like all other journalism in that it is always biased towards sensationalism and is done with the least effort possible with more emphasis on graphics and photography than on getting even the right concepts in the right order.

I agree with you about the importance of the Logothetis paper, but I fear it may be overshadowed by the controversy over Goldberg's piece. I read that Iacaboni has defended the piece, which I find very disturbing. If neuroscientists, who should know better, exaggerate the implications of brain imaging I guess we shouldn't be surprised when the general public jumps to unjustified conclusions.Certainly this is just the sort of thing that people like Logothetis and Steven Rose are trying to prevent.

Ginger Campbell, MD
Creator and host of the Brain Science Podcast

I'm a science journalist; in fact, I have an advanced degree in science journalism. I have never met an editor who serves the same function as a peer reviewer for a science journal. And most magazines don't have fact checkers that are scientists. What we are taught to do is go after more than one source or expert as a way to ensure we're not being fed b.s. That's really all there is to science writing. If you don't already have expertise in the subject you're covering then it is very easy to write something patently false without knowing it. It'll be interesting to see if Goldberg addresses the criticisms to his article.

The price of a Scientology E-meter hardly compares to the price of an MRI machine. The E-meter training poses a larger barrier. Jake Young's confidence in E-meter accuracy clarifies his education in this alternative area.

Terryeo:

Actually, if anything an e-meter is vastly cheaper, if you take the time to build your own rather than let yourself get sucked into the $cientology vortex. The flipside is that the e-meter seems designed to create equipment artifacts -- between the can electrodes (inviting manipulations that fastened electrodes aren't prone to), the undamped meter (allowing all kinds of random chaotic twitching that would be unacceptable in real test equipment), and the nonsensical operating instructions, it seems rather unlikely that anyone would learn anything useful from an e-meter. People may be going wild with fMRI interpretations, but at least an fMRI is capable of doing something useful.

I'm mostly with you - that's a neat account. (Although even if a vanity scan doesn't read your mind, it could pick up something worth knowing anatomically, and so at least has a shot at being useful, which e-meters don't.)