A Dead Salmon: Bestest Control Experiment EVAH!

When analyzing data, understanding the limitations of your data is critical. One of the things we need to understand is significance: how strong does an effect have to be to considered not a result of random chance. Typically, we assume that if an effect has a five percent or less probability of occurring due to random chance, then it is "significant." But significance becomes very problematic when making many simultaneous assessments. If we make one hundred assessments (e.g., comparisons) and not a single one is actually different (assume that the Omniscience of the Mad Biologist is operating here), then, on average, five of the 100 assessments will be "significant" even though there is no real phenomenon underlying the supposed significance.

I recently finished Cordelia Fine's Delusions of Gender: How Our Minds, Society, and Neurosexism Create Difference. An area of research that's in vogue (perhaps unjustifiably so) is seeing if male and female brain activation responds differently to certain stimuli, such as an emotionally disturbing picture. Then the results are ridiculously overinterpreted, and we conclude that boobies mean girls can't do math. Or something. So someone wanted to test just how reliable assessing significance of these scans (from Fine):

Could the sex differences in brain activation be spurious? When looking for changes in blood flow between two conditions, researchers search in thousands of tiny sections of the brain (called voxels), and many researchers are now arguing that the threshold commonly set for declaring that a difference is "significant" just isn't high enough.

And here's the Bestest Control Experiment EVAH! (italics mine):

To illustrate this point, some researchers recently scanned an Atlantic salmon while showing it emotionally charged photographs. The salmon-which, by the way, "was not alive at the time of scanning"-was "asked to determine what emotion the individual in the photo must have been experiencing." Using standard statistical procedures, they significant brain activity in one small region of the dead fish's brain while it performed the empathizing task, compared with brain activity during "rest." The researchers conclude not that this particular region of the brain is involved in postmortem piscine empathizing, but that the kind of statistical thresholds commonly used in neuroimaging studies (including Witelson's emotion-matching study) are inadequate because they allow too many spurious results through the net.

But did the salmon speak English? DUH! BAD EXPERIMENT.

This is awesome.

Cited article: Bennett CM, MB Miller and GL Wolford. 2009. Neural correlates of interspecies perspective taking in the post-mortem Atlantic Salmon: an argument for multiple comparisons correction. NeuroImage, Volume 47, Supplement 1, July 2009, Pages S39-S41. doi:10.1016/S1053-8119(09)71202-9

More like this

One of my favourite events in the science calendar is always the Ig Nobel Prize ceremony, which was held last night in Cambridge, MA. For those that don't know, the Ig Nobels celebrate the odd and unusual in scientific research, both genuine and not-quite. The Ig Nobel Prizes honor achievements…
Another list for your reading, gift giving and collection development pleasure. Atlantic: Great Sea Battles, Heroic Discoveries, Titanic Storms, and a Vast Ocean of a Million Stories by Simon Winchester Colossus: Hoover Dam and the Making of the American Century by Michael Hiltzik Deep Blue Home:…
The ability to interpret other peoples' emotions is vital for social interactions. We recognize emotions in others by observing their body language and facial expressions. The voice also betrays one's emotional state: words spoken in anger have a different rhythm, stress and intonation than those…
The podcast of Desiree Schell's interviews with me (recorded) and Cordelia Fine (author of Delusions of Gender: How Our Minds, Society, and Neurosexism Create Difference) ... is now available here. Check it out!

It appears your cited article has a bad link.

OMG WIN!!!!

The link incorporates a user ID, so it won't work for anyoneelse, sadly. But if you go to the www.sciencedirect.com site, it is possible to search for author Bennett, journal NeuroImage, vol 47, and find it anyway. Whether or not you can access the full thing unfortunately depends on whether or not you have a subscription to the ScienceDirect articles, I think.

By Luna_the_cat (not verified) on 14 Dec 2010 #permalink

Thanks. I liked the idea of a dead fish resting. ;)

As a Bayesian, I am not too impressed with Fisherian hypothesis testing. I do not think that it is useless, but that it should be interpreted differently. If we believed that empathy was an activity of specific parts of a dead salmon's brain, then the experiment indicates which parts they might be. Establishing our initial assumption is a separate question.

Often Fisherian hypothesis testing is applied to "null" hypotheses that no one believes, anyway. In such a case, no harm, no foul. :) I do not think that that applies to hypotheses about men's and women's brains, however.

I want to be the grad student that asked the salmon questions. This should end up on YouTube.

Wait, so was the salmon resting, or perhaps pining for the fjords?

Neither, it was an EX-salmon!

h/t Monty Python for those in the slow lane.

By Rob Monkey (not verified) on 14 Dec 2010 #permalink

It shouldn't be difficult to design MRI tests for (human, living) emotion and cognition where the testers don't know the gender of the subjects.

Have the testers in one room, giving instructions by telephone or internet to an assistant in the MRI room. The assistant does the tests and transmits the results, without mentioning the gender of the person they're scanning.

The tester on the other side of the hospital interprets the data 'at one remove' from the subject, who's identified onlt by a number. Their own biases and expectations of gender are bypassed.

Just a thought.

But was it a male salmon or a female salmon? That really could account for all the test results. And did they cook and eat the test subject after the experiment was done? Just how thorough was this testing?

By Silent Service (not verified) on 15 Dec 2010 #permalink

As Withnail put it: "Don't threaten me with a dead fish!"

Brilliant. There ought to be whole departments of people slinking around with bags over their heads. 'Course, this will be ignored with a sniff of hauture.