The Frontal Cortex

More Voodoo

I’ve got an interview with Ed Vul, the lead author of the recent paper on “Voodoo Correlations in Social Neuroscience,” over at Scientific American. Since the paper hit the web, it has provoked a flurry of rebuttals and responses. If you’d like a balanced perspective on the issue – and it’s worth pointing out that this issue isn’t unique to social neuroscience, but has implications for fMRI research in general – I’d suggest reading the interview and then perusing this eloquent and fiery response by Matthew Lieberman, Elliot Berkman and Tor Wager.

LEHRER: Your paper has prompted a great deal of debate among social neuroscientists, and some of the scientists have issued a rebuttal of your paper. (You have since rebutted this rebuttal.) What do you hope this debate leads to? What methodological changes would you like to see adopted by social neuroscientists using fMRI?

VUL: The debate we have spurred is quite interesting to watch. At first some of the authors whose papers we criticized challenged our statistical point, but–for good reason–that line of argument doesn’t seem to have caught on. Right now, so far as I know, everyone seems to concede that the analysis used in these studies was not kosher, in the sense of providing correlation numbers that can be taken seriously. Instead, we are mostly hearing a couple of other arguments at this point.

One is that the correlation values themselves don’t really matter–it’s just the fact there is a correlation in a certain spot in the head that matters. I don’t agree with this observation at all, and we think the fact that many of these papers appeared in such high profile places is because editors were (justifiably) impressed with big effects. If one can account for, say, three quarters of individual differences in something important such as anxiety or empathy–obviously, that’s a real breakthrough, and it tells you not only where future research ought to look, but also where it shouldn’t. On the other hand, if it’s just 3 percent of the variance, that’s a whole lot less impressive, and may reflect much more indirect kinds of associations.

I have also heard some people complain that even if we are right on the mathematical point, we presented our argument in a bit of a rough-mannered way–criticizing particular articles and drawing unfavorable outside attention to the field, and using the humorous term “voodoo.” We were as surprised as anyone by how much interest our paper sparked. Evidently it spread sort of “virally”–one neuroscientist we know said he got seven copies sent to him (none of them by us). The good side is that people are thinking harder now about how they do their analyses. The bad side is that all this publicity has left some authors feeling embarrassed and picked on. In our view, the statistical issues of independence and multiple comparisons are full of tricky pitfalls–we do not suggest that these were stupid mistakes people were making, and we regret hurting anyone’s feelings. I don’t think, however, that it would have made sense to write an article that did not “name names,” because if the scientific literature is to guide future research decisions, people have to know which results can be relied upon, and which cannot. (In fact, we suspect we only flagged a small fraction of the papers that have these problems, and some are in other fields, such as neurogenetics, cognitive neuroscience more broadly, and others.)

Comments

  1. #1 Luciana Christante
    January 29, 2009

    I’m thinking how terrified may be some editors and peers who are currently reviewing this sort of fMRI studies just submitted, not yet accepted. Let’s see what’s gonna happen next…

  2. #2 Thiri
    January 29, 2009

    Hi, I just found out this blog. Just want to say ‘proust was a neuroscientist’ is one of my favorite book! I major in biology and minor in studio arts, so I appreciate it very much. I will come back to read the blog later.

    Cheers,
    Thiri

  3. #3 Anibal
    January 30, 2009

    About the issue (correlation is no causation) not only is not exclusive of social neuroscience, even not exclusive of fMRI studies.

    If we remember the study of Fowler and Christakis on how happines propagates throughout social networks, Jason M. Fletcher and Ethan Cohen-Cole accused Fowler and Christakis of finding patterns where there isn´t nothing.

    Detecting implausible social network effects in acne, height,
    and headaches: longitudinal analysis

  4. #4 Rod
    February 2, 2009

    How about anti-depressants and long term effects? SSRI’s?

  5. #5 Neuroskeptic
    February 4, 2009

    Anibal: That’s not the issue. Well, it’s an issue, and an important one, but Vul et al. is nothing to do with it

    I’ve just put up a fairly long discussion of the paper in layman’s terms here…

  6. #6 Neuroskeptic
    February 4, 2009

    Here in fact! – typo.

  7. #7 Azariah
    February 13, 2009

    Уважаемый администратор, можно ли я данную статью задействую для создания реферата?

  8. #8 Tasanee
    February 15, 2009

    А вы эту информацию проверяли? Где гарантия того, что вы изложили точную информацию?

  9. #9 Barrick
    February 19, 2009

    ВЫ сами пишите статьи или рерайтете какой то материал? Или же у вас другой источник для постов?

  10. #10 Форекс блог
    August 20, 2010

    Да… Действительно очень полезная, а главное – очень важная информация!

  11. #11 Caroline
    May 16, 2011

    hi-ya, I like all your posts, keep them coming.

  12. #12 computer notdienst
    August 14, 2011

    Ganz toller Beitrag. Ich bin ganz gespannt auf weitere Artikel dieser Art.

  13. #13 Henry Galauiz
    August 15, 2011

    I wanted to address this short comment to thank you very much again on your outstanding result that you’ve highlighted here. I certainly love the generosity of people like you to release what many people would have just marketed as an ebook. These inspirational themes served alongside a good way to understand that other people like myself want to learn much more about the subject of this. I think there are a lot more attractive opportunities in the future for those who vigilantly read your blog.

  14. #14 fitness research
    August 20, 2011

    Health benefits are generally an interesting subject matter mainly because it entirely possible that every person seems to another country for replies and definitely this is also true throughout the us. Actually, the actual questionable subject matter of health advantages has become as well as may be such a very hot subject matter this well may at least somewhat decide whom our following leader will be.

  15. #15 Joann Guiliani
    August 22, 2011

    you look really cute in ur glasses!

  16. #16 Galvanni Shop
    August 28, 2011

    Are all of them articles written you or do you hire an author?

  17. #17 cut the rope online
    August 29, 2011

    Appreciate it for sharing More Voodoo : The Frontal Cortex with us keep update bro love your article about More Voodoo : The Frontal Cortex .

  18. #18 where to buy kindle
    August 31, 2011

    self help master shows Neuro-linguistic programming methods to men and women

  19. #19 Jayne Gardley
    September 12, 2011

    I agree. For a beginner at this, your video is visible, step by step, and I can follow what you are saying. I wish users of youtube would learn to do videos like yours because you never know who will be looking at the video aand wanting to understand what the person is presenting. Thanks for your clarity!!

  20. #20 kore dizileri
    September 15, 2011

    hey there and thank you for your info – I’ve certainly picked up anything new from right here. I did however expertise several technical points using this site, since I experienced to reload the site many times previous to I could get it to load correctly. I had been wondering if your web hosting is OK? Not that I’m complaining, but slow loading instances times will very frequently affect your placement in google and could damage your high-quality score if advertising and marketing with Adwords. Well I’m adding this RSS to my e-mail and can look out for a lot more of your respective intriguing content. Ensure that you update this again soon..

  21. #21 Frederick Boudrieau
    September 16, 2011

    **ALL SPOTLIGHTS OPEN* thread now open for posting!PLEASE post ALL Old Glory photos in our FLAG DAY thread.Posting in ANY THREAD makes you eligible for the front page.Thank you for posting in: AMERICA, AMERICA!(POST 1 ~ COMMENT on 2) NEW RULES!

  22. #22 Samuel Wickson
    September 17, 2011

    Your icecream is a staple in our house. And being from Pittsburgh, I have to say I love the new Blitzburgh Crunch!

  23. #23 rankingoptimierung
    September 18, 2011

    Ganz toller Artikel. Ich freue mich auf weitere Artikel dieser Art.

  24. #24 Yves Saint Laurent Shoes
    October 5, 2011

    alguien sabe como se llama esta cancion es muy buena pero no tengo idea de como se llama

  25. #25 Tiffany
    October 10, 2011

    Absolutely NO Discovery Channel employee in that building is armed to defend himself, right?

  26. #26 Knock Knock Jokes
    October 10, 2011

    Thankyou for sharing More Voodoo : The Frontal Cortex with us keep update bro love your article about More Voodoo : The Frontal Cortex .

  27. #27 scrapebox list
    October 25, 2011

    I believe this can be the single most vital information personally. And i am glad reading your article. But should remark on few general things, The positioning style is ideal, the articles is basically excellent : D. Good job, cheers

  28. #28 SportsKeen
    November 1, 2011

    Howdy. I clicked on your website by luck when I was browsing Bing for something off topic. I am pleased I found this material. Its so strenuous to find anybody who knows what they are talking about. Much Obliged.

  29. #29 liquid clenbuterol
    November 8, 2011

    Wohh exactly what I was looking for, appreciate it for posting.

  30. #30 ricDugDiuby
    November 9, 2011

    Автоматические торговые системы, служат как помощником, так и полностью автоматическим инструментом для ведения прибыльной торговли на рынке форекс. Ещё данные системы называют (советники, торговые роботы, МТС) они подразумевают под собой программу, в которой заложен чёткий алгоритм, по которому она открывает и закрывает сделки в терминале InstaTrader без участия человека.
    На данный момент советники (торговые роботы) пользуются большой популярностью среди трейдеров, так как благодаря им, у трейдера уже нет необходимости находиться перед монитором круглосуточно, советник сам анализирует ситуацию на рынке, принимая при этом взвешенные решения, и совершает сделки круглосуточно.
    Продолжение статьи forextoolss . blogspot . com

  31. #31 escort bayan ankara
    November 11, 2011

    I don’t even know how I ended up here, but I thought this post was great. I don’t know who you are but certainly you’re going to a famous blogger if you are not already ;) Cheers!

  32. #32 Android Apps
    November 13, 2011

    Great goods from you, man. More Voodoo : The Frontal Cortex I have understand your stuff previous to and you are just too wonderful. I actually like what you’ve acquired here, really like what you are stating and the way in which you say it. You make it enjoyable and you still take care of to keep it wise. I can’t wait to read far more from you. This is really a tremendous More Voodoo : The Frontal Cortex informations.

  33. #33 Android Apps
    November 13, 2011

    Excellent goods from you, man. More Voodoo : The Frontal Cortex I have understand your stuff previous to and you’re just too excellent. I actually like what you’ve acquired here, certainly like what you’re stating and the way in which you say it. You make it enjoyable and you still care for to keep it wise. I can’t wait to read far more from you. This is really a wonderful More Voodoo : The Frontal Cortex informations.

  34. #34 duke nukem apk
    November 24, 2011

    Magnificent goods from you, man. More Voodoo : The Frontal Cortex I’ve understand your stuff previous to and you are just too excellent. I actually like what you have acquired here, really like what you are stating and the way in which you say it. You make it enjoyable and you still care for to keep it wise. I can’t wait to read far more from you. This is actually a terrific More Voodoo : The Frontal Cortex informations.

  35. #35 LA Weight Loss
    November 30, 2011

    Fantastic goods from you, man. More Voodoo : The Frontal Cortex I have understand your stuff previous to and you are just extremely excellent. I really like what you have acquired here, certainly like what you’re stating and the way in which you say it. You make it entertaining and you still care for to keep it sensible. I can not wait to read much more from you. This is actually a great More Voodoo : The Frontal Cortex informations.