The Frontal Cortex

Scientific Imperfections

Here at Scienceblogs, we spent a lot of time debunking various types of unscientific falsehoods (aka “woo,” religious believers, and the conservapedia.) As far as I’m concerned, that’s just great. The world always is always suffering from a shortage of skepticism. We need more empiricism and less certainty.

But it’s worth reminding ourselves of the obvious: peer-reviewed science is also vulnerable to bad biases, false suppositions and sloppy interpretations. Data doesn’t generate itself. Over at Overcoming Bias, they’ve compiled a short list of recent examples. Here are the most damning:

A recent PLoS Medicine looked at 111 studies of soft drinks, juice, and milk that cited funding sources. 22% had all industry funding, 47% had no industry funding, and 32% had mixed funding. … the proportion with unfavorable [to industry] conclusions was 0% for all industry funding versus 37% for no industry funding.

In 2005, the Journal of American Medical Association found that of medical studies since 1990 cited 1000 times or more, 1/3 were contradicted by replications, and 1/4 had no replication attempts. Of 49 highly cited original clinical research studies, 45 claimed that the intervention was effective. Of these, 7 (16%) were contradicted by subsequent studies, 7 others (16%) had found effects that were stronger than those of subsequent studies, 20 (44%) were replicated, and 11 (24%) remained largely unchallenged. Five of 6 highly-cited nonrandomized studies had been contradicted or had found stronger effects vs 9 of 39 randomized controlled trials (P = .008).

That JAMA paper is downright scary, and not because 32 percent of medical studies were later contradicted. What worries me is that a medical study can be cited more than 1000 times but never be repeated or tested. It’s not that scientists don’t make mistakes, it’s that the scientific process requires repetition and confirmation in order to catch these mistakes. But if oft-cited studies aren’t being tested, then what is?

Comments

  1. #1 quitter
    February 28, 2007

    The repetition thing doesn’t bother me as much when it comes to stuff in the clinical research field, because a lot of these studies are huge multicenter trials that take a decade or more to complete. You can’t exactly wait for a repeat trial (which isn’t going to get published anyway if it isn’t novel) and ignore information that could be beneficial.

    We can’t beat up science over something like that, it’s just too difficult to due given the complexity of some of these studies.

  2. #2 Agnostic
    March 1, 2007

    I read a nice suggestion by A. Richard Palmer (a biologist) that the upper-tier journals should include a “Replications” section to encourage more people to confirm or disconfirm previous results. Maybe even a journal whose sole purpose was replicatory studies.

    Also, he suggested that grad students be required to conduct a replicatory study to receive their degree — pretty simple way to be introduced to the dirty work of research. Plus, if you get totatlly different results, you’ll be known for at least that much.

  3. #3 murat
    July 16, 2007

    Kurda Ak?l Gne? Do?ana Kadar Laz?md?r.

    Biz sizin orta??n?z De?iliz Hara kesebilece?iniz Adamlarda De?ilim Hele Hele Serseri Hi de?iliz.

  4. #4 oyunlar 1
    July 17, 2007

    In 2005, the Journal of American Medical Association found that of medical studies since 1990 cited 1000 times or more, 1/3 were contradicted by replications, and 1/4 had no replication attempts

    Pek inandirici gelmiyo bu yazi bana.?

  5. #5 izlee
    July 20, 2007

    Bir Gemi Bat?yorsa O Gemide Ben Yokum Demektir..

  6. #6 izlee
    August 6, 2007

    Biz sizin orta??n?z De?iliz Hara kesebilece?iniz Adamlarda De?ilim Hele Hele Serseri Hi de?iliz.

  7. #7 oyunlar 1
    September 10, 2007

    niye boyle yapiyosunuz halen anlayamadim cortx

  8. #8 oyunlar1
    October 13, 2007

    kankay?z be abi yapmayin yani ufak tefek hatalarimiz olabilir.

  9. #9 kral oyun
    October 13, 2007

    tamam kardes kizmanapalim harika bi icerik good.

  10. #10 Flash
    September 4, 2008

    Great article thanks.

  11. #11 oyun
    October 21, 2008

    Thanks very nice.

  12. #12 oyunboy
    October 31, 2008

    thanks very nice

  13. #13 oteller
    January 19, 2011

    Hello, thank you for your share. i cant find your twitter account on site, can you please share, i want became your follower in twitter.

  14. #14 fun kids games
    May 6, 2011

    My English communication isn’t so good butprobably I understand everything. Thank u so well for that awesome blog message. I jolly be glad reading it. I think you happen to be a superb author. At this time added your blog to my favorites and will appear back again to ur web web page. Hold up that magic work. I residence to see much more soon.

  15. #15 Wilbert Deluco
    June 6, 2011

    I do not know if it’s just me or if perhaps everyone else encountering problems with your blog. It appears as though some of the text in your content are running off the screen. Can someone else please comment and let me know if this is happening to them as well? This could be a problem with my internet browser because I’ve had this happen before. Thank you

  16. #16 Shantel Kauo
    June 20, 2011

    wheres your own feed? probably hidden in basic sight understanding me lol.

  17. #17 remy hair
    June 23, 2011

    DON’T WASTE YOUR TIME AT THE BEAUTY SUPPLY. WE HAVE THE BEST HAIR IN AMERICA

  18. #18 Dot Reihe
    July 7, 2011

    My neighbor and I were debating this specific issue, he is generally looking to show me wrong. Your view on this is wonderful and exactly how I really believe. I just sent him this blog to show him your opinion. After browsing around your site I book-marked and will be back to read your updates!

  19. #19 javier
    July 9, 2011

    I am glad to be one of many visitors on this outstanding site (:, thanks for putting up.

  20. #20 Glendora Liesman
    July 12, 2011

    Things you brought up made . Yet, consider this, what happens if you added a little bit more? What i’m saying is, I do not want to tell you ways to write ur blog, however if you added something which can get people’s awareness? Simply like a video or maybe a photo or 2 to have viewers interested about what you mentioned.

  21. #21 Cosmetica Economica
    July 12, 2011

    Tulips are fantastic. I always make analysts for supper

  22. #22 Burger King kostenlos
    July 14, 2011

    [….] Die besten Gutscheine für Burger King Hamburg Berlin München [….]

  23. #23 Taylor Gang Clothing
    July 16, 2011

    I wonder why Wiz never pushed Cabin Fever as an album. Added some songs from Rolling Papers and it’ll be a really good album.

  24. #24 Taylor Gang Hoodies
    July 16, 2011

    I think Wiz Khalifa is on par with rapper like Lil Wayne and Eminem because his movement is just so strong. Everybody I know rep Taylor Gang.

  25. #25 umschulung altenpflege
    July 16, 2011

    hey, exalted blog on jobs. parallel helped.

  26. #26 Rae Grriffin
    July 19, 2011

    Your place is valueble for me. Thanks!…

  27. #27 Gendon
    July 19, 2011

    Hi, I totally agree with you! Just obey my previous comment

  28. #28 Mastic Siding
    July 20, 2011

    This is an amazing post, but sometimes I think I have to argue against your takes and ideas on certain subjects. However, I just can’t stop reading what you write because it is written so flawless and elegantly. It’s great to know that other people still take the time to think about the things they are posting about. I have a mastic siding related website and if you are interested it would be wonderful if you could check it out. I am just starting out and I could really use a couple tips or pointers. Hopefully one day my site can be as good as yours and I will have people that actually stick around. Anyway apologies for the lengthy comment and I hope you continue to post on your blog unlike some people who just all of a sudden stop. Thank you and I will be checking back often!

  29. #29 Sandra Riller
    July 21, 2011

    If you’re inquisitive about having a guest blog websiteer please let me apprehend. I will be able to provide you with distinctive content for your webstie, thanks.

  30. #30 garmin nuvi 255w gps
    July 26, 2011

    Comfortably, the blog post is during truthfulness a hottest on this subject well known subject matter. I agree with ones conclusions and often will desperately look ahead to your . Saying thanks a lot will not just be sufficient, for ones wonderful ability in your producing. I will immediately grab ones own feed to stay knowledgeable from any sort of update versions. get the job done and much success with yourbusiness results!

  31. It’s hilarious that you would post this as I have been in the progress of looking for data on this discussion. Thanks for the data on the subject.

  32. #32 cover letter
    August 3, 2011

    Scientific Imperfections : The Frontal Cortex Fantastic goods from you, man. I’ve understand your stuff previous to and you’re just too magnificent. I actually like what you have acquired here, certainly like what you’re saying and the way in which you say it. You make it entertaining and you still care for to keep it wise. I cant wait to read much more Scientific Imperfections : The Frontal Cortex again from you. Thanks For Share .

  33. #33 robo defense
    August 9, 2011

    I like Your Article about Scientific Imperfections : The Frontal Cortex Perfect just what I was searching for! .

  34. #34 John Burden
    August 12, 2011

    Hi, I totally agree with you! Just obey my previous comment

  35. #35 libertyreserve
    August 19, 2011

    Thanks i love your article about Scientific Imperfections : The Frontal Cortex

  36. #36 Internet Consulting
    August 24, 2011

    Nice website you have here. You seem to a a knack for this.

  37. #37 magicofmakingup review
    August 25, 2011

    Are you interested in having guest bloggers?

  38. #38 new sports cars
    August 28, 2011

    Thanks i love your article about Scientific Imperfections : The Frontal Cortex

  39. #39 Sherise Tromley
    September 5, 2011

    I always seem to benefit from these type of blogs. Everyone people have a different prospective which allows the read to be more diverse and sometimes controversial which can open our minds.

  40. #40 Lawanda Sodek
    September 9, 2011

    this is a useful thing 4 sure Here’s some payback: Thought for the day? : I don’t suffer from insanity. I enjoy every minute of it.

  41. #41 Fae Debruin
    September 12, 2011

    This is just what I was attempting to identify. I enjoy it.

  42. #42 Discount cell-phones
    October 10, 2011

    Thanks for helping me to attain new ideas about computers. I also hold the belief that one of the best ways to maintain your laptop in prime condition is by using a hard plastic case, or maybe shell, that fits over the top of your computer. These kinds of protective gear are usually model unique since they are made to fit perfectly in the natural housing. You can buy all of them directly from the owner, or from third party places if they are available for your notebook computer, however not all laptop will have a cover on the market. All over again, thanks for your ideas.

  43. #43 housekeeping jobs
    October 14, 2011

    Hey guys i watched Camp Rock and i think it was ausome!!! i am a huge fan of the Jonas Brothers so i was realy excited to watch it and i think that Demi is an amazeing artest and has an amazeing voice so dont bad mouth her,just give her a chance!!!!

  44. #44 Florentina Pearman
    October 19, 2011

    I agree with this. The case of “Apple”, for instance, perfectly illustrates this point. There is virtually no chance the kid will ever meet another person with that name. I see no reason for Gwyneth Paltrow to have any regrets over that name.

  45. #45 Elijah Kalberer
    November 3, 2011

    Recomending this to my friends, they’ll like it as well.

  46. #46 Android Apps
    November 13, 2011

    Magnificent goods from you, man. Scientific Imperfections : The Frontal Cortex I have understand your stuff previous to and you are just extremely excellent. I actually like what you’ve acquired here, really like what you’re saying and the way in which you say it. You make it enjoyable and you still care for to keep it smart. I cant wait to read much more from you. This is really a wonderful Scientific Imperfections : The Frontal Cortex informations.

  47. #47 Android Apps
    November 13, 2011

    Magnificent goods from you, man. Scientific Imperfections : The Frontal Cortex I have understand your stuff previous to and you are just too magnificent. I actually like what you’ve acquired here, certainly like what you are stating and the way in which you say it. You make it entertaining and you still take care of to keep it wise. I can not wait to read far more from you. This is actually a tremendous Scientific Imperfections : The Frontal Cortex informations.

  48. #48 sprinkle mediocre apk
    November 13, 2011

    I got what you think, thanks for swing up. Woh I am glad to grow this website finished google. Thanks For Share Scientific Imperfections : The Frontal Cortex.

  49. #49 ukraine escort
    November 15, 2011

    Wonderful story! Ive been searching google for many hours searching for applicable data on th is, they undoubtedly require to position your website on the initially web page!

  50. I agree with this. The case of “Apple”, for instance, perfectly illustrates this point. There is virtually no chance the kid will ever meet another person with that name. I see no reason for Gwyneth Paltrow to have any regrets over that name

  51. #51 Lesley Paris
    November 21, 2011

    Great post, very informative, hopefully it will being some of those lurkers out into the open.

  52. #52 Lakenya Gwartney
    November 22, 2011

    Thanks for taking the time to talk about this, I feel strongly about it and enjoy mastering extra on this topic. If possible, as you acquire expertise, would you mind updating your weblog with far more data? It is very useful for me.buy guild wars 2 gold

  53. #53 LA Weight Loss
    November 30, 2011

    Wonderful goods from you, man. Scientific Imperfections : The Frontal Cortex I have understand your stuff previous to and you’re just too great. I really like what you have acquired here, certainly like what you’re stating and the way in which you say it. You make it entertaining and you still care for to keep it wise. I cant wait to read far more from you. This is really a terrific Scientific Imperfections : The Frontal Cortex informations.

  54. #54 Kip Accola
    December 1, 2011

    Please let me know if you’re looking for a writer for your site. You have some really good articles and I think I would be a good asset. If you ever want to take some of the load off, I’d love to write some articles for your blog in exchange for a link back to mine. Please send me an email if interested. Thanks!

  55. #55 Julio Sanjuan
    December 12, 2011

    I precisely had to thank you so much yet again. I do not know the things that I would have handled in the absence of the recommendations contributed by you about this field. It has been an absolute fearsome circumstance in my opinion, nevertheless witnessing a new well-written way you resolved it forced me to jump with contentment. I’m grateful for your work and then have high hopes you are aware of a powerful job you were doing teaching the rest with the aid of a blog. I am certain you’ve never encountered all of us.

The site is currently under maintenance and will be back shortly. New comments have been disabled during this time, please check back soon.