Over at Mixing Memory, there's an excellent and fierce critique of a recent fMRI paper on linguistic relativity. Although the post is shot through with overly broad insults - he or she complains about "how much cognitive neuroscience sucks" - it still manages to carefully dissect the data. In short, the author concludes that the earlier behavioral work was more interesting and definitive than the more recent study that looked inside the black box of the brain. Anyways, it's worth a read. (Longtime readers will know that I've got a few quibbles with brain imaging boom myself.) What I'd really love to see is the authors of this fMRI study respond to the criticisms.
As a science journalist, though, there was something frustrating about the post: it was written anonymously. It's really hard to get scientists to be critical of each other on the record, or in language that isn't heavily freighted with acronyms, hypothetical sub-clauses and hedged bets. I've spent enough time around scientists to know that they are masterful critics and talkers of shit. And yet, when you ask them as a journalist about some study that they would happily eviscerate in lab meeting, they suddenly start talking about how the research is "interesting but inconclusive...more study is needed...it's one possible hypothesis, but there are alternative explanations...etc, etc." It's like they're unwilling to tarnish the tribe in public.
Needless to say, I think this a terrible thing, and not only because it makes my life more difficult. A few years ago, I wrote an article for Seed about how we need more science critics. These people would criticize (and praise) the latest science papers in the same way that a music critic evaluates the philharmonic or an art critic writes about a new museum exhibition. In other words, they would be experts working within their field of expertise, constantly reminding us that the latest science papers are only a first draft of reality, an imperfect attempt to decipher a complex problem. As Karl Popper, an eminent defender of science wrote, "It is imperative that we give up the idea of ultimate sources of knowledge, and admit that all knowledge is human; that it is mixed with our errors, our prejudices, our dreams, and our hopes; that all we can do is to grope for truth even though it is beyond our reach. There is no authority beyond the reach of criticism."
One of the best things about the science blogosphere is that it allows non-scientists to see the sort of criticism (some catty, some constructive) that shapes the scientific process in real life, behind the scenes of peer-review. It gives people a glimpse of scientists as people, complete with egos and ambitions. But back to the specter of anonymity. I find it a little depressing that the science bloggers who regularly launch intelligent and informed criticism of their scientific field do so without revealing who they are. (I'm thinking here of Mixing Memory and the Neurocritic.) I find the anonymity troubling for a few reasons. First of all, it's easy to be nasty when you don't give your name. I always put more stock in criticism when it's signed. Secondly, the fact that these scientists don't feel comfortable talking smack in public suggests that the scientific community needs a sort of glasnost. The peer-review process is necessary but it is not sufficient. (It's also often anonymous, which is a big problem.) Simply put, I believe that science would be better off if scientists were more willing to be critical of each other in public, if they were willing to say what they really think on the record. It's easy to complain about bad science journalism. It takes more guts to complain about bad science.
- Log in to post comments
First, let me say the overly broad insults are just meant to be humorous (though I do think most cog neuro sucks).
Second, I go back and forth about my anonymity these days. On the one hand, one of the reasons I blog is to correct misconceptions and misreporting about cognitive science and psychology, and doing so inevitably means stepping on the toes of big names in the field. Without a secure (i.e., tenured) position, that's not always a smart thing to do.
On the other hand, I don't like to criticize people anonymously. So, I have a policy of identifying myself to anyone whose work I criticize, if they contact me. And that's happened a few times. At this point, in fact, pretty much everyone in my sub-field knows my super-secret identity. So I try in the ways that I can, without having to worry too much about potential career issues, to be open about who I am.
Agreed 100%. I think there is very little sensible criticism or discourse in major areas of science because the ramifications are so far-reaching. Take any of the following scientific topics - it's easy to imagine what's at stake politically and culturally:
-Global warming (right wing vs left wing, recycling vs big business)
-Stem cell research (religion, right wing vs left wing)
-Evolution (religion, right wing vs left wing)
-The effectiveness of anti-depressants (the success of the drug industry, religion, scientology ;)
-The biological or non-biological causality of homosexuality (gay marriage, homophobia in the work place, the military, etc)
These aren't just scientific issues, most were major platforms of the last US presidential election.
The problem seems two-fold. A biologist or "bright" (barf) is hesitant to question any tenant of evolution, however minor because they are worried it lends credence to the creationists' message that evolution is "just a theory" (understandably). And when it comes to the general public, evolution is either "right or wrong." That being the case, scientific findings are referenced and interpreted like a bible - an absolute answer or solution to a given problem. This forces scientists to take a simplified stand for fear of polluting the cultural/political/moral statement attached to it.
I also assume scientists are afraid to criticize as it probably invites critique of their own work. Simply, no one wants their results or extrapolations questioned.
Damn, I meant to leave that last comment anonymously.
Basically, most scientists are loathe to say anything definite or decisive to a journalist, since we've been burned, or seen colleagues be burned, when a journalist quote-mined or otherwise abused the information to play up some damaging angle. We tend to learn the hard way that journalists will sometimes deceive you, will break any promise or assurance and will distort what you say as needed to fit the mould of their story, even if it means damage to you or to your experimental subjects or to the research.
In internal discussions people do still focus on the subject; people don't often try to win cheap points by intentionally misrepresenting the intent of others arguments (and when someone does, it's typically seen for what it is; a failure to come up with substantial criticism). If, as you state it, someone would eviscerate a different groups' study, how many journalists, honestly, would refrain from portraying that as an attack - perhaps even a feud - between competing groups? Which even in the best case would put a strain on cooperation for a while and might in the worst case actually precipitate exactly the kind of destructive conflict that the journalist made up in the first place.
You get the hemming and the hawing, the endless qualifiers and the zen-like non-commitment to any actual opinion because you journalists, as a group, have come to richly deserve it. Me, I advise every new lab member to memorize the following line "I'm afraid I can not comment on anything like that. But here, let me give you the number to our media relations people ..."
In a perfect world, yes, non-anonymous critiques of current research is a great idea. In this world, however, we need jobs and for jobs we need friends and networking. So public criticism just doesn't work. As just a first-year PhD student, the idea of blogging under my name and criticising another group's work quite honestly scares me. I'm not giving up on the idea! But that's just how it is.
I'd argue that you don't see signed commentary and criticism of research like this because there's absolutely no incentive for doing so (I've argued this on my blog) Two main reasons, even though most journals have mechanisms for leaving comments on papers:
1) There's no career advancement credit given for spending the large amount of time needed to write up a thorough critique of someone else's paper. Why spend your time doing that when you could be doing your own experiments, or writing up your own results?
2) Critcism has ramifications. What if the person you publicly humiliate is sitting on the study section for your next grant application? What if they get assigned to peer review your next paper? What if they're chairing the hiring committee where you're hoping to get a job? Chairing a meeting where you're hoping to speak? How likely are they to treat you fairly after you've savaged their work?
Due to point 2, anonymity is going to be important if you want honest criticism of results. Then again, it reinforces point 1, if it's anonymous, you get even less credit.
The peer review system (in my field, at least) is nearly always anonymous. This two-way mirror holds for both papers and grants. From what I've heard, reviewers have been able to get away with nasty, insulting, ill-conceived attacks on good solid research they don't happen to like, mostly because it conflicts with their own results or contradicts their favorite theory. If a reviewer's identity were out in the open, they'd have to be more accountable for their remarks.
There are rare exceptions:
(1) A reviewer signs his/her review and the editor allows it.
(2) In the PLoS journals, somewhat, although the policy varies from PLoS Biology to PLoS ONE to PLoS Medicine. [links to these removed to avoid comment limbo]
So for as long as journals and funding agencies employ anonymous reviewers, I will remain anonymous.
Jonah made the point that when scientists are unwilling to "tarnish the tribe in public" science journalists are essentially left to do the job. This job requires having the proverbial seeds, but also having the capability or responsible urgency to scrutinize all aspects of a science research publication.
I am concerned about science writers in the mainstream media. From my experience reading this genre of science writing (in local news papers and major news websites), I worry that the writers in the mainstream do not collectively demonstrate a critical eye for science, or a consideration for non-science lay people. Case in point: we have thousands of vaccination-autism cases pending in federal court. Did the science writers who sensationalized these vaccination findings thoroughly critique the method sections of these scholarly research publications? Or at least offer up the subtle yet powerful dichotomy between correlation and causation to the lay public? Would these concessions have impacted the evolution of the autism-vaccination controversy? This case is complicated; and who knows what the outcome may be. The main point is autism-vaccination reporting in the mainstream seemed to be less of a critique considering the validity of the strong claims that vaccinations cause autism, but moreover tremendous hand-waving that a causative link between the two had been found- as far as I am concerned.
The influence of mainstream science writers is becoming increasingly more potent, and so is my concern. A recent editorial in Nature reported on the Pew Research Center's "The State of the News Media 2008" appraisal. Two major supporting points: the internet is overtaking television and the media industry is moving online, and two-thirds of all who consult the internet for news are looking for science and health news.
Is there a problem translating academic scholarship into the public sector? Are the present mainstream science writers collectively a good conduit for the transmission of science to the public? Is there a need for scientists to write for the science section of the local paper or major news websites? After reading this post and pondering, I wish either scientists would grow some seeds and "tarnish their tribe" or science writers would start critically analyzing findings and unearthing the limitations and considerations of science research findings for the public to see- but both would be preferable.
It seems to me the issue of credit is moot. Anon crits are a positive to me because: 1) In most any field, writing a good crit is a useful exercise for the author as well as serving the group, even though no precious brownie points are to be had. 2) As long as they are done with intellectual honesty and not some sort of distorted self serving political reason, science should benefit from more pointed analysis which anonymous posting encourages I think. 3) Solid analysis can stand on its own without hiding behind the reputation of any particular author, so there is no scientific reason to discount such contributions.
What's not to love about anonymous (competent) crits online? Is there any evidence they are distorted or dishonest somehow?
.. there r some psychic patients r controlling the journals.
so v shud think abt PSYCHO JOURNALOMA like mental diseases
www.iipponline.org
wt is this disease?..OMA style diseases are introducing as cancer.psycho journaloma is a symbolic term about social sins or socio dimerits..?
waiting+curiosity
nasi
why todayâs media is only about sensationalism. Its main aim is to create awareness but today the media has considered itself to be demi god and is attacking peopleâs personal lives . thus the journalism is converted to a fattal disease. so this type of Journalism is cancer to our people. this is the meanby dr zuhoori?
hida
why todayâs media is only about sensationalism. Its main aim is to create awareness but today the media has considered itself to be demi god and is attacking peopleâs personal lives . thus our journals are converting as a cancer.this is the mean by dr zuhoori?
hida
The kind which will contribute to the betterment of society. Todays 24hrs news channel not only give highly exaggerated and sensationalised news but also add their opinions to it, distorting the whole picture. Less facts, high doses of opinions is the trend today. A lot off good happens around. Work of NGOâs, inititative of may responsible citizens, socially responsible entrepreneurship projects etc. They are not covered. Today news channel are giving everyone a depression pill.sometimes dr zuhoori means to this 'disturbing jornal cells'..?!
vijay
Dr Zuhoori's presentation about PSYCHO JOURNALOMA is the psycho pathological study of yellow news culture
www.journaloma.blogspot.com is the net media for this discussions
vijay
psycho journaloma is the study of scandaling journalism
Journalism scandals are high-profile incidents or acts, whether intentional or accidental, that run contrary to the generally accepted ethics and standards of journalism, or otherwise violate the 'ideal' mission of journalism: to report news events and issues accurately and fairly
@marissa... My thoughts on the matter of science-reporting in the media tend toward the following.
Almost every larger city and all states have large public universities with the normal range of colleges, departments and divisions that are chock-full of "experts" on most of the fields of science that are covered in daily press releases. It seems to me that, although it is too much to expect that a "science journalist" - or that person's editor - would be sufficiently familiar will the myriad fields of study - much less, the nuanced controls that are peculiar to each field, it would not be too much to expect that a professional science journalist would have (or should develop) a stable of expert informal/ad hoc consultants in those universities (at least one for each of the major areas of science that they cover) against which to "bounce" their writing before going to press. Those are, after all, public employees in public institutions who should give a small and reasonable amount of their time to helping inform that public accurately.
That assumes, of course, that the journalist's motive is to inform the public (v. producing sensational headlines and reader-grabbing conclusions, regardless). Failing that test of motive, they deserve the criticism that they get.