I just read an EXCELLENT opinion in the July 6th edition of Nature, Illuminating the black box. Note to biologists: submissions to Nature should contain complete descriptions of materials and reagents used.
Sounds familiar? I've complained about it before.
From the Nature article:
This journal aims to publish papers that are not only interesting and thought-provoking, but reproducible and useful. In order to do this, novel materials and reagents need to be carefully described and readily available to interested scientists.That might seem obvious. But despite the efforts of our editors and referees, papers in the biological sciences are still being submitted -- and occasionally published -- that do not adequately describe the reagents used. Unless efforts are redoubled to eliminate this practice, we could see an era of 'black box' biology, in which outside researchers cannot work out what was done in an experiment.
Yes, yes, yes. Something's got to be done. And its not just researchers. Many kits are unknowns. Try asking your biotech kit vendor "what is in solution A?" Their response: it's proprietary.
Arggghhh.
So how to solve this? Crack down. But biotech companies should play ball too. We need you and you certainly need us.
But beyond the kit problem there are too many holes in our materials and methods sections. If society goes down the tubes and future generations read our papers, what will they make of all our data, of our theories, and of our "progress", when they couldn't possibly figure out what we actually did?
- Log in to post comments
I agree with the black box and I am guilty of writing methods that aren't very useful. Nature is very guilty of this. It would also help if they formatted the supplemental section to be easily downloaded in the same PDF as the original paper. It sucks to have to download four things to get a complete story.
I could care less about formulations from vendors. Those can stay a black box, they have to make their money somehow. If they gave full protocols no one would buy their stuff.
Yeah that's a good point. If Nature wants us to provide acurate M&M they should really provide the space for authors to expand. I hate it when journals want compact M&M that do not provide for enough space to write out the complete protocols (or enough space to display all the control experiments).
Oh, my goodness!Let's be honest. Publications like Nature and Science are not really working scientific journals, they are magazines posing as them whose articles are accepted on faith based on reputation.Without proper M&M, one is not subjected to scrutiny of reproducibility.Note that the subject Nature article left things fairly vague about enforcement. Retraction of the subject paper based on breach of ethics of authors (misrepresentation since they agree to disclose and distribute) upon publication would have made them more credible.[Despite everything, I believe that people are really good at heart--Anne Frank]
This is a tricky issue. It's either you leave things without being mentioned, which is not the best scientific practice, or you write a M&M section longer than the rest of the article. Also, in many cases, the papers published in high impact journals make use or develop new techniques for their research which once explained in detail would make the M&M section even longer. I think that there should be some sort of expanded M&M sections in on-line journals, like with the additional figures.
Something similar applies to referees: Either you wait six months to get a paper thoroughly peer reviewed, which is not practical, or you loose the scrutiny process and probably let pass some flawed data.