I’ve railed against the rise of supplemental data and methods before, but, having just reviewed a paper where I spent more time reading the supplemental sections versus the actual fucking paper, what Scicurious wrote struck a chord with me:
Sci wishes she could make her own flowchart of supplemental data. It might look like this:
Does the journal have ridiculously short page/word limits compared to the ridiculously huge amount of data they require for publication? Yes.
Are your methods way too numerous and complicated due to the ridiculously huge amount of data required for publication? Yes.
Did a reviewer demand extra experiments which in no way help your paper’s cause and which no one but that particular reviewer cares about? Yes.
Do you feel the need to publish ALL OF YOUR FIGURES. ALL OF THEM? Yes.
Then you need SUPPLEMENTAL DATA!!!
Now, are any of the above items NECESSARY to the publication of decent science?
Scicurious then explains what doing away with supplemental science will mean:
Of course, the final result of HAVING supplemental data in high end journals is that you end up with these tiny, tight little stories with ridiculously small word limits in Science, and with the “oh, yeah, here’s what we REALLY did” in the fifty kabillion supplemental pages. It also means that you can get away with not particularly well crafted stories, as your many little side experiments get relegated to the supplemental data. So I’m glad that J Neuroscience is going to be getting rid of Supplemental data, but I also hope that they, and their REVIEWERS, keep this in mind. It’s going to mean, maybe demanding tighter stories, and maybe ALSO not demanding stories that are as sweeping and huge as the big journals like to go for. And it’s going to mean thinking carefully before reviewers suggest that you do fifty billion extra experiments to tie in to their little corner of the field. It may also mean, that if Journals really want to publish well-done, carefully crafted, high end experiments, with difficult methods, they may need to expand their sometimes rather paltry word limits.
We forget that, ultimately, whether journals have supplemental sections (and if we have to review and write them) is not up to the journals but us. Consider the publishers’ business model:
What has always steamed me about the for-profit publishers is that they charge so much for something they have very little part in manufacturing.
They don’t pay the salaries of the those who provide the product–the research and accompanying article. They don’t pay the reviewers. They often don’t pay most of the editors. To top it all off, they take publicly funded research and makes the results inaccessible to many citizens who, often, have supported that research.
Oh yeah, researchers in the developing world often can’t afford the articles to the point that there was a non-profit, Satellife, that used to transcribe select articles and email them to universities and hospitals (email was used since, in the 1990s, bandwidth and reliability were serious issues).
But other than that, there are no problems at all.
So to get all Marxist and shit*, I see two options here:
1) Simply refuse to review for and, if your career can take it, publish in journals that have supplemental sections.
2) When asked to review supplemental sections, which is a pain in the ass (I agreed to review one paper, not two or three), ‘reorganize’ the paper. Make your ‘grade’ of the paper (e.g., accept with major revisions) contingent on moving what belongs in the paper into the paper, and then cutting the rest**. The worst case scenario is that editors don’t send you manuscripts to review. They’re pretty desperate for reviewers though, so they might just have to grin and bear it.
There’s no reason to subject ourselves to this, and we can do something about it.
*I’m not a Marxist, not by a long shot, but they’re never around to talk about ownership of the mean of production when you need them.
**There’s nothing wrong with appendices–a place to put data that isn’t to be reviewed (i.e., strains used, long lists of genes found in one genome and not the other, etc.), although I would argue the appendices should still be part of the paper, at least online, if they’re worth communicating.