Mike the Mad Biologist

Over the last couple of weeks, I’ve been catching up on my science reading, and that’s reminded me just how much I hate it when journal articles refer to supplemental materials. I’m not bothered when tables that used to go in what we old-timers once called appendices wind up in supplemental materials: very few people want to read the descriptions of 2,000 bacterial strains, for instance.

But that’s not what most supplemental materials are, or how they arose (we’ll get to that in a bit). Not only is it a hassle to download an article, only to realize that there’s a supplement you need and isn’t readily available (e.g., you’re not at the office, so the institutional subscription doesn’t work), but, more importantly, it leads to poor writing.

As far as I can tell, the glamour mags, such as Science and Nature, first implemented supplemental methods. Given their severe page limitations, they began moving methods sections to online-only supplements (never mind that for a significant fraction of papers, the only thing I’m interested in is the methods).

Then they, and other journals, rather than either editing articles to shorter lengths or changing their publication format to meet scientific needs, began adding results and data to the supplements. This is bad for two reasons. First, if the data are important, then they belong in the paper. If not, then cut that section. You’re not supposed to communicate every result, only the ones that matter. Second, if these data are only for the ‘specialists’, then you probably can’t answer their specific questions in a supplemental section–they’ll want the actual data (to a few journals’ credit, they actual post data in supplements). Either the supplement belongs in the article, or it’s superfluous. Articles are not ftp sites and aren’t supposed to be data barfs.

So what does online only publishing have to do with anything? One of the reasons I like the PLoS journals is that they don’t really have page limits because there are no printing costs. Some papers are very short; some are over twenty pages long. There’s also plenty of space for figures. When the editing process works (and it doesn’t always), the article is as long as it needs to be.

And that’s how it should be: the communication needs of scientists should dictate length, not journals’ printing policies.

Comments

#1 DrugMonkey
May 18, 2010

Word, MtM, word. That ‘supplemental materials’ crap is an abomination before the body scientifique.
#2 Eric Lund
May 18, 2010

Just say No to supplemental material. Nein. Non. Nyet. Ie. Bu yao. Either the data and methods are sufficiently important to be in the body of the paper, or they are not important enough to include at all.

In my field, supplemental material is usually in the form of animations or video footage. OK, fine, the printed page can’t deal with that sort of thing. Unfortunately, the journals I work with do not insist on video codecs that everyone can see with default software installations, as I discovered a couple of years ago while attempting to referee a paper that included supplemental videos critical to the interpretation of the results. My position was that the authors were making an unreasonable request of the reader–some people work in places where they are not allowed to install the necessary software, and the rest of us would prefer not to go to the trouble if it weren’t necessary. The journal editor overruled me, so I had to go out and find something that could read the codec.
#3 ecologist
May 18, 2010

Yes. Absolutely. Supplemental materials make sense for things that cannot be printed (videos, for example), or things that are more easily used online (computer code, or big tables of numbers, for example, that you would rather download in usable form). But I do not like the tendency of journals to move more and more of the meat of a paper to someplace separate from the text.
#4 RonM
May 18, 2010

Another big plus for electronic journals it that they are usually open access. It is particularly annoying to those of us not affiliated with institutional subscriptions to see how much publicly funded research is still being published in journals owned by publishers such as Springer and Elsevier who refuse to give open access after a reasonable amount of time (6 – 12 months would be appropriate).
The law requires open access to all NIH-funded research (which is most biomedical research). I will quote the law below. Unfortunately it appears to be largely ignored. I think that before NIH funds grant continuations, the investigators should need to show that they are in compliance.
It looks like the publishers managed to get the last sentence put into the law (below)which probably gives them a loophole and seems to create a bit of a legal paradox. The proper way to follow the law would be for NIH funded researchers to only publish in journals that open their access after 12 months. Publishing in Springer and Elsevier’s journals under their current access policies is clearly illegal.
-
Division G, Title II, Section 218 of PL 110-161 (Consolidated Appropriations Act, 2008) states:

SEC. 218. The Director of the National Institutes of Health shall require that all investigators funded by the NIH submit or have submitted for them to the National Library of Medicine’s PubMed Central an electronic version of their final peer-reviewed manuscripts upon acceptance for publication, to be made publicly available no later than 12 months after the official date of publication: Provided, That the NIH shall implement the public access policy in a manner consistent with copyright law.
#5 Otto
May 19, 2010

“My position was that the authors were making an unreasonable request of the reader”

Back in the good old days, the ApJ used to bundle in a VHS tape every once in a while. NTSC only, if I recall properly. Oh, for the days of tipping in fold-out tables.
#6 Mary
May 19, 2010

Amen. I do a lot of reading on the road. The number of times I’ve accessed something that I want to read later, only to find out in some airport while I’m trying to kill time that I’m missing the pieces I really need….And I refuse to pay the fee to access the airport wireless for that. Of course, most flights also don’t have wifi yet either.

If all the material was in the body that would be great. Or at least structure it so the whole package comes together as an option.
#7 Joan
May 21, 2010

As a librarian concerned with both access and preservation issues, I’m afraid you may be throwing the baby out with the bath water. While supplemental material may be better dealt with in an online only format, I’m afraid that long term preservation currently is not. There are still many issues including digital standards and migration that have not been adequately resolved. Not to mention whether publishers can be trusted to maintain their perpetual access contracts. I think the scientific community would be much better off resolving the relatively small issue of how to deal with supplemental materials in both a print and electronic environment, rather than creating an even larger problem: the possibility of losing access to any or all of the material 10-20 years from now.
#8 neil
May 23, 2010

Firstly, let me agree that there is no excuse for ever relegating methods to being supplemental.

But, frustrating as result sups may be as a reader, they often play an important role in the review process. As an author, you might like to write a piece of scientific prose and argument that is as snappy and self-contained as it possibly can be. But there are almost always finnicky little control experiments that you do that are sufficiently inane or so morbidly technical that they detract from the flow of the story, and obfuscate the main results to a point where you are likely to confuse the reader more than anything else. Stuff which falls under the data barf category, more or less.

In most cases, if these controls are important enough, you’d weave them into the main text. But then the review process begins. And lo, unsurprisingly, there’s a reviewer who starts asking technical questions, or inane questions, and wants details about things you really think are not of great importantance. So, to convince the reviewer of the validity of your results, you provide the technical/inane details. And, given the nature of peer review, it is a fair rule of thumb (and a worthwhile policy for peer review to maintain) to assume that if this extra information / set of controls is relevant to the reviewer, then it may be relevant to a reader — so it should be published too.

Ok, so let’s say there are two main situations here. One is that your work is largely flawed or incomplete without the inclusion of these extra details/controls. In this case, you should really rewrite your paper to include this in the main body of the text. In this case, I would hope that the reviewers insist on it. But then in the more inane case? It seems a fair compromise to allow the sups to be a repository for addressing such minor questions.

Nonetheless, I would always prefer a “download all” button to hunting for the separate pdf.
#9 Electronic Cigarettes
March 30, 2011

\SEC. 218. The Director of the National Institutes of Health shall require that all investigators funded by the NIH submit or have submitted for them to the National Library of Medicine’s PubMed Central an electronic version of their final peer-reviewed manuscripts upon acceptance for publication, to be made publicly available no later than 12 months after the official date of publication: Provided, That the NIH shall implement the public access policy in a manner consistent with copyright law.