Is PLoS Coming of Age?

Heavyweight science journalist Sir Delcan Butler has published an update, of sorts, on the status of the Public Library of Science (PLoS), published today in the journal Nature.* In it, he presents a study carried out by Nature on the financial status of PLoS, and describes the ups and downs of Open Access publishing.

PLoS uses the business model that has emerged among Open Access journals, whereby it charges authors for their works to be published, but does not charge individuals, libraries, or other institutions for access (thus the term “Open Access”). This has been a welcome way of doing things for cash-strapped libraries and individuals. Indeed, this model represents a kind of democratization of science. Most people believe that PLoS is on the cutting edge of this new approach.

However, as pointed out in Butler’s article, there may be some sour grapes elsewhere in the publishing world. Butler cites a variety of both named and unnamed individuals in other publishing houses who are not entirely positive about PLoS, yet seem to have nothing substantive to say. For instance, an unidentified source whom Butler implies to be linked with or at least very knowledgeable of BioMed Central, another publishing concern that sees itself as OpenSource, criticizes PLoS for not having made any profit to date. One has to wonder, however, if profit is the primary motive for an ideologically motivated concern such as PLoS.

One of the most important innovations at PLoS is a journal venue called PLoS ONE. PLoS ONE published a wide range of material with an innovative peer review system, in which reviewers look at the quality of the work and writing but not the perceived importance or media-attractiveness of it. Some may say that this could reduce the quality of material published in PLoS one. However, this is a serious misunderstanding of how scientific publishing works.

The truth is that scientific publishing, including traditional publication and peer review is often criticized and in many areas is considered broken. It is probably the case that peer review is an important winnowing process for journals with limited print ‘real estate,’ which does not apply to the same degree to an On Line journal like PLoS. In these cases it could be argued that only the ‘best’ papers can be published. The problem is that one persons ‘best’ is not another’s. Under such conditions, legitimate and critically important diversity will always give way to narrowly defined standards that limit creativity and, in the end, the progress of science.

How badly broken is scientific publishing and peer review? It depends on who you ask and what field you consider.

I cannot tell you how many times I’ve heard the question “Is Nature peer reviewed?” referring to the journal Nature (not the outside Mother Nature type nature). In the business, it is widely believed that the process of getting a piece published in Nature Letters (the primary form of publication in that journal) is a matter of bing in the right time at the right place in the Old Boy Network. I have no idea if this is true, but that is the reputation the famous old British journal has, at least on this side of the Atlantic. For our part, in America, the PNAS has a similar reputation, equally damaging to the process of peer review (which is, after all, a perceptual and regulatory, not objective and measured, process) regardless of the truth of it.

In this context, an explicitly more open form of a journal, like PLoS ONE, is very welcome and will likely be transformative. This is especially true given that articles on PLoS can be commented on, and journal groups can formed to discuss them. Dynamic interest-based open peer review of Open Access published articles … It’s the future.

PLoS may be said to be growing slowly – perhaps it is not making any money yet. But most journals fail, and those that do not are rarely cash cows. Indeed, it is the hunger for profits in the journal producing industry that has turned most academic vehemently against commercial publishers to the point of foaming at the mouth vitriol. In a world where academics see the journal producers as adversaries and pirates, an ideological driven, willing to experiment entity like PLoS is very welcome!

Having said all that, I still had questions about PLoS, how it is working, what the future may bring, and what current difficulties face the new enterprise. So I dug around a bit and came up with some interesting information about both PLoS and the Nature piece.

Impact factors seem to be everything. How does PLoS (and/or it’s various projects) stand in relation to other journals in relation to impact factors? From a piece describing 2007 Impact factors for PLoS Journals:

…impact factors should be interpreted with caution and only as one of a number of measures which provide insight into a journal’s, or rather its articles’, impact. Nevertheless, the 2007 figures for PLoS Biology and PLoS Medicine are consistent with the many other indicators (e.g. submission volume, web statistics, reader and community feedback) that these journals are firmly established as top-flight open-access general interest journals in the life and health sciences respectively.

The increases in the impact factors for the discipline-based, community-run PLoS journals also tally with indicators that these journals are going from strength to strength. For example, submissions to PLoS Computational Biology, PLoS Genetics and PLoS Pathogens have almost doubled over the past year – each journal now routinely receives 80-120 submissions per month of which around 20-25 are published. The hard work and commitment of the Editors-in-Chief and the Editorial Boards (here, here and here) are setting the highest possible standards for community-run open-access journals. .,.. read the rest here.

I wondered what subject areas are most heavily published in in PLoS ONE and if there an area where PLoS is seeking more growth. A source familiar with PLoS recently told me “As the oldest PLoS journals are biomedical, it is not surprising that much of ONE output is also biomedical. However, over time, the journal has amassed quite a few articles in (neuro) psychology, anthropology, palaeontology, ecology, conservation, evolutionary biology and is expanding into more and more areas every week.”

For more information about the Journal Club I mention above, have a look at these links:

Journal Club Archives
Journal Clubs – think of the future!
Journal Clubs are back!

Some will say that the Nature piece by Butler is negative or even cynical regarding PLoS. Maybe. But on close examination, perhaps Butler is just doing his job as a journalist, asking questions, probing, seeking clarity. In the mean time, PLoS clearly stands up well against these questions.

One thing that does bother me a bit about the Nature piece is this: To the extent that it can be seen as negative, it must be seen as negative about a competitor. Normally one would expect a discloser statement indicating that where PLoS loses financially, Nature gains. The sources cited by Butler who have negative things to say about PLoS are also either competitors or simply anti-OpenSource.

I am absolutely confident that Butler and Nature will address (or even redtress) this apparent misstep.

For more information, here are three other posts that I know of on this issue:

From the Tree of Life
From Drug Monkey
From Frontal Blogotomy
From Gene Expression
From Living the Scientific Life

_______

* I would give you a link to the article, but since Nature is not Open Access I assume you will either already have this at hand (because you are special) or can’t have access to it (because you are not special). Here is the citation:

Butler, Declan. 2008. PLoS stays afloat with bulk publishing. Nature V.454, p. 11. DOI: 10.1038/454011a

Comments

  1. #1 bill
    July 2, 2008

    PLoS has a unique business model

    Around 30-40% of OA journals follow the same model; PLoS is neither unique in this, nor were they the first to test it (BMC started up in 1998 and was OA by 1999, PLoS launched in 2001 or 2002 iirc).

    The truth is that traditional peer review is widely considered broken.

    Um, this strikes me as just plain wrong. It’s widely acknowledged as an indispensable cornerstone of science. There is plenty of debate over how to improve it but I’ve yet to hear it described as “broken”. Also, I don’t know whether you intended the connotation or not, but your paragraph there reads as though PLoS ONE does not carry out “traditional” peer review — which it does. All PLoS journals use standard double-blind peer review; PLoS ONE adds a couple of layers, like allowing public comments and offering reviewers the choice of discarding anonymity. If anything, the focus on “mediaworthiness” at big name journals like Nature is the “non-traditional” version of peer review.

  2. #2 Greg Laden
    July 2, 2008

    Bill: It is widely acknowledged as a broken indefensible corner stone of science. Yes, a corner stone. Yes, broken. Not fully either, not fully neither.

    I do not say that PLoS does not do traditoinal peer review, but the distinction I state for PLoS ONE is right out of their manual and accurate.

    I’d love to agree with you that Nature is out of wack for media worthiness. In truth, though, this varies a great deal across the different journals. It is not like everyone ignores impact of this kind but a few. There is simply a wide range of variation.

    Your comments about the non-uniqueness of PLoS’s business model are correct. I’ve misused the word “unique” … I mean to say progressive, new, cool, etc. like some of those other OA journals.

  3. #3 Ian
    July 2, 2008

    “Widely acknowledged” is the classic weasel phrase for “I believe it and I want to pretend I have lots of support”. You may not intend it that way, but that’s how it comes across.

    Like bill, I strongly disagree with your claim; it is not widely acknowledged that peer review is broken, save perhaps among the chattering classes online. Bloggers and their commenters — even those who know anything about science, let alone those who are practicing scientists — are a tiny minority; hardly any scientists I know have a significant online presence, yet I believe the vast majority of them would deny that peer review is “broken”.

    It is possible (I am not convinced it’s true, but I accept that it’s possible) that a significant number of science bloggers and their commenters believe peer review is broken. Equally, it was “widely acknowledged” online that Ron Paul was a realistic candidate for president. The attacks on peer review online have exactly as much credibility to me as did the Ron Paul For President fanatics.

    That’s not to say I do, or do not, believe that peer review is a perfect, great, good, or acceptable system. It’s just that I’m getting tired of the claques believing their own blog releases.

  4. #4 Blake Stacey
    July 3, 2008

    Last I heard, peer review was “widely acknowledged” to be a fallible system for judging whether a paper was obviously wrong or obviously boring (to paraphrase Cosma Shalizi). It’s the first line of defense, not the last.

  5. #5 Greg Laden
    July 3, 2008

    Well, Ian, the reason for your disagreement with me on this issue could be that I’m totally wrong, or it could be something else. Your problem may just be that “widely” does not include you. Your implication seems to be that if a blogger is saying it, this must be a thing felt among bloggers. Not only is that not applicable, but it ignores the growing undercurrent of literature on and discussion on the topic.

    I’m a scientist and most people who would give two cents in relation to peer review that I know are scientists. I hardly know any bloggers and I hardly read any blogs. So my assertion does not arise from some cabal of bloggers or non-scientists.

    This may well be, and you might do well to acknowledge this, that there is variation among scientists in this regard. For instance, those in the medical research profession may have very different experiences than those in other areas. I work with very few medical researchers.

    Indeed, just two nights ago I spent a long period of time in conversation with a couple of colleagues in my area and a large part of this was regarding peer review and it’s broken-ness.

    Also there may be some misunderstanding over the use of my term “broken.” That term does not mean and I do not mean to imply with it that the peer review system is utterly and in every way not working. I have two cars. On has hardly anything wrong with it and the other has many things wrong with it. I would call the former not broken, almost everyone would call the latter broken. They get us around. Both ‘work’ and one of the two is broken.

    Perhaps I have overstated the case somewhat, but perhaps you are being oversensitive.

  6. #6 Ian
    July 3, 2008

    Perhaps I am being oversensitive, Greg, and perhaps there is huge variation among different subsets of science. I’m not specifically a medical researcher, but I and most of my colleagues are biologists — traditional, wet-bench biologists, most with at least a stated goal of understanding medical problems; not a small corner of science — and your suggestion doesn’t ring true to me.

    I was particularly baffled by the “Is Nature peer reviewed?” comment — was that a serious question, or snark? If serious, the questioner seems to me to be completely clueless. Hasn’t she even thought about submitting to Nature, to the point of looking at the requirements? (If not, where’s the ambition?) Hasn’t he done peer reviews for Nature and its ilk? I’m far from being a giant in the field, and I’m asked to peer review for Nature semi-regularly.

    Similarly, the “Only members of the club get to publish in Nature” — a more common claim, I’ll grant — comes across as sour grapes, not as a scathing indictment of peer review. It’s clearly false; certainly Nature has a reasonable number of repeats, but even cursory checks will show that there’s a regular parade of newcomers who publish there. (My own first Nature paper came without any Old Boys Club help.)

    Comments like those are widespread in the anti-peer review crowd, and they’re really counterproductive. Like many of the arguments against peer review I’ve seen, they come across as naive at best.

    Again, I’m not supporting Nature in particular — but as bill says, Nature’s more egregious flaws, its hunt for the limelight is because of the way Nature avoids traditional peer review, not because of peer review. The same applies to PNAS, which often blatantly avoids peer review (in spite of the revamping of the method, which certainly has improved the worst of the abuses). Science, which has a similar impact factor to Nature, rarely seems to get called out in these complaints, yet (I believe) much more strictly adheres to traditional peer review.

  7. #7 Cesar Sanchez
    July 3, 2008

    Greg, I found that this post was aggregated by ResearchBlogging. However, the Nature piece is NOT peer-reviewed research (not even research), but just a news story. Don’t you agree with me?

  8. #8 Greg Laden
    July 3, 2008

    the Nature piece is NOT peer-reviewed research
    And as such we challenge established beliefs, yes? Which papers on BPR from Nature ARE peer reviewed? Do you know? Have you asked this question before?

    Ian: I could be wrong, but my understanding is that the vast majority of manuscripts submitted to nature are rejected (that is to be expected, of course, they have very limited space) and that the vast majority of these are rejected by the editorial staff (which is not uncommon). One can very reasonably ask what the peer review process has to do with these rejections. My comments above (“is nature peer reviewed”) is not about a certain person or a sour grapes situation. (Sour grapes happen, I certainly recognize that, know it when I see it, and ignore it.) I’m not talking about one instance. I’m talking about a general sense and repeated instances over decades not just about Nature.

    Look, the peer review system works. I am not claiming that it does not work. But so does my 22 year old car. I still want a new car.

    You have a very good point in relation to Nature and PNAS. I should be speaking more broadly: The system by which we publish our scientific work has some problems, some annoying, some big, worth fixing. I include the circumstance in which something defined as “peer review” happens as well as those in which it does not. I see no pragmatic reason to define the problem around these distinctions. If we did that, then BPR (see above) is going to have to de-aggregate a lot of blog posts!!!

  9. #9 RBH
    July 3, 2008

    For instance, a source from Biomed Central, another publishing concern that sees itself as OpenSource criticizes PLoS for not having made any profit to date. One has to wonder, however, if profit is the primary motive for an ideologically motivated concern such as PLoS.

    I ran a non-profit for seven years, a private corporation that employed 250 cognitively disabled adults. One of my most difficult tasks was to get staff to realize that making a profit on operations (defined as revenue in excess of current expenditures) was vital to us because it enabled us to build a reserve for lean times, make capital improvements, pay for staff development, buy new equipment, and generally upgrade the services we provided. Many people working in my non-profit (at least at that time, in the 1980s) thought that if we generated excess revenues (defined as above) we were immorally stealing from those being served.