The secret inner workings of PLoS ONE revealed

I have obtained a document that describes the secret, inner workings of the on line publication PLoS ONE. The document also exposes future plans for the enterprise.

The link is below the fold.

The link for the PDF of the document is here. Don't tell anyone where you got it.

From the Abstract, which I have decoded for you:

PLoS ONE, a peer-reviewed Open Access academic journal published by the Public Library of Science, was founded in 2006 with the intent of reevaluating many of the aspects of the scholarly journal. As a result, PLoS ONE has taken elements of the traditional publishing model for scholarly journals and separated them into those functions that are most effectively carried out before publication (for example, peer review in order to evaluate whether the article deserves to join the scien- tific literature) and those that can most effectively be carried out after publication (for example, how impactful the article was once it joined the literature). With this basic premise in place, and using the online tools that are now available, the journal has grown to the extent that in 2009 it will become one of the largest journals in the world (by publication volume). This article overviews the development of the journal to date - how it differs from most other journals and how it engages with its core audiences. In March 2009, the journal (along with other PLoS titles) began a program to place 'article-level metrics' on each publication, and this article out- lines how this has been achieved, as well as plans for further development. In conclusion, this article looks forward to the future developments of this transformational journal.

One of the most important parts of this paper is probably the discussion of PLoS's secret weapon, which is highly classified, of article level metrics. The essence of article level metrics is devious in its simplicity.

Heretofore, articles have been measured in various ways with respect to their qualities, but one of the most important measures has been the journal of publication itself. This began decades ago when some tenuring bodies made lists of articles that were "high" vs. "medium" vs "low" (or some other metric, sometimes using a rather fine tuned numeric point system) with respect to importance in someone getting tenure or promotion. This eventually evolved into the official "impact factor" which you can read about here.

In the old days, it was more the article itself that mattered, but that 'metric' has disappeared for a number of very bad reasons. (And maybe some good ones, but I know of none.) One of those reasons is that people are so often highly specialized within their field that other departmental members who are supposed to decide on tenure cannot possibly evaluate the person's work. In these cases, having your work published in a blue ribbon journal produces the necessary "wow" factor that even your idiot colleagues can understand how important you are.

This system, however, is highly flawed because there is potentially huge variation in the actual quality of articles across all publications, and it is not at all uncommon for an article that has very little 'wow' when it is first published not because it is mediocre but because reviewers don't understand the importance of it yet. This happens far more often than one might think.

PLoS's secret weapon is to develop an article based assessment of 'impact,' as it were.

PLoS believes that the journal in which an article is published should not be the primary mechanism to determine whether that article will have any worth. Instead, PLoS feels that each article should be evaluated based solely on its own contribution to the literature

This is roughly parallel to the widespread Internet-brooded (if not born) idea that "argument from authority" is wrong.

The other developing contribution from PLoS ONE that is of interest is the post publication messing around people can do with the papers, such as various Web 2.0 thingies which are all top secret projects being discussed at the highest levels in the boardrooms of major publishing companies. Just don't get found out participating in any of these post publication activities.

The publication process itself is also different than traditional publicatoi, but secret details of this process have already leaked out and have probably been discussed in the open more than any other feature of PLoS generally or PLoS ONE in particular.

Now that you have access to this closely guarded document, I do hope you spend a little time reading it. But don't tell anyone where you got it.

Please visit this post by Bora to tell him what you think of PLoS ONE and beat up on the trolls.

Binfield, Peter (2009). PLoS ONE: Background, future development, and article-level metrics PLoS ONE