Mark Patterson writes in Bringing Peer Review Out of the Shadows:
-----------------------Hauser and Fehr propose a system for holding late reviewers to account by penalizing them when it's their turn to be an author. A slow reviewer's paper would be "held in editorial limbo" for a length of time that reflects their own tardiness as a reviewer. The short article was intended to provoke a discussion about how to improve peer review - an opening card as Hauser and Fehr put it.
So far, 16 responses have been added from readers, and the general view seems to be that incentives would be more effective than the punishment that Hauser and Fehr propose. As for an incentive, quite a number of respondents favour a system whereby reviewers are paid for their efforts - although not to a level that would fully reimburse the time spent.
--------------------snip-------------------
As an editor, I've been privileged to be party to some incredibly thoughtful and constructive discussions between authors and reviewers. It's not always that way of course, but when it works, it's fantastic. I've often felt that it's a pity that these exchanges haven't been shared more broadly, and as pointed out by others in the discussion, there are many who feel that greater transparency in peer review is the way to go - pre- and post-publication.
----------------------snip------------------
The formula for more transparent peer review might not be perfect yet, but there is great potential and further experimentation is a must. Ultimately, improving the peer review process will take the same kind of thoughtful and constructive discussions that help researchers identify the extra step that will maximize the significance of their results. We invite you to join in that discussion.
Please go there and add your thoughts!
Education & Careers
Comments
Reviewing should be double-blind. Reviewers should not be privy to whom they are reviewing.
Posted by: Jeb, FCD | October 27, 2007 10:14 PM
Thanks for the link -- I already commented there but want to comment here as well.
Mark says that paying reviewers wouldn't be much use because it would "end up shuffling money around within the community of researchers."
Aren't publication costs usually paid through the institution via grants and whatnot, however? If so, then paying reviewers directly would indeed shuffle money around, but from grants into academics' pockets. This raises an interesting ethical question about the use of government funds to pay reviewers, but I'm guessing most academics wouldn't really consider this a problem.
Anyway, I'm very much in favor of the idea. I think it would raise the quality and timeliness of reviews, since academics would want to be seen as high-quality reviewers by the editor to get called upon more often. And anything that speeds up the review process would be most welcome. I'm sitting waiting for a set of reviews right now...it's rough as a junior researcher, with so much seemingly on the line, knowing that comments are so often harsh, and nothing to do but wait.
Posted by: Josh | October 28, 2007 1:30 AM
Josh- Reviewers comments (when they are good quality and constructive) are the sort of master class in science that you really can't get anywhere else. Don't take the harsh comments too personally- As long as the reviewers are playing the ball (ideas) and not the man (personal abuse). And if they are playing the man it may mean that you've made a point that the reviewers don't like but can't refute rationally (i.e. you've quite possibly won the argument).
The BMJ has open review where everybody knows who everybody else is after the 1st reviews are completed. Conflicts of interest are declared (supposedly) and everybody has to be constructively polite and give a good quality review of the work because it's going to appear with their names on it. I'm not sure if Bora has mentioned it but I think that publishing the reviews online with the paper might well be the review system of the future- the downside, I think, is that less people are going to be willing to review given the time and effort it will take to change your comments into the sort of writing suitable for publication (It will up the workload of a review substantially).
Jeb (what is FCD?)-I don't think that Double-blind works. There are usually enough clues in any paper by a reasonably active reasearch group for you to work out who they are (or at least the lab chief's name). And it gives the reviewer an ability to hide their identity which some reviewers clearly abuse.
Posted by: Nat | October 28, 2007 5:31 PM
About FCD
I know there are usually enough clues in the paper to let the reviewer know who wrote it. But Reviewees don't have the same privilege. Besides, if ti was double-blinded, no one would be sure. Either make it double-blind, or do away with anonymous reviewing.
Posted by: Jeb, FCD | October 29, 2007 2:02 PM