Peter Lawrence has an opinion piece in Current Biology on the problems with evaluating scientists, amongst other things. He hits upon a few important points, including journal impact factors, the cost of high risk research, hyping up publications, and networking with the right people to improve your publications. One passage was quite salient given a private discussion I’ve been having with some folks:

Fourth, there is the way that science is done and papers are authored. These measures are pushing people into having larger groups. It is a simple matter of arithmetic. Since the group leader authors all the papers, the more people, the more papers. If a larger proportion of young scientists in a larger group fail, as I suspect, this is not recorded. And because no account is taken of wasted lives and broken dreams, these failures do not make a group leader look less productive. I believe the need to maximise publications is also causing students to be treated more like technicians. Thus, increasingly, students are told what to do, so they miss out on learning how to become researchers. Also, group leaders often write up their students’ work: if the writer is not fully aware of the results or exactly how they were obtained this can conveniently permit a more adventurous interpretation of the data. Ignorance can be turned into bliss if the outcome is publication in a higher impact journal. The downside is that papers may be less truthful, not something that is easily measured. Certainly, the student does not get a proper education and may not find his or her PhD period enjoyable — however, even this can be advantageous for the group leader, as that student is less likely to go on and become a competitor!

There’s a lot covered in that one paragraph, but I’ve emphasized one point: the role of grad students. While I have noticed a lot of technician style grad-students, I’m happy to say that most of the grad students I know are not merely following directions. This is especially true in the fields of ecology and evolution, where one grad student or post-doc can design and carry out an independent research project in the span of a few years. In other fields, it seems like grad students are much more tied to the projects being carried out in their adviser’s lab, to the point that they are given a small question to address that will somehow fit into a larger puzzle that the adviser is trying to work out.

Lawrence’s main focus is on how institutions (funding agencies, universities, etc) evaluate academic performance. Because this relies heavily on publishing records (securing external funding is also important), he discusses the publishing industry quite extensively. I’m pleased that he spends a fair bit of text on giving appropriate credit to authors on scientific papers, but it would have also been nice to see some mention of open access publishing as a simple way to make one’s research more visible (without employing some of the more deplorable maneuvers that Lawrence warns against). But, alas, Current Biology is a non-Open Access journal, so that’s not their point of emphasis, although they did let Michael Ashburner shout out for the cause.

Kevin at The Other 95% offers his perspective as a taxono-systematic-ecologist.

Lawrence PA. 2007. The mismeasurement of science. Curr. Biol. 17: R583-R585. doi:10.1016/j.cub.2007.06.014