The Frontal Cortex

Are Scientists Productive?

Posted by Jonah Lehrer on July 27, 2006
(6)
More »

After my last post on the frustrating inefficiencies of experimental failure, I recieved an interesting comment:

I discovered in the late stages of graduate school that my extremely long hours (upwards of 80/week) were extraordinarily unproductive. I was doing cell culture and electrophysiology and while I had reams of data, it wasn’t going anywhere. Only when I switched to a lab doing slice electrophysiology, where the length of the day is limited by the survival of the slice (~6hrs after cutting, making for a 8-9 hr day), did I discover that I could get more work done in less time by increasing my productivity. I’d been fooling myself that my long hours were necessary, when they were really a hinderance (I was so drained that my productivity was poor).

As far as the cheap labor arguments go (and technicians are nearly as expensive as many postdocs), it’s all supply and demand. Postdocs w/demonstrated productivity can command much higher wages if they’re smart (fund yourself w/an NRSA and demand extra from the boss), while others are left out in the cold… And the dearth of PI jobs will continue indefinitely..

Is this a general phenomenon in experimental science? My own experience in the lab suggests that it is. Sleep deprived post-docs are the ones who forget to add the antibody, or add the wrong buffer, or can’t think through the experiment. Scientists certainly work longer hours than they used to, and yet it takes longer than ever to get published. From the perspective of an economist, scientific productivity (at least when measured in terms of papers per hour) is stagnant at best.

But are the numbers missing some important variable? After all, you’d think that with all the technological improvements in lab techniques in the last 15 years (PCR’s, mini-prep kits, etc.), scientific productivity would have gone through the roof. What happened? Have our scientific questions gotten that much more complicated? And how should we measure scientific productivity anyways?

P.S. I’ll venture an tentative answer to the last question. Instead of simply counting the number of papers published as a measure of productivity, we should focus on the number of citations those papers subsequently produce. This way scientists won’t be forced to only tackle tiny questions for fear of not getting published.

(6)
More »

Comments

  1. #1 J.P.
    July 27, 2006

    First of all, I am from engineering background, electrical engineering. So my opinion is biased and limited.

    Using number of citations to measure productivity is not perfect, but better than number of publications? However, when we do the counting, we need to make sure to look at a time span that is long enough. Research is a slow and accumulative process. Some seemingly pointless ideas might turn out to be inspiring 10 years down the road. Unfortunately, both our self -confidence and the society’s merit system aren’t that patient.

    Is it harder to getting published? I definitely think so. Publication in engieering field requires solid data to back up your statement. As we start tackling more complex systems, that isn’t getting easier. I guess it is true to experimental science, to some extent. I wonder if the life of a mathematician or an astrophysicist is easier, since there is no way we can verify that there must be a singularity in every black hole. Note, I am not saying their research is not serious. In fact, personally I think we should encourage speculation type of paper.

  2. #2 Mouth of the Yellow River
    July 27, 2006

    Ni Hao! Kannichi Wa!

    The main reason is the influence of reduction of biology to DNA and protein sequence and subsequent evolution of the corpus scientific endeavor into a production mill of isolated empirical bits of data with little conceptual biological framework.

    There are an estimated 35000 genes in the primary genome sequence alone not to mention coding potential in higher order genome structures. There are myriads of cell types, tissues and organisms in which to empirically describe the gene products in all their variations. This will stimulate keeping the data mills running ad infinitum.

    What happened? Have our scientific questions gotten that much more complicated? Yes, with increased data generation, the novel conceptual integrative answers have become more challenging. Design of novel hypotheses on which to guide data generation in testing them has become more challenging. Thus it is much easier to drop back on less challenging, often redundant, empirical data generation with the hands while mind is elsewhere. Here quantity rather than quality is the measure of productivity.

    And how should we measure scientific productivity anyways? A metric based on quality versus quantity would be a first step. However, citation indices are deficient in dissecting the “popularity contest” factor from true long term scientific impact. Novel discoveries often go unrecognized for years, even by the investigator.

    Some of my old professors speak of the “good ole days” when spending so many hours in the research environment was not viewed as a “negative,” but so much fun and motivating intellectually to the extent that it was the most desirable place to be with ones waking hours. The evolution of research operations into data generation mills staffed by individuals that are kept isolated as competitive individual units of production has made this a nostalgic dream.

    MOTYR

  3. #3 DavidD
    July 28, 2006

    When I was proving to myself that I was better at other things than research, someone who had been around Julius Axelrod told me how Dr. Axelrod would come in on Monday, plan his attack for the week, and usually by Friday have something that he could publish. I was sure then that if I had wanted to be a success, I should be doing biochemistry. Yet I didn’t care about biochemistry, so I kept doing electrophysiology until I no longer owed the government money if I stopped. Then I stopped.

    Just because a problem is interesting doesn’t mean this is the year to tackle it. When a technique comes along that can produce data like never before, that’s the time one can be efficient. At least that’s how it’s looked to me. I never quite got the hang of it myself.

  4. #4 Nick
    July 30, 2006

    On the complexity of scientific questions today, I don’t think it is so much the questions as the means to finding the answer and the many ways you can approach a problem these days, that makes science complex. I agree on the point on sleep. The research institute where I was working during my honours thesis saw me wandering around in the early morning and not leaving before midnight. That said, the times that I was able to synthesize my data and get down to writing were the times that I had said ‘screw it’, took the day off, went to a concert, and came back the following day ready to work.

    That said, more often than not I would wander around my cubicle, feeling particularly bad if I would stray to far from my mac.

    For an interesting read, the physicist and Nobel prize-winner Murray Gell-Mann has a great introduction to his book The Quark and the Jaguar. He starts off in the jungle, thwacking through brush while getting ideas related to quantum physics.

  5. #5 bigTom
    July 31, 2006

    I’m coming at this from the standpoint of a computational scientist, but I think the issue is pretty common across fields.
    I think the biggest problem is exhaustion of easy problems. Certainly in computation there were lots of important problems that could be addresses with a small scale effort, but these have been solved, added to libraries of known methods, and we move on. Now of course there are always those important original ideas, that can produce important results with only modest efforts, but these are few and far in between. Given that most fields have hundreds or thousands of researchers, and a very limited number of big ideas (maybe a few a decade per field), the odds are against any particular researcher coming across one of them. So the only way for the vast numbers of less blessed researchers to make contributions is to make small advances by working hard.
    Every now and again, a revolutionary idea comes along, and breaks a field wide open, allowing a lot of people to make progress quickly, but after a few centuries of scientific progress, these events are becoming all too rare.

  6. #6 fussball deutschland
    March 4, 2009

    Gute Arbeit hier! Gute Inhalte.