Chris Rowan is a geologist specialising in the dark arts of paleomagnetism, and getting people to pay him to travel to exotic destinations for fieldwork. Having drilled up New Zealand during his PhD, and South Africa in his first post-doc, he now works at the University of Edinburgh.Janet brings us some rather vitriolic criticism by Sir Peter Medewar:
The scientific paper in its orthodox form does embody a totally mistaken conception, even a travesty, of the nature of scientific thought.
The argument seems to be that the usual structure of a scientific paper reinforces the notion of science as a purely inductive process: we start with an empty mind, collect the data, and then proceed to deduce theories and models which explain them. This, people like Medewar argue, plays down the importance of generating ideas as a first step to doing science, by pretending that we use data to generate hypotheses, rather than also using hypotheses to design our experiments (and decide what the hell we're going to research anyway). These are deep philosophical waters for an innocent geologist to swim in, but it seems to me that this criticism, whilst valid in a sense, misses an important point: we don't write scientific papers for historians and philosophers of science. We write them for other scientists.
As such, and interesting as that might be, our major concern is not to provide a narrative of the often tortuous mental process which led us to the theory or model that we are presenting; whilst generating new hypotheses to test is a fundamental challenge for the individual scientist, the fact that the idea for flux capacitance popped into your head after you slipped off the toilet and banged your head is not relevant to your argument that flux capacitance makes time travel possible. Instead, our goal is to present the clearest and strongest possible scientific case for our models, in order to convince others in our field of its validity. Thus:
- You describe the background to the problem you've been investigating, and why it is important (Introduction).
- You explain how you did your experiments (Methods).
- You present the results for your experiments (Results).
- You explain what you think that your results mean (Discussion/Conclusions).
It's true that we are imposing an artificial structure, but that's because we're trying to build a logical argument, not tell a story. This does not mean that preexisting or preliminary hypotheses should be ignored, of course; talking about competing hypotheses and their consequences in the introduction can be a vital part of the quest to show that your research is relevant and interesting. It is true that many papers fail to do this, but I'd suggest that this is more a symptom of bad writing than conscious fraud, or an attempt to meet some impossible inductive ideal.
However, the most troubling part of Sir Peter's criticism is his particular disdain for the usual habit of separating the data, and the interpretation of said data, into separate 'results' and 'discussion' sections:
The section called "results" consists of a stream of factual information in which it is considered extremely bad form to discuss the significance of the results you are getting. You have to pretend that your mind is, so to speak, a virgin receptacle, an empty vessel, for information which floods into it from the external world for no reason which you yourself have revealed. You reserve all appraisal of the scientific evidence until the "discussion" section, and in the discussion you adopt the ludicrous pretense of asking yourself if the information you have collected actually means anything.
Again, he's right that this division is not part of natural thought processes, which is why I spend most of my time when teaching field mapping trying to force the students to properly separate observation from interpretation (or, in many cases, actually record the observations which led to their interpretation, but I digress). The reason I do this, and the reason that this should always be done as much as possible in scientific papers, is that it increases the long-term value of the data, for the simple reason that our interpretations may be wrong. In fact, the scientific process almost guarantees that they will be wrong in at least some aspects, because it is continually generating new data which force refinements, or even major rethinks. Conclusions come and go, but good experimental data lives forever; therefore making the effort to keep the two separate mean that the latter can always be easily put back to work, constraining the improved models. If not, vital observations may languish in the middle of a few paragraphs of discredited extemporizing.
Don't let me be misunderstood; I strongly believe that scientists should spend much more time talking about how they actually do science. But criticizing a scientific paper for not representing this process is invalid, because that is not what a scientific paper is designed for. We need to use other methods to tell that story, in more appropriate venues. Perhaps some sort of informal collection of musings about sciency things. We could even download them onto that Internet thing...
Comments
I think scientist need to live with the fact that after one writes something it is out there for the people who read it. I am not saying there is a way to address the needs of every possible reader but I do think that writing should not be taken lightly.
Moreover, whatever you do you are presenting a narrative. I think the criticism is justified because the type of narrative science papers present is flawless and clear. The facts are perfectly ordered, the language one uses to describe the facts appears again in the interpretation. There appears to be no possible 'other way'. This is great for clarity but lacks in ... science. As a scientist you know the problems with that approach, the possible ways out. Moreover you are open to new ways of seeing facts and modify the theory. Writing a 'science paper' with that clear pattern that you mention actually cuts the problems off. For that paper there is no other possibility and when it goes out to some reader who is not very critical it becomes a fact. This way of writing makes it also easier for non scientist to believe they see the truth of science and, since there appear to be no doubts involved, assume it as final truth. Look at the headlines in the newspapers: 'Time travel is possible!', 'Men like xxx or yyy women' etc. They all spring from legitimate experiments and they do say something like what the title says but in a very, very different way. Scientist will never take something like this for granted (meaning they will understand the words for what they worth) but many use headlines like these as arguments in their conversation and in their life decisions. We are in a limited language and words that have some meaning in the jargon do not mean the same thing for the public.
I would also challenge your statement that data is data. I am convinced (and there are a number of studies showing it) that perception itself is biased by the theory behind. There are people who research exactly what were the conditions over history that made people believe their experiments are correct. etc etc. Data in not just valuable in itself but also because the scientists describe in detail how they measure it - Data and context are the interesting part in science.
The criticism is justified because this is the actual effect that the 'scientific paper' has regardless of what it is originally designed to do. At the same time I do believe that there are things scientists can do to minimize that effect and this is why I think it is important to talk about it.
Posted by: Lucian | July 14, 2007 9:15 AM
The "artificial" structure of separating data and results may be a reflection of how the research was carried out. As a research tech I generated data. While I knew the general hypothesis, my role was to ensure my results were reproducible. At meetings we discussed whether to collect further data but the discussion and conclusions were really for the priniciple investigator and co-authors.
I've always regarded scientific papers as methods for relaying scientific information. Perhaps organizations that produce scientific journals should also produce more general reading type magazines that will generate interest.
Posted by: Cass | July 14, 2007 11:13 AM
I wholeheartedly agree with everything you said, Chris. The data and interpretation must be separated. For my master's work I called upon a set of papers that were written in the 1930s and 40s. The ideas about what the geology represents are antiquated (this was pre-plate tectonics), but the data was extremely valuable to me. Like you said, it's the long-term value. These old papers were so well-organized in this regard. When I read some more modern papers, there's a messy mixture of data and interpretation. When paradigms shift these papers will be lost.
I think it is also important to think about the distinctions between historical and experimental sciences. When the scientific endeavor is all about designing and running experiments, the traditional scientific method we all learn in school is relatively straightforward to follow. But, in a historical science where you are trying to unravel what nature has already produced, this workflow is modified. When characterizing the geology of an area, what exactly is the experiment? In these cases, the 'experiment' is testing various ideas against the objective information. And as Lucian noted above, what's objective data to one might be very different for another.
When geologists go out to the field, they have to have some framework of what kinds of questions they are interested in. The amount of information that nature has produced and the various ways in which we measure, collect, describe, characterize are seemingly infinite. What we decide to focus on is part of our experimental design. If i'm interested in the depositional processes of a Cretaceous sedimentary rock, I'm not gonna spend my time measuring fracture orientations that were generated in the Miocene. Being selective in this regard is part of our experimental design.
Medawar states: "The discussion which in the traditional scientific paper goes last should surely come at the beginning."
The papers I read (and try to write) already do this. A good Introduction will discuss the major concepts or models that are being scrutinized. In a way, it's a preview to the Discussion section. And then when you get to the Discussion, after you've evaluated the data and interpretations for yourself, it becomes that much more valuable. Perhaps Medawar is reading papers that inadequately introduce the problem at the beginning of the paper.
Posted by: Brian | July 14, 2007 1:47 PM
I didn't read the comments and I certainly agree with you overall. However, I will say that publishing in journals which allow combined a 'results & discussion' section (there are many in the biological sciences) is really nice.
The paper flows so much more naturally and the reader gets the instant explanation of data that may keep them interested in finishing the article. Besides, the data is still present in a pure form in the figures/tables.
Anyway, I guess I just didn't think it was "the most troubling part of Sir Peter's criticism..."
Posted by: Pelio | July 14, 2007 3:03 PM
I tell all my trainees that really all they need to look at in a scientific paper is the data: figures and figure legends. They should come to their own conclusions about the interpretation of the data, their significance, and how they fit into what is already known in the field.
If they care to, of course they can look at the Intro and Discussion to see the authors' views on these matters, but I emphasize to them that the authors' views on these matters are no more privileged or worthy of respect than their own.
Posted by: PhysioProf | July 14, 2007 3:36 PM
I do that too, in my structural geology field labs. In fact, I make students write up their field labs as an "observations" or "structural data" section and an "interpretation" section. My initial reason wasn't to force students into a rigid scientific writing style - my reason, actually, was that I had had problems getting students to really think about whether their observations fit with the answers they assumed were correct. Sometimes the students remembered interpretations from intro classes, and sometimes they tried to deduce the correct answer from things I said, or things the brightest students in the class said. But they weren't really thinking about whether their interpretations really fit what they observed in outcrop. So I forced them to separate their observations from their interpretations, and then graded based on whether their interpretations made sense given the data they collected. (I had other ways of making sure their data was reasonable.)
One thing that I discovered was that, in the process of thinking about what should count as data vs interpretation, the students actually thought a lot more about the subjectivity of their measurements and descriptions. (Should calling a certain plane "cleavage" be a description or an interpretation, for instance? Does the answer change if it's the first lab of the semester vs the last lab of the semester?) So the writing style may appear to make science seem more certain and objective than it is, but I think the process of separating interpretation encourages scientists to consider alternate interpretations. Though if it only allows other scientists to offer alternate interpretations, I think it's still a valuable mental exercise. (And I would argue that reviewers, in many cases, will offer alternate interpretations for data, so scientists frequently have to confront possibilities that they hadn't considered.)
(May I link to this post for my structural geology and technical writing classes? This is a fantastic discussion of the nature of a scientific paper from the perspective of a geologist, and I would like my students to read it and talk about it.)
Posted by: Kim | July 14, 2007 3:51 PM
If you think you can use it constructively, be my guest.
Posted by: Chris Rowan | July 15, 2007 5:34 AM
Hi Chris. Came across this by accident..looks like you've been busy since you left us here at NOCS! Looking forward to seeing you in August.
I don't see any problem with the way a modern science papers is written. Research is hypothesis driven, period. Well, at least it is for some of us geologists, stamp collectors apart. The ancient template that separates out results, interpretation and conclusion hides the fact that, generally speaking , scientists are not very good at writing. Whilst it is important to observe first and interpret second during any research exercise, it is hardly inappropriate when it comes to communicating that science to expect a modicum of fluidity. This not a recipe for a messy mixture of data and interpretation and it is ridiculous to suggest that in the future this style of writing will condemn our ideas to the bin. Most science is forgotten because most research that goes on will never be life changing or of much relevance to solving any 'big problem'.
Anyone that claims to be an academic should be able to examine the data independent of the author's opinion. However, I think that that is the whole point about writing science papers..to say what you think. Science is an iterative process and science papers should reflect that and not be bound by some out of date rigid framework for presentation. Most data that are published today can be found on databases or on associated supplementary files. Hey, if you rarely believe the hype and want to test someones conclusions then email the author and ask them for their data. I do this a lot...most people don't appear to mind.
Posted by: Ian Bailey | July 15, 2007 7:59 AM
Good thread; important meta-subject.
"Perhaps some sort of informal collection of musings about sciency things. We could even download them onto that Internet thing..."
More to the point: the webby thing, as originally intended at CERN, does indeed have more and more raw data, analyses therefrom, preprints of papers about, final versions of papers, and reactions to paper.
Open Source science publishing is as revolutionary as moveable type books, in my humble opinion.
I spend more and more time writing and reading arXiv papers, PLOS papers, papers intended for interdisciplinary conferences with on-line proceedings, in online journals, and in open source-ish edited databases (such as, for instance, the 125,000+ page Online Encyclopedia of Integers Sequences, edited brilliantly by Dr. Neil J.A. Sloane and associates at AT&T Research Labs, and often carefully unedited (but marked "uned" as such).
My impovershed inner city High School summerschool students (while I'm between professorships) were surprised that their essays on "Why I Hate Math" were blogged, although unclear on what a blog was, beyond "kind of a diary on the internet."
To me, the killer app of the net is collaborationware. More and more, I am managing an ad hoc network on multiple continents of people in multiple fields, many of whom I've never met face-to-face, who found me via the web, and communicate via email and blogs. Actual refereed papers result. Plus the pseudopapers and minipapers and micropapers and quasipapers alluded to.
It's revolution, or meta-revolution. What do YOU think?
Posted by: Jonathan Vos Post | July 15, 2007 3:03 PM