SVP Ethics Education Committee "best practices" for research, publication, and museum work.

As promised, in this post I'm examining the "best practices" document (PDF) issued by the Society of Vertebrate Paleontology Ethics Education Committee in the wake of the "Aetogate" allegations. Here, I'll discuss the specific recommendations made in that document. And in an upcoming post, I'll turn to some of the discussions paleontologists are now having (through the magic of the Internet) on the accepted practices in their field, in hopes of gaining some insight to the fit between actual practices and the "best practices" described by the SVP Ethics Education Committee.

The first section of the document deals with best practices for research and publication. The two go together since knowledge-building in a scientific field is a project of the community. If you do research, the job isn't done until you communicate your results to the rest of the community. (Indeed, even then it's not done, since people are supposed to use those published findings, to examine them and perhaps even challenge them, but that whole ongoing conversation never gets started if scientists just do research but fail to communicate their results.) We'll take each of the specific recommendations in turn:

1) Publishing original work. Although science is a progressive process and ideas and evidence will always build on each other, including the work of other researchers, plagiarism or other misuse of the intellectual property of others is unethical and may constitute a copyright violation under U.S. and international law. Plagiarism includes copying of text, data, or ideas without proper attribution; such actions work against scientific honesty. Moreover, individuals should endeavor to avoid the appearance of plagiarism by thoroughly reviewing and citing all relevant literature.

First, a matter of phrasing that bugs me: "such actions work against scientific honesty"? Why not just say "such actions are scientifically dishonest"? Because they are.

Here, at the top of the list, is the "don't plagiarize" item. Note that this definition of plagiarism (like most of the definitions in use at universities, journals, etc.) includes ideas (not just words or data), and that it does not require intent. In other words, it's not the standard applied by the SVP Ethics Education Committee in adjudicating the allegations against Lucas et al. More on this in a moment.

Also, notice that this recommendation makes clear that researchers have a positive duty to engage with (and properly acknowledge) the relevant literature. Arguably, the duty here stems from more than just the interest of "avoiding the appearance of plagiarism", though. You ought to engage with (and properly acknowledge) the relevant literature because doing so will improve your contribution to that body of knowledge -- the existing work contributes to the theoretical framework and the evidential basis in which your work is grounded. Moreover, actually acknowledging this existing work is paying the other members of your professional community for their contributions to the shared body of knowledge in the coin of the realm. Failing to so recognize their contributions does them harm, at least relative to the score-keeping system that is tied to scientific careers. Not harming other members of your professional community strikes me as a more honorable goal than avoiding the appearance of plagiarism (which the distinct flavor of butt-covering).

Do these best practices for the community of vertebrate paleontologists set the bar higher than what members of the community can get away with when allegations are brought? If the case against Lucas et al. is an indication, it would appear so. However, it may be appropriate to aim for a better level of conduct than the minimum you would hold people to in defending themselves from allegations of wrongdoing. Lower levels of conduct may result in a worse body of knowledge, worse relations between members of the professional community, and actual harm to particular members of the community, but there may be a gray area between living the best practices and setting out to screw over your competitors (or even screwing them over through a policy of willful neglect). Moreover, formal organs of professional communities may be hesitant to impose sanctions unless a high burden of proof is met against the accused parties.

This doesn't mean that nothing should happen to members of the professional community who fall far short of the best practices. To the extent that the best practices are to be taken seriously -- to the extent that the recommendations are hoped to inspire actions from members of the professional community -- there must be some real consequence for ignoring the recognized standard. That consequence could be a good old fashioned shunning by one's professional peers, but there has to be something.

Otherwise, these recommendations are just empty words.

In any case, the practices of Lucas et al. as acknowledged by Lucas et al. and by the SVP Ethics Education Committee fell far short of the best practices described in this first item.

2) Seeking independent review. Reviews of published work should be sought from individuals who have no current or past associations with the author(s) that might bias their review. This sort of thorough editorial review shall be scrupulously practiced by the society's own publications, and should be the standard for our members in other publications.

This point emphasizes the importance of doing peer review well. It's not just about getting published, but about building good knowledge -- something that has been throughly critiqued and interrogated prior to publication, yielding results that are reliable and useful. Ideally, the review should be focused on matters of substance -- not clouded by personal affection or animosity. This makes it reasonable to seek peer reviewers who don't have personal connections to the authors one way or another.

By all accounts, this is not what was happening at the NMMNHS Bulletin.

Note that this is a "should", not a "must". How will this standard be lived in the professional community of vertebrate paleontologists? Will paleontologists insist on transparent reviewing practices from the journals they consume and to which they submit their own work? Will members of the community display a preference towards submitting to, and citing from, those journals whose practices are closest to this standard? Will they cultivate a healthy skepticism about work published in journals that depart from this standard?

Note, however, an inescapable source of temptation: given the ICZN rules on naming, the first to publish a new name establishes the official name to be cited in further literature on the critter. For the paleontologist trying to establish priority for a new name, this provides a pragmatic reason to use a journal with lax reviewing standards, since such a journal is also likely to have a quicker turnaround time between submission and publication. Within the professional community, are there any consequences for paleontologists taking this shortcut?

3) Avoiding conflicts of interest. Researchers should not let personal interests or monetary compensation bias the results of their research or their reviews of others' research. Any conflicts of interest should be avoided, and if this is not possible, should be explicitly stated.

This builds on the recommendation in item 2. If scientific knowledge is to be objective, scientists must make serious efforts at the individual level to screen out their own individual interests and biases that might cloud their results, their conclusions, and their evaluation of the results and conclusions of other workers in their community.

Of course, in real life this can be a difficult goal to enact perfectly. In some instances, there may only be a small group of scientists working on the same set of puzzles. This can mean that the people who know the area well enough to provide a serious review of your work are likely to be your competitors and your collaborators. Either one of these relationships could color the review. Noting these relationships up front reminds the scientist doing the review to make an extra effort to provide an objective evaluation and flags this as a situation in which others in the community may want to be on the lookout for biases that conscious effort was not sufficient to screen out.

4) Substantial contribution for authorship. Individuals should appear as an author only on those publications in which the individual has contributed substantially to the design, data retrieval, analysis, interpretation or writing of the published work.

In other words, guest authorship bad, a judgment with which I heartily concur.

I do find it interesting, though, that this recommendation is presented with an "only" rather than "all and only". Arguably, someone who has "contributed substantially to the design, data retrieval, analysis, interpretation or writing of the published work" is entitled to appear as an author, and including her is just as much a matter of good practices as is not including an "author" who has not made such a substantial contribution.

Possibly there is no problem in the vertebrate paleontology community with substantial contributions being overlooked when the author line is worked out?

5) Approving publications. All authors should approve the final version of publications on which he or she appears as an author.

Once again, this sounds right to me.

6) Publishing work in a timely manner. Long delays to publication are at odds with our mission of active dissemination of results, especially when this practice is associated with restricted access to fossil or other specimens for study by others.

Here, I'm going to quote my earlier post, where I wrote:

There is certainly an extreme at which activity is too slow -- where the slowness of a scientist in conducting the research and sharing the results might impact the community's ability to build important knowledge (including the knowledge that might be built using the results-that-never-resulted as a starting point). I'd imagine molasses-like progress on the part of a paleontologist who controls certain rare specimens might present an extra difficulty, given that while those specimens are "under study" other paleontologists are excluded from accessing them. On the other extreme, moving too fast might mean publishing findings that are not well thought out, well supported, or scrutinized for weaknesses before they are offered up as knowledge to the professional community. What kind of time is required to expose one's own conclusions to the appropriate level of scrutiny? Is the ability to expose your conclusions to scrutiny helped or hindered by restricted access to your fossils "under study"? Is there a uniform level of serious organized skepticism in the peer review applied to manuscripts by the different journals in a field like paleontology (or do different levels of stringency in peer review account for different average times between manuscript submission and publication)?

Finding the goldilocks pace that is "just right," the level of scrutiny of your results that is enough before you put them out there, is sometimes a challenge.

From the point of view of taking enough time to draw good conclusions from sufficient data, you don't want to rush. To the extent that your conclusions should undergo a thorough review (see item #2), that adds time to the process as well.

But if you're dawdling simply to maintain control of your specimens "under study" -- especially if you do this when you know other researchers are waiting for them to become available for them to study -- you're just being a jerk, and you are doing so in a way that does harm to other members of your professional community.

That's not a best practice.

After item 6, there's an exhortation in favor of "free and open communication" between members of the professional community, and then there is this:

It is especially incumbent upon more senior scientists to uphold the highest standards for professional conduct, as they serve as role models for younger scientists and graduate students.

This is true, for good or for bad. Those entering the scientific community look at how those who are training them actually behave. These displays are what the trainee has to draw on, in terms of working out appropriate behavior in that community.

Again, this means that best practices must be lived values, clearly guiding scientists' conduct. Otherwise this carefully crafted document amounts to nothing but empty words on a page.

Theres a bit more verbiage that seems to throw the chief responsibility for policing plagiarism and "other misconduct in publishing" (which presumably includes falsification and fabrication) back to the journals, the universities, and the museums.

Again, let me quote from my earlier post:

If there's anything this case seems to show, it is that employers and journal editors may find a multitude of reasons not to take allegations of wrong-doing against their employees or authors seriously. Universities, museums, and journals all have many interests, some of which may lead them to minimize the legitimate interests of parties not in their employ (or editorial board). Moreover, ignoring or minimizing allegations promises to be less resource-intensive than mounting thoroughgoing investigations of those whose names and prestige are already linked with yours.

But was the alleged misconduct primarily a matter of offenses against an employer, or against a journal editor? Or is it more properly understood in terms of offenses against a professional community and its members?

There are lots of entities here that probably -- according to their own official rules -- ought to be involved in policing bad behavior when it happens. However, given that the kinds of bad behavior alleged are ultimately doing harm to the functioning of the paleontology community, that community needs to be actively involved in responding to it.

But this raises a practical question: What kind of power does a professional community have to enforce its own standards -- especially in cases where journals and employers fail to enforce these standards or concern themselves with different standards? Here, for the time being let me just suggest that it is possible to mount a meaningful and effective response to bad behavior within a professional community even if you do not have the power to fire the bad actor or to cause the withdrawal of her publication or the freezing of her grant money.

The remainder of the document concerns best practices in museum research. The committee prefaces the four specific recommendations in this part of the document with a reminder of how the data on which paleontologists draw are importantly different from the data typically encountered in other scientific disciplines:

In the interests of advancing vertebrate paleontological science, museums and other professional repositories housing vertebrate fossil remains should provide access to those fossils for qualified researchers with legitimate research programs. Access may also be warranted in may cases for educational and artistic endeavors. However, we emphasize that repositories are primarily responsible for maintaining and conserving the integrity of the fossil remains and data under their care. In all cases, repositories must evaluate whether the proposed activities may impact the integrity of the fossils and the potential data that the fossils provide, and determine rights of access accordingly. Visiting researchers should understand that some fossils might be too delicate to be studied intensively, to rare to be sampled destructively, or currently inaccessible because of legitimate study by other researchers.

The paleontologists' data come from scarce and fragile resources, which is why those resources are typically kept in museum collections. Any practices that harm the fossil remains, obviously, pose a threat to the knowledge-building project of the community of paleontologists.

1) When making arrangements to study material in a museum's collection, visiting researchers should make sure that the museum they are visiting is aware of what they are studying and why, and what they intend to do with the observations made at the museum. In general, permission to study material in a museum's collection resides with either the collections manager or the curator in charge, but this is not always specified. It is most effective to copy all relevant curatorial personnel on the correspondence.

The keepers of the fossils are trying to fulfill their obligations to care for those fossils so they can be put to good use by the community of paleontologists. Thus, it is your duty to establishing that it's OK for you to study a specimen, both from the point of view of preserving a specimen for further research, and from the point of view of respecting someone else's legitimate claim to work on a particular specimen before you do.

Making sure that you've secure permission from the person or persons empowered to grant permission ensures that your use of the specimens is consistent with their preservation for use by future researchers. Presumably the person empowered to grant permission because is the one in charge of preservation and knows which specimens are currently under active study (about which more when we get to item #3).

Reading between the lines here, possibly it is acceptable for more than one researcher to study the same specimen simultaneously if they are working on different research questions involving that specimen. (This is a good reason to be clear about what one is studying and why -- to identify non-overlapping research that could be conducted in the same time frame, logistics permitting.)

2) All museums and repositories should have policies regarding access to material in collections for research purposes, although these policies may not be written or stated explicitly. Museum and repository curatorial staff are responsible for ensuring, preferably in writing, that visiting researchers are fully cognizant of all pertinent institutional collection-care policies, procedures, and restrictions. ... Permission to observe material is not necessarily equivalent to permission to publish on it, so researchers should be sure that they have express permission to publish on material before doing so.

Be clear about your policies (and formulate some, rather than just making it up as you go). Communicate them clearly to the researchers interested in accessing specimens in your collection. Be transparent about which specimens are "under study" and thus not fair game to study in order to generate results to submit for publication.

Here, it's worth asking whether research for a thesis or dissertation be fair game under these circumstances (at least is the thesis or dissertation is not turned into publications until after the team "officially" studying the specimens have published their results). Again, this is connected to item #3.

3) It is understood that researchers working in museums and other professional repositories may be actively studying the fossils and data under their direct care. In these cases, it is acceptable to withdraw such specimens from more general research access for a reasonable period of time, until the repository researchers have completed and published the results of their investigations. However, it is also incumbent upon the repositories in these cases to clearly inform visiting researchers of the status of these fossils, so that conflicts do not arise. We emphasize that repository personnel should endeavor to make the fossils under study available following the publication of their results. Science is based upon verification and repeatability, and these often require that access be provided for outside researchers. Where multiple curatorial and collections personnel at a given repository are actively conducting research on fossils under their care, we recommend open and frequent communication among these scientists about their research programs.

There is a need to be clear about what's under study and what isn't.

The recommendation here seems to indicate that once you've published on a particular specimen, you ought to open it up to other researchers for study. That's how you build reliable, objective knowledge. In some sense, it's not just the knowledge that belongs to the professional community -- the specimens with which paleontological knowledge is built belong to the community, too.

Keeping fossil remains out of circulation for too long hurts the prospects for knowledge-building that are the very point of paleontology. This does harm not only to individual paleontologist whose research plans may be thwarted, but to the professional community as a whole.

4) Visiting researchers should inform the museum of the results of their work based on the museum's collections. ... [T]he museum will want to know what visiting scientists have done with results of the observations on their specimens, and especially what abstracts or papers are published that include reference to materials in their collections. Published papers, published abstracts, dissertations, and theses should be provided to the repository in a timely fashion.

Building a coherent body of scientific knowledge involves communication, and communication is a two-way street. Knowledge already built about particular specimens ought to be useful in the building of further knowledge about those specimens. Moreover, sharing the knowledge you've built helps avoid the problem (at least from the point of view of career rewards) of someone else unknowingly building that knowledge again.

Of course, there's still a duty on the part of the recipients of published papers and abstracts, and unpublished dissertations and theses, to actually read them, and read them carefully.

The best practices articulated here by the SVP Ethics Education Committee seem like reasonable ones which, if reflected in the behavior of members of the professional community, could improve their coordinated efforts to build a reliable body of knowledge. The crucial question is how to ensure that everyday practices follow these best practices. Maybe, to a significant extent, they already do. If so, that still leaves open the question of how best for the community to deal with bad actors in its midst.

In any case, I'll be writing a follow-up post in which I examine what working paleontologists have been saying about what they see as the accepted practices within their professional community. It may take a few days, but it's coming!