As promised, I've been thinking about the details of Chandok v. Klessig. To recap, we have a case where a postdoc (Meena Chandok) generated some exciting scientific findings. She and her supervisor (Daniel F. Klessig), along with some coworkers, published those findings. Then, in the fullness of time, after others working with Klessig tried to reproduce those findings on the way to extending the work, Klessig decided that the results were not sufficiently reproducible.
At that point, Klessig decided that the published papers reported those findings needed to be retracted. Retracting a paper, as we've had occasion to discuss before, communicates something about the results (namely that the authors cannot stand behind them anymore). By extension, a retraction can also communicate something to the scientific community about the researcher responsible for generating those results -- perhaps that she was too quick to decide a result was robust and rush it into print, or that she made an honest mistake that was not discovered until after the paper was published, or that her coauthors no longer trust that her scientific reports are reliable.
The issue is complicated, I think, by the fact that there were coauthors on the papers in question. Coauthors share the labor of doing the scientific work, and they share the credit for the findings described in their paper. You might expect, therefore, that they would share responsibility for quality control on the scientific work, and for making sure that the findings are robust before the manuscript goes off to the journal. (In my first post on this case, I noted that "before the work was submitted to Cell, Klessig had one of his doctoral students try to verify it, and this attempt was at least good enough not to put the brakes on the manuscript submission." However, given that further efforts to reproduce the findings seem not to have succeeded, I suspect opinions will vary on whether this pre-submission replication was enough quality control on the authors' parts.) And, you might expect that it would be the rare case where a problem with a published manuscript would come to rest on the shoulders of a single author in the group.
If credit is shared, why isn't blame?
Whatever you think ought to be the standard assumptions when a collaborative piece of scientific work does not hold up, in this particular case the blame seemed to fall on Chandok. She took issue with the implication of the retractions (among other communications) that she was unreliable as a scientific researcher. Probably she considered the importance of trust and accountability in the scientific community, recognizing that if she were not trusted by her fellow scientists and if her work were viewed as presumptively unreliable, she would not have much of a scientific career ahead of her. So, she sought legal remedy for this harm to her scientific reputation and career prospects by pursuing a defamation claim against Klessig.
There are separable issues at play here. One is the question of what is required in the eyes of the law to prove a claim of defamation. Another is what would constitute "best practices" for scientific work, both in terms of dealing with data and conclusions, and in terms of dealing with the scientists who generate the data and conclusions (and who are the main audience for the findings reported by other scientists). Here, I think "dealing with" encompasses more than simply classifying fellow scientists by whether or not you can trust their scientific output. It includes interactions with collaborators (and competitors) , not to mention interactions in scientific training relationships.
We might quibble about where a postdoc falls in the process of scientific training and development. Nevertheless, if the PI supervising a postdoc is supposed to be teaching her something (rather than just using her as another pair of hands, however well trained, in the lab), he may have specific responsibilities to mentor her and help her get established as a PI herself. Sorting out what those responsibilities are -- and what other responsibilities could trump them -- might be useful in preventing this kind of acrimonious outcome in other cases.
We'll return to considering the broader lessons we might draw from this situation, but first let's continue laying out the facts of Chandok v. Klessig, 5:05-cv-01076. (Again, I'm indebted to the reader who helpfully sent me the PDF of District Judge Joseph M. Hood's ruling in this case, which is what I'm quoting below.)
When Klessig and his coworkers at Boyce Thompson Institute for plant research (BTI) ran into trouble trying to reproduce Chandok's work, he sent a request to Chandok (who had by that time moved on to another position in a different state) that she return to help them with their attempts. As I read it, though, this request for help was paired with a threat:
After Dr. Chandok left BTI, Dr. Klessig and Lucy Pola, Human Resources director at BTI, sent a letter to Dr. Chandok stating that her results still had not been duplicated. The letter requested that Dr. Chandok return to BTI to assist in verifying her results and indicated that, should she fail to return, Dr. Klessig would begin a scientific misconduct investigation and withdraw support for her visa application. [Ex. 35.] Dr. Chandok did not return to assist in the research. Dr. Klessig initiated the investigation by reporting the possibility of scientific misconduct to Dr. Stern, then President of BTI, and Lucy Pola. (3)
Arguably, Chandok had a responsibility to assist Klessig and coworkers in their efforts. After all, she was claiming that the results she reported (and that she reported with Klessig, putting his name and reputation behind them as well as her own) were legitimate. In the scientific arena, asserting that your results are legitimate amounts to asserting that they are reproducible -- in other times, in other places, by other researchers. Reproducibility is how you establish that your results are legitimate, that what you've found is a robust effect rather than a fluke.
But, should the first request from a frustrated coauthor struggling to reproduce a reported finding contain an explicit threat? One wonders if there were other requests that didn't come up in the legal proceedings (although if there were, I can't imagine why Klessig wouldn't want that fact on record). I also can't help wondering whether there was something in Klessig's history with Chandok that made him feel that including the threat with this request for assistance was a good idea (either from the point of view of making it more likely that Chandok would actually provide assistance, or from the point of view of covering Klessig's own scientific butt).
The description of events in the judge's ruling continues:
After reviewing the data, Dr. Stern concluded that the investigation should go forward and formed a committee. Dr. Klessig sent an email agreeing with Dr. Stern. Then, Dr. Klessig submitted several suggested phrasings of an allegation of scientific misconduct to the BTI investigation committee. Dr. Susan Ridley from the National Science Foundation and Dr. James Anderson from the National Institute of Health, the relevant federal agencies to whom scientific misconduct should be reported, were next notified by Dr. Klessig. Dr. Klessig then began discussing the phrasing of the retraction letter that would be sent to Cell magazine and PNAS. Several drafts were suggested to Pres. Stern, Ms. Pola, Dr. Ekengren, and Dr. Martin. Once a final form was agreed upon, it was sent to the editors of Cell and PNAS as the formal retraction. (4)
Here, remember the two separate but connected issues that were on the table: whether the results that had been reported by Chandok could be trusted, and whether Chandok herself could be trusted. Retracting results because they are judged unreliable does not necessitate deciding that the results you're retracting have been fabricated or falsified -- and it seems like the burden of proof for the first of these judgments (at least scientifically speaking) is lower than that for the second.
We've already had a look at the allegedly defamatory claims Klessig made. Many of these claims seem to convey Klessig's personal conviction that misconduct was the source of the problems with Chandok's results -- and, as far as I can tell from the timeline in the judge's ruling, to communicate this conviction to the journals and other scientists before the BTI investigation had reached a conclusion on the matter. The gap between what Klessig believed and what he could prove seems like it might be important both in the scientific arena and in a court of law.
The ruling next lays out the relevant legal standard:
To establish a claim of defamation under New York law, a Plaintiff must establish 1) that the statement averred was defamatory; 2) that the statement was published by the defendant; 3) that the statement was communicated to a party who was not the plaintiff; and 4) the resultant injury to the plaintiff. (13)
Points 2 and 3 seem not to be in dispute. Point 4 seems arguably true from the point of view of the good reputation one needs to function successfully within a scientific community. Point 1 is the question I asked you to ponder in the earlier post with the 23 claims.
The judge's take on these claims is pretty shrewd:
Each of the twenty-three statements is reasonably susceptible to a defamatory meaning. Defendant argues that Statements 2 and 9-19 do not concern Plaintiff, but simply the "results" or "data" obtained through Plaintiff's research. The Court disagrees. Statements 2 and 9-19 are centered around the results of Plaintiff's research, and that fact that, despite numerous attempts, no other scientist was unable to replicate Plaintiff's results. The individuals to whom Defendant published Statements 2 and 9-18 were members of the scientific community, many of whom collaborated with Plaintiff on the NOS research and attempted to replicate the results. While a member of the general population may not understand Statements 2 and 9-19 to refer to a particular individual's work, the individuals to whom these Statements were published certainly might.
Among the allegedly defamatory statements are Defendant's comments that "there MUST be an investigation . . . given the evidence of falsification," [Statement 1], that there were "difficulties in reproducing the data," [Statement 11], that other scientists "have been unable to repeat the results," [Statement 13], and that the data was "shaky" and "unreliable." [Statement 19]. The scientifically sophisticated individuals to whom these Statements were communicated could very well understand the references to falsification, difficulty reproducing data, and an investigation to be a statement that someone, i.e., Plaintiff, falsified or fabricated her research. Upon reading these communications in full and giving consideration to the context in which they were sent, the Court finds that Statements 2 and 9-19 could reasonably be considered to be susceptible to a defamatory meaning. (14-15)
The judge, in other words, recognized that the reliability of the findings and the reliability of the researcher who produced them are entangled. As such, he didn't buy Klessig's claim that assertions impugning the goodness of the data didn't also impugn Chandok.
Of course, if Klessig had solid evidence that Chandok committed misconduct, that would be a reasonable defense -- remember that an element of defamation is that the defamatory statements are factually false.
Did Klessig have enough evidence of misconduct to support the statements about Chandok and her work that he made to others? For that matter, did the people with whom he was communicating have access to the facts needed to evaluate his claims -- those that supported a judgment of misconduct and those that supported an alternative conclusions (either that the attempts to reproduce the original findings might themselves be flawed, or that the problems with the original results were due to honest mistakes)?
As it happens, the judge sidesteps that question:
A statement is a pure opinion if, inter alia, it is accompanied by a recitation of the facts upon which the statement is based and does not imply or assert the existence of any undisclosed facts unknown to the audience. ... However, if the statement is accompanied by assertions that it was based on facts that are not disclosed and unknown to the audience, then the statement is actionable as a mixed opinion. ... If the recitation purports to be all the facts on which the decision was based, then it affords the audience an opportunity to evaluate the opinion, including whether the recited facts were sufficient to warrant it.
Defendant argues that the Statements are protected as opinion. While he concedes that each of the Statements did not fully recite the facts upon he based his opinion, he argues that because they were published to an audience already familiar with the relevant facts, republication to the entirety of the facts was not required. The Court notes Defendant's argument, but finds it unnecessary to decide this Motion on those grounds, as the Court will grant summary judgment in favor of defendant because Plaintiff, a public figure, has failed to meet the heightened burden of proof required of a public figure. (16-17)
What is it that makes a postdoc in plant biology a public figure? As laid out in the ruling:
The first and foremost consideration in determining whether Plaintiff is a limited issue public figure, is Plaintiff's degree of voluntarily involvement in the public controversy. ... There is no question that Dr. Chandok has met this requirement. Scientific articles are inherently subject to robust criticism, and for good reason. Dr. Chandok has chosen and worked diligently in furtherance of a career where, through publication, entry into controversy and debate is expected and even required as a matter of course. She cannot be said to have entered the public arena haphazardly or otherwise in the absence of her own volition. Furthermore, Dr. Chandok published the scholarly papers at the core of this lawsuit and is credited as the lead author thereof, a designation that she has defended vehemently. In her own deposition, Dr. Chandok admits that she is well known within the plant biology community. ... controversy, with regard to a published article, extends not only to its direct subject matter and conclusion, but necessarily to the competence and integrity of the author. The real issue in the instant case falls to whether the plant biology community is sufficiently "public." This court finds that it is. (18-19)
(Bold emphases added.)
Since scientific discourse is expected to involve organized skepticism that extends to findings reported in the literature, practicing science is stepping into a public sphere where controversies will arise and be thrashed out. Scientists will have opinions about the findings others in this arena present (and on the methods used to generate those findings, and on whether the conclusions drawn were the most reasonable ones to draw in light of the data, etc., etc.) -- and the collective scientific discourse works better if scientists feel free to voice these opinions. Of course, it's good if their opinions have factual support, but if they don't, that's the sort of thing that other scientists in the discourse are likely to notice and point out.
While those training to be scientists have a pretty good idea that they will need to be able to receive criticism of their work and respond to it effectively, it's not clear that they understand that they will be embarking on a career that makes them public figures. Just what that entails may be fodder for a fruitful lab group discussion.
By the way, notice that as the judge is recognizing the link between how we judge a scientific result and how we judge the scientist who produced it, he is also acknowledging that competence, as much as integrity, may explain findings that don't hold up. (Being judged a less than competent scientist is not a good thing for one's scientific career, of course, but it's a different thing from being judged a cheater.)
As a public figure, the plaintiff here has to meet a higher burden of proof to establish the defamation claim:
Dr. Chandok has willfully interjected herself into a public controversy by way of creating the very subject of the controversy, and the controversy and community are sufficiently public to invoke the constitutional protection of free speech. In few other spheres is the need for a free marketplace of ideas as indispensable to the very operation of the endeavor as it is to scientific research. The public good would be ill-served by the interjection of such a murky field of law. This Court finds that Dr. Chandok is a limited issue public figure and that Plaintiff must establish the additional elements of 1) falsity with convincing clarity, 2) actual malice with convincing clarity, and, beyond a preponderance of the evidence, 3) some degree of fault. (19-20)
In other words, since the law recognizes scientists as public figures here, the question shifts from whether Klessig can prove that his claims about Chandok's findings and her integrity are true, to whether Chandok can prove that they are false. Moreover, regardless of the harm these statements might actually have caused for Chandok, to establish defamation here she would need to establish that Klessig made them with malice.
Of course, "malice" has a technical meaning in this legal context:
Actual malice is defined as knowledge of falsity or reckless disregard for the truth. ... Plaintiff failed to establish with convincing clarity that Defendant was aware of the falsity of any Statement, or that he recklessly disregarded the truth or falsity of any Statement.18 Each of the twenty-three Statements somehow references the inability of numerous scientists to duplicate Plaintiff's result and the implications thereof. As evidence that Defendant knew the Statements were false and acted with actual malice in publishing them, Plaintiff cites a letter from Defendant to the INS, in which Defendant admitted that the postdoctoral students assigned to attempt to replicate Plaintiff's results were inexperienced and initially did not follow the correct protocol. A conclusion that Defendant had actual malice is not warranted. (20)
Given this definition of malice, Chandok would have to go further than arguing that Klessig's claims assert misconduct without proving misconduct. Rather, to demonstrate malice she would have to prove that Klessig knew that Chandok had not committed misconduct (or, perhaps, that he was willfully ignoring evidence that would clear her of misconduct).
That's a pretty high bar.
Indeed, here the judge seems to make much of the opportunities Chandok had (but did not use) to present positive evidence that her original findings were reliable. For example, in Footnote 18 of the ruling, Judge Hood writes:
In fact, Plaintiff never contends that Defendant's comments that numerous other scientists were unable to duplicate Plaintiff's results are false. Plaintiff does not appear to take issue with the factual portions of the Statements, only with the veracity of Defendant's conclusions as to the implications of those facts - that if numerous other scientists could not replicate the results, the original results must have been fabricated or falsified. (20)
Once again, we see that the questions of the reliability of the results and the reliability of the researcher who produced them are separable but connected. Chandok's legal claim apparently did not dispute that the efforts to reproduce her results failed. If we stipulate that these efforts failed, what can we conclude? Either that Chandok's original results were flawed (whether due to her competence or her integrity), or that the attempts at replication were flawed (whether due to the competence or the integrity of the researchers mounting these attempts), or that some unforeseen obstacle to replication (like a poorly controlled parameter in the experimental system) has reared its head. While concluding from the failed attempts at replication that Chandok fabricated or falsified her results is not the only conclusion one could draw, it is a conclusion that fits with the available facts. Moreover, if Chandok were confident that her results were reproducible, it would be at least a little puzzling that she did not offer her assistance in the efforts to reproduce them.
In this scientific controversy in a public arena, Klessig offered his opinions on the basis of the facts he had -- including the fact that three separate researchers in his lab who attempted to reproduce Chandok's work could not. If Chandok disputes the reliability of the data offered by those researchers (whether because she believes that they were not competent to conduct the work or because she doubts their integrity), by her own lights would saying so amount to defamation?
Whatever Chandok might say, the judge felt evidence for malice was lacking:
The Statements were based on data provided by three recent doctoral graduates, and made after investigation by three senior researchers following a standardized protocol. Plaintiff argues, in a somewhat conclusory manner, that Defendant's ill-will prompted him to make statements with knowing falsity. However, Plaintiff has not alleged any such motive for the list of individuals who agreed with Defendant's assessment. It is not a reasonable inference that the existence of data was substantially false or that Dr. Klessig knew that it was false, or certainly that the references to such data were made with reckless disregard for the truth. (21)
Given that Chandok did not provide the requested help as far as reproducing her results, her basis for questioning the veracity of the failed attempts to reproduce them is shaky. And, it stretches credulity that the other researchers would blow the experiments in an attempt to help Klessig smear Chandok -- especially given that, conceivably, researchers in other labs might also attempt to reproduce these results and, if successful, end up doubting the competence or integrity of Klessig and his associates.
[W]hile Plaintiff attempts to paint Defendant's actions in reporting the failure of the replication studies and going about the business of retracting the Cell paper as evidencing Defendant's actual malice toward Plaintiff, those actions do not provide clearly convincing evidence of actual malice. Defendant reported the discrepancies and the failure to reproduce the results, as he was required to do. He invited Plaintiff back to BTI to attempt to replicate the results herself or to more thoroughly explain how she arrived at her results, something her sparse lab notebooks failed to do. Defendant gave Plaintiff every opportunity to help explain the inability of other scientists to duplicate her work, efforts that are far from a clearly convincing showing of actual malice. (21-22)
I think there is still room to disagree about what degree of data-supported confidence one ought to achieve before reporting a result in the scientific literature, or before retracting a result you've published, or before asserting that your coauthor probably committed misconduct in generating the finding you reported. In science, the link between assertions and evidence is supposed to be very strong. However, what the ruling in Chandok v. Klessig makes clear is that scientific discourse is rife with public disagreements about what the facts are and about what conclusions they support. Those who participate in scientific discourse cannot count on being shielded from disagreement, nor can they assert a right to remove their results, their competence in generating those results, or their integrity as scientists from the topics about which other scientists might dispute.
Drawing conclusions from too little evidence -- including conclusions about a fellow scientist's integrity -- is not an especially good thing to do. However, it seems possible that this is something better addressed through more speech, not less. And in any event, Judge Hood's ruling here suggests that legal actions to restrict scientists' speech in this sort of controversy ought not, and will not, be successful.
- Log in to post comments
Nothing to add, I just wanted to say I appreciated this analysis.
Fascinating.
Thank you for your highly informational dissection of this case. It is a very positive step to see someone with obvious expertise in research ethics pay attention to specific cases of scientific misconduct. Especially since you have paid such careful attention to the gray areas of unknown intention and experimental variance.
Thanks for the posts on this...
I'm quite impressed with this judge. Those are pretty much the right issues and the ruling is downright reasonable.
I'd like to note one thing though:
Chandok had one obvious recourse which would flip the tables completely... just help any lab (Klessig's included) replicate the results. If she had a compelling reason why she couldn't return to BTI, she could have at least clarified the protocols.
On that note, even if the original results were not falsified, her work was obviously incompetent to an extent. Part of publishing a paper is providing sufficient details of the methods that they are reproducable! (I'm assuming writing up those methods was her job since she did the experiment and was first author after all.) At the very least, supplemental material or a correction to clarify the methods was in order.
Finally, we have a pretty well established culture regarding authorship of papers. The first author is responsible for the overall paper... it is *their* paper. Other authors may write sections, contribute results, and of course review/edit. They have responsibility too of course, but it isn't the same as the first author. Fields differ in how much contribution/responsibility is associated with being an 'et al' and what the order should be, but first author is pretty conserved.
I'm with MRW; thanks for all your work on this. My ex-wife worked in a research lab while we were married, and similar issues of 'professional communications' being less than professionally accomplished came up. It made me realize how much soap-opera drama was involved not simply in the gossip at work, but the actual work at work. So I suspected all along that this was a matter of "I'll sue him for pointing out how incompetent I am," but I was also concerned this could be just another PI screwing over a post-doc to cover his own ass. Gratifying to see the Judge saw it essentially the same way, and was able to sort out the facts and laws involved.
Great write-up. This case has been an eye-opener regarding the way that science is conducted, but I think the judge got the key issues right.
One question: you say
Does 'public' in this context have the common-usage meaning, or does it mean that Chandok was a public figure in the limited sphere where her reputation would be damaged by this business? I can see the logic in an argument that she had made herself a public figure in the only sphere where anyone would be likely to hear of it, so condition (3) of the defamation definition would apply in that area only. But someone who actually knows something about US law would have to enlighten me.
One very important topic that I haven't really seen in your discussion of this case so far is whether or not these claims of unreproducibility, unreliability, and falsification of experiments and data should be decided by the legal system at all. The judge seems to have made the right call here, but why was a judge involved in the first place? Shouldn't scientists, of all people, be able to sort out these claims without getting lawyers involved? On a related topic, I'm curious about why Chandok filed this lawsuit, and when. How long did she try to settle these claims through scientific institutions and procedures before turning to the law? Did she feel like she had absolutely no recourse left within science, and only reluctant filed suit, or (on the other extreme) did she feel that going to court is just what people do when they have disagreements?
None of which is to say that your discussion hasn't been insightful. :-)
One wonders if there were other requests that didn't come up in the legal proceedings (although if there were, I can't imagine why Klessig wouldn't want that fact on record).
Maybe the other communications weren't recorded, and therefore were difficult to introduce?
Additionally, without the original communication, the level of "threat" is difficult to ascertain. There's a world of difference between the formal description by the judge, and say an email where Klessig says something along the lines of "We're in trouble here -- if I can't find some way to reproduce your data, I won't be able to stop an investigation or give you that reference letter for the visa. My ass will be on the line as well."
Hard to know -- particularly without all the other data that doesn't go into a court case, and especially make it into the judges ruling.
Just because the summary of the facts presented in the court's ruling omits certain things, does not mean they were not put into evidence by the parties to the suit.
Can you post the pdf of the judge's decision?