Posted by At the request of femalechemist, I’m going to revisit the Sames/Sezen controversy. You’ll recall that Dalibor Sames, a professor at Columbia University, retracted seven papers on which he was senior author. Bengu Sezen, also an author on each of the retracted papers and a graduate of the Sames lab, performed the experiments in question.
Sames says he retracted the papers because the current members of his lab could not reproduce the original findings. Sezen says that the experiments reported worked for her and for other experimenters in the Sames lab. Moreover, she says that Sames did not contact her about any problems reproducing the results, and that he asked the journals to retract the papers without letting her know he was doing so.
I am not now, nor was I ever, an organic chemist, so I’m not going to try to do the experiments myself (repeatedly, with appropriate consultation of the people who developed the original protocols) to see who’s right. That’s not the kind of light I can shed on this case. However, I can break down the key issues at play here:
1. Are the experiments reproducible?
This is the question of interest to others working in organic chemistry. Do these syntheses work or don’t they? To get a definitive answer to this question, people outside the Sames lab will have to go to the lab and try it. (Why people outside the Sames lab? Because all the principals in this scandal — and their close associates — have too much at stake in the outcome of these experiments to have a reasonable chance of being unbiased.)
Of course, they will want to follow not only the “materials and methods” from the papers in question, but they will want to get tips for running the experiment from the person who says she has made it work. A number of scientists, in other contexts, have noted that there are some parts of experimental protocols that may seem dispensable, until you dispense with them and can no longer get the desired outcome. At least some of the attempts at replication ought to take these “pigeon dance” steps seriously.
2. Did Sezen really get the experiments to work in the first place (or at least, did she really believe she got them to work)?
The fact that Sezen is on record saying that she would come back to the Sames lab to assist them in getting the experiment to work suggests that she either really believes the experiments can be accomplished, or she’s an incredibly gutsy liar. Lying in science is a lot harder when you’ve got all kinds of attention on you (say, from press coverage of your dispute with your old advisor and co-author). If Sezen were still willing to step up and work with other scientists to reproduce the reported results, I’d be inclined to think that in her own mind, she was convinced of the goodness of her original results.
Do note that one can be convinced of the goodness of one’s own results and still turn out to be wrong. If one has made a concerted effort to design and conduct careful, controlled experiments (and to collect enough results to be sure they were not a fluke and that the results are not biased), this is what we call an honest mistake.
3. Did Sames have good reason to believe the experiments worked back when he authored those papers? Does he have good reason to believe they don’t work now?
Presumably, if Sames was willing to trust Sezen to do the experiments, then to send out the results with his own name on the manuscript in the senior author slot, he believed in the results. One would hope that, in his capacity as an advisor, he would push his graduate students to be sufficiently skeptical of their own results — and that he would make sure he was walked through the entire chain of scientific evidence. If he was putting his name on the paper, he’d probably want to be at least as critical of the not-yet-published results as the referees at the journal would be, not to mention competitors after the fact.
Not only would he want good evidence to support the claimed results of the syntheses, but he’d also want something like good evidence to support his trust in Sezen. If her trustworthiness was ever in doubt, dealing with that before co-authoring papers with her would have been a much better idea.
It is possible that since those papers were published, Sames came face to face with certain facts that were enough to convince him that Sezen was not to be trusted. These would have to be facts beyond the difficulties his lab was having reproducing Sezen’s results, or else he would have to at least entertain the “honest mistake” hypothesis. Maybe someone is aware of the existence of such damning facts against Sezen, but I haven’t seen them mentioned in the news coverage. (That doesn’t mean they don’t exist, though.)
4. Why wouldn’t Sames tell Sezen there were problems with the recent attempts to replicate the experiments? Why wouldn’t he contact her before sending retractions to the journals?
This, to me, is the most puzzling question. The possibilities range all the way from Sames viewing Sezen as an irredeemable liar, to Sames deciding for his own evil reasons to try to destroy Sezen’s reputation, to his not being able to locate her to communicate with her (even though reporters were able to locate Sezen fairly quickly as the story was breaking), with many other possibilities between these. What is clear is that there isn’t currently trust between the two parties. It is much less clear on which side the trust was breached, and whether it was breached on the basis of good evidence or an unsupported hunch (or vendetta).
Make no mistake: these retractions are not just about the reliability of the results reported in the retracted papers. They are also about the credibility of the authors of those papers. Sames’ retraction is also as good as an assertion that Sezen is not a reliable source of scientific information. The reported experimental results were supposed to live up to a certain standard of proof. What about the retractions’ implications for Sezen’s credibility — what standard of proof do they meet?
It’s worth noting that even without the attending scandal about leaving Sezen out of the loop (or allegedly not really making serious attmpts at replication), seven retractions would not leave Sames’ reputatioin pristine. At the least, they would mark him as a scientist willing to put results into the scientific literature before they had received sufficient scrutiny. That couldn’t help his future publications. (“Hmm, Sames has another synthesis. I wonder if this one is really worked out.”) On the other hand, I suppose you get a few more credibility points for retracting a result once you’ve discovered it’s mistaken rather than leaving it out there and hoping no one notices.
But this is not your standard difficult retraction. This is a retraction where not all the authors are acting in concert. This is a fight, and sadly, even if one party really is in the right and the other is in the wrong, neither comes out looking like someone you’d trust or with whom you’d want to collaborate.
5. So can you ever trust your collaborator, or should scientists all author their papers alone? Can advisors ever trust their graduate students, or graduate students their advisors?
The alternative to trusting other scientists, whether by way of collaborations or by consulting the scientific literature, is doing all the science you care about all by yourself. And if that’s your plan, you really have too much to do to be reading blogs. Scoot!
On the other hand, since scientists have some acquaintance with the idea of backing their beliefs with facts, it’s good to base your trust on facts, too. This would be easier if collaborators took the time to get to know something about the pieces of the project contributed by their collaborators, if PIs still got involved in conducting experiments themselves, if grad students worked up their data with their advisors at least some of the time rather than only delivering the finished product with a pretty bow on top. Yes, it’s possible for an accomplished liar to seem to provide all kinds of evidence of trustworthiness, but it takes a lot of work. Better to try to have good reasons for trusting people rather than to decide preemptively that no reliable evidence can be had.
If I were a PI right now, I’d want to have a conversation with my graduate students about the importance of honesty and trust within the group. I might look into ways that my grad students could attempt prepublication replications of each other’s results — or even step up myself to try to reproduce their results.
If I were a science graduate student right now, I’d want to have a discussion with my advisor about keeping me in the loop if results we’re certain of now — and publish — end up being less certain later. I’d want to talk about the possible career consequences of a retraction due to an honest mistake. I’d also want to have a discussion about what kind of evidence you need to see to be confident enough about a result to publish it — and whether there are strategies he or she uses to give tentative results one more run through the wringer.
Without trust, there is no science. Of course, we’re talking evidence-based trust, not gullibility. And I don’t see how we get that trust without scientists actually talking to each other.
August 3, 2006
August 3, 2006
August 4, 2006
August 5, 2006
August 8, 2006
December 18, 2008