The headlines bring news of another scientist (this time a physician-scientist) caught committing fraud, rather than science. This story is of interest in part because of the scale of the deception — not a paper or two, but perhaps dozens — and in part because the scientist’s area of research, the treatment of pain, strikes a nerve with many non-scientists whose medical treatment may have been (mis-)informed by the fraudulent results.

From Anesthesiology News:

Scott S. Reuben, MD, of Baystate Medical Center in Springfield, Mass., a pioneer in the area of multimodal analgesia, is said to have fabricated his results in at least 21, and perhaps many more, articles dating back to 1996. The confirmed articles were published in Anesthesiology, Anesthesia and Analgesia, the Journal of Clinical Anesthesia and other titles, which have retracted the papers or will soon do so, according to people familiar with the scandal (see list). The journals stressed that Dr. Reuben’s co-authors on those papers have not been accused of wrongdoing.

In addition to allegedly falsifying data, Dr. Reuben seems to have committed publishing forgery. Evan Ekman, MD, an orthopedic surgeon in Columbia, S.C., said his name appeared as a co-author on at least two of the retracted papers, despite his having had no hand in the manuscripts. “My names were forgeries on the documents,” Dr. Ekman told Anesthesiology News.

Fabrication, you’ll recall, is making up data rather than actually collecting them. Science is an activity that attempts to build a body of reliable knowledge about the world — which means that its claims need to be supported with actual data, not data from someone’s imagination.

Apparently, Dr. Reuben compounded his lies about the data he collected (or in this case, didn’t collect) by lying about which other scientists were responsible for the work he reported. In theory, all the authors of a scientific paper are putting up their credibility to warrant the credibility of the results reported in the paper. In the case of an “author” who was put on the paper without his knowledge, his name (and track record within his scientific community) may make the reported results look more credible — but since he didn’t decide to be an author (and maybe didn’t even participate in the research), this added credibility is illusory.

I’d be interested to see, for all of Reuben’s authors on the retracted papers:

• the portion of the reported results for which the author feels personally comfortable vouching
• whether the author was aware of being an author when the manuscript was prepared and submitted
• whether the author actually exists

I’m guessing if you’re willing to make up data out of whole cloth, inventing co-authors out of whole cloth isn’t a tremendous leap of depravity.

The retractions came after an internal investigation by Baystate turned up evidence of widespread fraud in Dr. Reuben’s research. Jane Albert, a spokeswoman for Baystate, said the inquiry was undertaken after an internal reviewer at the medical center had raised questions last year. Ms. Albert said the hospital’s investigation raised “no allegations concerning any patient care. This was focused on academic integrity.” …

Rumors of a problem with Dr. Reuben’s research have been circulating among academic anesthesiologists for a year, according to people familiar with the matter.

“Interestingly, when you look at Scott’s output over the last 15 years, he never had a negative study,” said one colleague, who spoke on the condition of anonymity. “In fact, they were all very robust results–where others had failed to show much difference. I just don’t understand why anyone would do this or how anyone could pull this off for so long.”

The titles of Reuben’s papers suggest to me that he was conducting clinical (rather than laboratory) research. I’m guessing that this makes the possibility that Reuben had “magic hands” an unlikely explanation — controlling human patients precisely is a much harder thing than setting up your lab apparatus and making your reagents much better than anyone else. Unless the design of his clinical trials was significantly different from those used by other researchers in the field, you’d expect that at least some of those other researchers would be able to find similar results. Indeed, especially since his published papers presumably laid out his experimental design, you’d expect that other researchers in the field would try to use that experimental design to find similar results.

But only Reuben seemed to be obtaining these robust results. Since we’re talking about research that was used to inform treatment decisions — where many different physicians were administering the treatment — this should have set off alarm bells. It did set off alarm bells, of course, but the question is why it took so long. And I’m not sure there’s a neat answer to the question of how long the scientific community ought to wait for one researcher’s “robust result” to be replicated by others. This is one of those situations where scientists need to work out how to balance their excitement about potentially useful new results, and their default trust of the member of their professional community who produced the results, with their skepticism.

What’s particularly surprising given the dimensions of the case, Ms. [Josephine] Johnston [an attorney specializing in research integrity at the Hastings Center] said, is that Dr. Reuben’s research managed to raise no alarms among peer reviewers. However, she added, “the peer review system can only do so much. Trust is a major component of the academic world. It’s backed up by the implication that your reputation will be destroyed if you violate that trust.”

It seems safe to assume, at this point, that Dr. Reuben’s reputation has been destroyed.

On what peer review ought to have uncovered, we should recall that what actually occurs in peer review may not match what non-scientists imagine. Peer reviewers are not, in most cases, trying to replicate the experiments reported, nor are they even likely to reread all the references cited. Rather, they’re looking at the manuscript to see if there’s a coherent scientific argument, if the data included support that argument, if the interpretation of results makes sense against the background of the other scientific knowledge in this scientific area, and whether there are any obvious mistakes.

Blogging about this case at Respectful Insolence, Orac also wonders how Reuben was able to make up data in so many papers for so many years:

Whenever I see an example of fraud like this, I wonder: How could he get away with, in essence, making it up for so long? Clinical trials are complex; they inevitably involve statisticians who analyze the data. Many journals these days will not even consider publishing the results of a clinical trial without a biostatistician listed among the authors.

You might expect that a problem with the statistical analysis of the data would be the kind of thing that would jump out at peer reviewers. However, that assumes that the folks reviewing manuscripts for anesthesiology journals are biostatisticians (rather than, say, anesthesiologists). I don’t know if that’s a good assumption. Also, I don’t think we can rule out the possibility that each of these bad papers had a biostatistician listed among the authors. Whether the biostatistician listed among the authors knew that he or she was an author is another question.

So, how did Reuben’s fabrication come to light? According to the New York Times:

Dr. Reuben’s activities were spotted by Baystate after questions were raised about two study abstracts that he filed last spring, Ms. Albert said. The health system determined that he had not received approval to conduct human research, Ms. Albert said.

This, of course, raises another question: what was the status of those clinical trials whose “results” were reported in the twenty-odd retracted papers? Did these trials with human subjects not happen at all? Were they conducted as described, except that real data were not collected and/or reported from them? Given the logistics of conducting clinical trials, shouldn’t there have been some earlier clues that something was not on the level in Reuben’s research?

And, given that lack of IRB approval for two abstracts provided the smoking gun that helped expose Reuben’s fraud, I have to wonder whether the earlier studies Reuben published had protocols submitted to, and approved by, Baystate Medical Center’s IRB. I hope that IRB is now re-examining the level of oversight it brings to research conducted at Baystate Medical Center. Even though the Baystate Medical Center investigation of Reuben “was focused on academic integrity” and didn’t raise “allegations concerning any patient care,” it’s reasonable to worry that the participants in Reuben’s clinical trials (if such participants were not also fabricated) were not actually undertaking the risks they did in the service of the production of reliable knowledge about pain treatment. That constitutes a harm to them.

Where did Reuben get the funding to conduct his research? It may not surprise you to learn that a good portion of this funding came from a pharmaceutical company. Anesthesiology News reports:

A cornerstone of Dr. Reuben’s approach has been the use of the selective cyclooxygenase-2 inhibitor celecoxib (Celebrex) and the neuropathic pain agent pregabalin (Lyrica), both manufactured by Pfizer. Dr. Reuben has received research grants from the company and is a member of its speakers’ bureau. However, a source told Anesthesiology News that Pfizer recently alerted its speakers to remove any reference to Dr. Reuben’s data from their presentations. Pfizer was unable to comment by the time this article went to press. The company has not been accused of wrongdoing in the matter.

The New York Times notes:

The drug giant Pfizer underwrote much of Dr. Reuben’s research from 2002 to 2007. Many of his trials found that Celebrex and Lyrica, Pfizer drugs, were effective against postoperative pain.

“Independent clinical research advances disease treatments and improves the lives of patients,” said Raymond F. Kerins Jr., a Pfizer spokesman. “As part of such research, we count on independent researchers to be truthful and motivated by a desire to advance care for patients. It is very disappointing to learn about Dr. Scott Reuben’s alleged actions.”

Drug companies routinely hire community physicians to conduct studies of already-approved medicines. In some cases, prosecutors have charged companies with underwriting studies of little scientific merit in hopes of persuading doctors to prescribe the medicines more often.

“When researchers are beholden to companies for much of their income, there is an incredible tendency to get results that are favorable to the company,” said Dr. Jerome Kassirer, a former editor of The New England Journal of Medicine and the author of a book about conflicts of interest.

This is one of those situations that makes the slogan “trust, but verify” seem like really good advice.

Even without imputing evil intent, there is reason to believe pharmaceutical companies aren’t just interested in what the truth is in clinical trials of their drugs. They want the compounds they have developed (and patented) to work, and to be purchased. This is how you stay in business. Pfizer is not a disinterested party here.

But there’s a big difference (at least from a public relations point of view) between subtle bias — the sort of thing you might plausibly dispute is even operating — and a guy working on your funding who makes up results out of whole cloth to support the efficacy of your product. Pfizer wants as much distance from Reuben as possible, stat. I’m guessing that some people at Pfizer who help hire community physicians to conduct independent drug trials are probably re-examining the level of oversight — or at least of vetting of the researchers — they should bring to such arrangements in the future.

I’d be interested to know when the scrubbing of Reuben’s data from Pfizer speakers’ presentations was initiated. Was it prompted by the launch Baystate investigation? By the withdrawal of the papers from the journals at the conclusion of the investigation? Or did Pfizer notice something fishy and alert Baystate to a potential problem?

For all that the influence of pharmaceutical companies in research raises suspicions, I wonder, too, the extent to which Reuben’s results (mow revealed to be fraudulent) set the direction for further drug development at Pfizer and its competitors.

Of course, the impact of Reuben’s fraud is bound to be felt well beyond Pfizer’s R&D department. The New York Times reports:

Dr. Steve Shafer, the editor in chief of Anesthesia & Analgesia, which published many of the papers, said he was considering withdrawing any study in which Dr. Reuben served a pivotal role.

“He was one of the most prolific investigators in the area of postoperative pain management,” Dr. Shafer said. His fraud “sets back our knowledge in the field tremendously.”

Anesthesiology News quotes Dr. Shafer further:

“We are left with a large hole in our understanding of this field. There are substantial tendrils from this body of work that reach throughout the discipline of postoperative pain management,” Dr. Shafer said. “Those tendrils mean that almost every aspect will need to be carefully thought through. What do we still believe to be true? Do the conclusions hold up to scrutiny?”

Dr. Shafer said that although he still believes “philosophically” in multimodal analgesia, he can no longer be absolutely certain of its benefits without confirmation from future studies.

There is a reason, dear readers, that philosophers don’t have the authority to prescribe drugs. Believing in something philosophically is well and good, but science-based medicine requires that beliefs on which treatment decisions are based be grounded in empirical evidence.

Now, those working in the field of anesthesiology and relying on the goodness of results reported in that field’s body of scientific literature have a problem. Not only have more than 20 studies that they took as reliable been revealed to be untrustworthy, but there also seems to be good reason to regard the scientist who generated these studies as untrustworthy — making his other publications in the literature suspect as well. (Depending on what kind of involvement one judges likely from Reuben’s co-authors in his fraudulent papers, these co-authors might also be judged presumptively untrustworthy, leaving an even bigger hole in the reliable literature.)

This knowledge-hole is non-trivial. It is a problem for researchers in the field who trusted the work reported by Reuben and his co-authors and who may have spent time trying to build on it. It is a problem for health care providers who based their treatment of patients on the fraudulent results. It is a problem for those charged with teaching medical students about pain management. The fraudulent research may even have informed how hospitals, insurance companies, and regulatory agencies (like the DEA) conceived of proper pain management.

To recover from this fraud, the scientists in this field need to track down money for new research to generate reliable (rather than fraudulent) knowledge. The money spent to fund Reuben’s original research is now long gone. Will federal funding agencies be able to provide these funds? Will drug companies? Or will we have to live with less knowledge about pain treatment then we thought we had?

Finally, the New York Times provides the obligatory quote from Reuben’s attorney:

Paul Cirel, a lawyer for Dr. Reuben, said that he could not discuss the case because Baystate had investigated it as part of a confidential peer-review process. Baystate officials “were aware of extenuating circumstances,” Mr. Cirel said.

I wonder what could count as an “extenuating circumstance” in a case like this, where you’ve fabricated the results of more than 20 publications. “My client was sick the day of the mandatory ethics lecture,” perhaps?