Effect Measure

There is a class of legal cases that are so blatant lawyers call them Oh My God cases, you know, the kind when you see the facts you say, “Oh my God” (NB: don’t give me grief because I’m an atheist. I’m allowed to use colloquial phrases that have their origins in myth and superstition).

Back to the subject. I’m a journal editor and also a frequent peer reviewer of scientific articles for other journals (I’m procrastinating reviewing three of them by writing this post). And in that context, I’d call this story an Oh my God story:

A peer reviewer leaked a paper due to appear in The New England Journal of Medicine to the manufacturer of a drug criticized in the manuscript, according to a news article in Nature.

The reviewer, Steven M. Haffner, a professor of internal medicine at the University of Texas Health Science Center at San Antonio, broke the journal’s confidentiality rules by faxing a copy of a review of studies on the diabetes drug Avandia to a colleague at GlaxoSmithKline, the pharmaceutical company. Dr. Haffner has received consulting fees and speaker’s honoraria from the company.

The review article linked taking Avandia to an increased risk of heart attack. When it appeared online in the journal last May, Glaxo’s stock fell by 13 percent, according to Nature. (Lila Guterman, Chronicle for Higher Education)

Dr. Haffner is reportedly shamefaced and admits to “bad judgment.” Yes, I’d say that all right. I’ll also admit to feeling a little sorry for him. But at the same time, all I can say is, “Oh my God!”

Comments

#1 Russell
January 31, 2008

Ah, but did he short Glaxo’s stock?
#2 Shannon
January 31, 2008

Would that this was the only example. I will try to refrain from posting on this thread as most know of my penchant for high dudgeon when this subject is raised.
#3 Shannon
January 31, 2008

Oh the hell I will.
http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1123081
[snip]
They found that over 10% (56/548) of all new chemical entities approved from 1975 to 1999 acquired a new �black box warning��indicating a serious adverse reaction that may result in death or serious injury�or were withdrawn from the market. Sixteen drugs (3%) were withdrawn and 45 (8.2%) required one or more new black box warnings.

Half of the withdrawals occurred during the first two years after the drug’s introduction, and half of the new black box warnings occurred during the first seven years. Thus in 50% of the drugs an adverse reaction went undetected until relatively late in the course of a drug’s appearance.
[snip]

The peer review process is only part of the flawed system. What happens after a drug is approved? There are just 55 people in the drug surveillance division. Many doctors are unaware of procedures for alerting the F.D.A. if they see a serious side-effect with a drug. http://query.nytimes.com/gst/fullpage.html?res=9E01E0DA113DF934A35753C1A961958260

The F.D.A. needs more money and more clout. Or/and, establish an independent drug monitoring agency, paid for by the pharmaceutical industry. Their responsibility would be to assess the safety of drugs after they are on the market.
The peer review process needs more clout as well. If there is a serious drawback to a drug then it needs to be clearly stated form the outset. I realize all drugs have the potential for unwanted effects. But, we need to examine how drugs like Acutain, Vioxx, fen-phen all made it through trials.
#4 Orac
January 31, 2008

There is a class of legal cases that are so blatant lawyers call them Oh My God cases, you know, the kind when you see the facts you say, “Oh my God”

I think you might mean the legal termres ipsa loquitur, which is commonly translated “the thing speaks for itself.” It’s usually used to refer to cases where the facts are so blatantly obvious and beyond doubt that there is nothing left but to determine liability.
#5 revere
January 31, 2008

No. I mean, “Oh my God.” Res ipsa is used for many things, not usually the one you cite. It does imply obviousness but not necessarily of the kind I am suggesting. I am speaking colloquially but using terms many litigators use.
#6 Marissa
January 31, 2008

I hope at the very least the journal will ban him fom reviewing any more papers.
#7 Sven DiMilo
January 31, 2008

What a dope (ha ha! see what I dod there?)
This makes me feel somehow smug about the otherwise depressing fact that all of the papers I write and review are of interest to about 15 people in the world and have no commercial relevance whatsoever.
#8 Caledonian
January 31, 2008

Are there any grounds for the journal to sue the reviewer in question? (If not, would the courts say that they weren’t injured and so have no grounds? Or are the confidentiality rules not a contract between the journal and the reviewers?)
#9 revere
January 31, 2008

Caledonian: No grounds I can think of. What are the (monetary) damages of the journal? You can’t sue just because you don’t like something or someone. You have to have damages. And the confidentiality agreement isn’t a contract as far as I understand it. This guy is being hurt plenty by the bad publicity.
#10 jen_m
January 31, 2008

IANAL, but I’m pretty sure the journal won’t be suing anyone.

About “banning” and “suing” – reviewers are invited by the journal, and reviewers are volunteers. It’s generally understood that there’s a risk of being reviewed negatively or having a paper held up in review by a dishonorable competitor, so at the time of submission authors may request not to be reviewed by some people, and at some journals authors are even invited to suggest potential reviewers. The journal makes the final choice of reviewer (and the process is usually confidential on both sides) so there’s no need to “ban” Dr. Haffner – this blow to his reputation will probably put a tidy end to most of his reviewing activities.

Ultimately, though, this exchange is not contractually based, and it would probably be chilling to the whole peer review system if journals took reviewers to court, given it’s a volunteer-driven system.
#11 Caledonian
January 31, 2008

What are the (monetary) damages of the journal?

Dunno. I was thinking that his actions have besmirched the reputation of the journal, and possibly dissuaded future contributors from publishing their articles in the journal, leading to decreased value.

I guess with all of the social damage this guy is taking, suing to damage him monetarily would probably be redundant. Who’d want to associate with him after this stunt?
#12 jen_m
February 1, 2008

Oh, NEJM is still pretty darn prestigious, and not very likely to suffer much because of one crooked reviewer.
#13 PFT
February 1, 2008

Obviously, the writer of the paper could care less that GSK saw their paper early, and NEJM may be a bit peeved at the confidentiality agreement being broken, but thats all.

All it amounted to was GSK got a heads up and may have accelerated the issuance of an interim study that supposedly was underway and was published 2 weeks later. Not a bad thing, or was it.

What prompted the letter?. Concern over GSK profiting from the knowledge by insider trading? I doubt it, the stock did not drop until the paper was published, although they could have shorted the stock, as could the reviewers and writer of the paper.

Looking at the letter it appears they were looking into the interim report. Perhaps wanting to know if GSK had prior knowledge of the papers results but were sitting on these interim results, or perhaps even made them up? I think this is more likely.

How it got out that the paper was given to GSK is interesting. A whistle blower from within GSK? Could he/she also have blown the whistle on the interim study that was released?

So Haffner sending the paper to GSK is a red herring IMHO.
Getting caught and publicly named may have been embarassing, but thats it. Of course, if he was linked to that interim study he might be a bit uncomfortable, especially if there was any funny business.

The interim study might be the “Oh My God” if it was being delayed, or massaged after the paper was received, and we really can only speculate at this point, but I know where I would put my money.

The FDA is a legal cover for agribusiness and pharmaceutical companies with their stamp of approval. Making them bigger or giving them more clout is useless unless you can find a way to prevent the people working there from being corrupted. The best and brightest at FDA do not stay that long, they put in some time, build up the right kind of rep, and then go to work for a nice piece of change for those they “regulated” while at FDA. Some working at the pharmaceuticals and agribusiness companies, and with their help, get appointed to the FDA to work from within for a time, and then return to collect their bonus for a job well done. This is true of all the agencies getting paid to protect you.
#14 revere
February 1, 2008

PFT: From the journal/reviewer side it’s a big deal, however. Reviewers are privy to information of use or interest to other scientists and that means a reviewer can appropriate the work of others without giving credit for their own benefit. This was a serious breach of ethics on the science side, which is why the perpetrator is so humiliated. He was caught with his hand in the cookie jar. Probably stealing a coupe of pennies out of a blind man’s cup isn’t going to make much difference to the blind man but the crime speaks for itself in the eyes of most people.
#15 traumatized
February 1, 2008

Reviewers don’t have to reveal conflicts of interest the way authors do? Why was this guy reviewing the paper in the first place?
#16 revere
February 1, 2008

traumatized: Most journals (including mine) and I believe the NEJM do require a statement of competing interests on the part of reviewers. That statement doesn’t preclude reviewing (it is very hard to get reviewers; it is voluntary and hard work with no pay) but the statement should be taken into account by the editors in judging the review. Our journal has open reviews so the interests of reviewers can also be judged by readers if they wish to look at the reviews but most non open access journals don’t have open review.

Peer review can be very valuable but it has many pitfalls. Many reviews are not well and conscientiously done, many reviewers let their prejudices get in the way and even with good reviews errors and flaws slip through. As a colleague once put it, “Real peer review starts after publication.”
#17 Farmer
February 1, 2008

“Real peer review starts after publication.”

Just like ‘real’ drug safety trials start after the drug is approved and is used in a large population.
#18 revere
February 1, 2008

Farmer: Not really. Science works by having its results replicated and examined by the scientific community. Drugs are usually not examined further unless they cause a problem. Moreover certain adverse effects are not detectable by clinical trials because they are rare but still show up if given to tens of millions of people. So it is quite a different thing. For a scientific paper either the methods, results, data, etc., are fine or there is some fatal flaw. For drugs the adverse effects are stochastic. I’m no friend of Big Pharma (on the contrary; I think this is a greedy, pernicious and often socially harmful industry) but the technical problem of detecting uncommon adverse events prior to marketing to a hundred million potential users is not solved by a premarket clinical trial.
#19 Adam
February 1, 2008

If folks at SKG saw the paper and as a result sold stock (i.e., used insider information not available to the investing community), the “leaker” would likely be culpable for SEC prosecution. Hope he’s both “shamefaced” AND has a good lawyer.
#20 pauls lane
February 1, 2008

I have to agree with revere (a first I think) except the part about Big Pharma being greedy, pernicious, and socially harmful. Except for that part he’s pretty much dead on. I worked for a IT security firm that performed security testing, studies, etc. on all kinds of IT equipment and software. We had testers, reviewers, and reviewers of the reviewers. Then the US Government reviewed everything. Our reports were never released to anyone except the owner of the product and the Government. The owner was trying to sell to the Government, the Government wouldn’t buy unless the product went through this rigorous security examination. I can’t begin to tell you the comical situations that arose when we all met to discuss a product and the testing with me in the middle between the gov and the prod owner. Anyway, even after all of that flaws still found their way through the system. Every step in the process was documented, repeated, and documented again. Then a different team would repeat and review and document. Then a 3rd team would review. Then the Government would review. The problem was that a review team on one project is the lead on another. So sometimes things didn’t recieve the full attention of the review teams. I imagine all peer reviews processes have the same type things happening.
#21 Dr. Free-Ride
February 1, 2008

Pseudo-trackback
#22 outlier
February 3, 2008

I have to say, I’m not at all sure what’s supposed to be OMG! here (besides the breach of confidentiality). GSK found out about a study a few weeks or months before they would have otherwise? If there’s more to it than that, you need to make it explicit.
#23 revere
February 3, 2008

outlier: There are just some things everyone who review knows you should not do. This is one of them. Maybe there’s no real harm to anyone of serving ham at an Orthodox Bar Mitzvah but you don’t do it if you are a caterer.