Effect Measure

When the Wall Street Journal called attention to a claim that the Journal of the American Medical Association called a whistle-blower a “nobody” and a “nothing,” a claim JAMA denied, I didn’t know what to think. I was inclined to give JAMA the benefit of the doubt. Whatever dealings I’ve had (and they are few) with JAMA’s editor in chief, Dr. Catherine DeAngelis, she’s been pleasant and has a reputation for being a tough and intelligent editor. It sounded as if someone had gotten a little irritated and maybe said things in a way that wasn’t quite appropriate, but these things happen. But now what was just dumbass is being elevated to the level of policy.

First some background.

JAMA published a paper on Lexapro, an anti-depressant a year ago. Papers evaluating drugs have been a constant source of conflict of interest problems, since the studies are often bankrolled by the drug company. Jonathan Leo, a neuroanatomy professor at Lincoln Memorial University (very small; I never heard of it) wasn’t happy with it. For one thing, he said, it didn’t acknowledge that behavioral therapy worked as well as the drug. For another, the senior author hadn’t disclosed he had a relationship (of the money kind) with the maker of Lexpro, Forrester Labs. And he complained via a letter on the website of the BMJ (aka, the British Medical Journal). What happened next is contested. Here’s the Wall Street Journal’s account:

Leo says he received an angry call from JAMA executive deputy editor Phil Fontanarosa last week, shortly after Leo?s article was published on the BMJ Web site. ?He said, ?Who do you think you are,? ? says Leo. ?He then said, ?You are banned from JAMA for life. You will be sorry. Your school will be sorry. Your students will be sorry.? Fontanarosa referred a call for comment to a JAMA spokeswoman, who said Leo?s retelling of the conversation was ?inaccurate.? (WSJ, MARCH 13, 2009; no link)

Ewww. There’s more. The Editor in Chief, DeAngelis then called Leo’s superiors to pressure him to retract the BMJ piece. So Leo called DeAngelis directly to find out what it was she was so concerned about. He says she didn’t make any specific complaints, only acted “very upset.” Maybe he was exaggerating. But the WSJ also called DeAngelis and reported this:

In a conversation with us, DeAngelis was none too happy to be questioned about the dust-up with Leo.

?This guy is a nobody and a nothing? she said of Leo. ?He is trying to make a name for himself. Please call me about something important.? She added that Leo ?should be spending time with his students instead of doing this.?

When asked if she called his superiors and what she said to them, DeAngelis said ?it is none of your business.? She added that she did not threaten Leo or anyone at the school.

Ewww again.

Leo says he went to JAMA 5 months earlier with his concerns but nothing happened until the week after his disclosure at BMJ, when JAMA published a letter from the author fessing up to the conflict but denying the sponsors affected the work in any way. It appears the disclosure letter published by JAMA and Leo’s submission to BMJ “crossed in the mail” so that JAMA was indeed taking the appropriate action, even without Leo’s BMJ letter. So an unfortunate combination of circumstances combined with some bad personal interactions seemed to produce this unhappy outcome. Too bad for everyone concerned, but JAMA can take it. Or so most of us thought.

Instead of letting what sounds like some bad behavior just die a quiet death, the two JAMA editors (Editor in Chief DeAngelis and Deputy Fontarosa) decided to make their bad behavior journal policy:

The Journal of the American Medical Association, in an editorial published on Friday, has warned that anyone raising a conflict-of-interest complaint about one of its authors should do so in private to the editors, without telling any outsiders.

[snip]

In its editorial, JAMA affirmed the need to guard against conflicts of interest. Yet JAMA said that, in the future, anyone suspecting a conflict involving one of its authors should tell JAMA and ?should not reveal this information to third parties or the media while the investigation is under way.? JAMA said it could be trusted to handle the matter fairly. (Chronicle for Higher Education)

The Editorial seems to confirm Leo’s allegation that he was told that henceforth he shouldn’t expect to publish in JAMA, although a decidedly more benign version is given:

Leo also was informed that, if his actions represented his apparent lack of confidence in and regard for JAMA, he certainly should not plan to submit future manuscripts or letters for publication. (Editorial, JAMA)

Ewww, a third time.

The record is muddy, so we won’t pass judgment on what happened here. But JAMA’s policy response is stupid and smacks of arrogance. JAMA’s Editor was upset that Leo contacted the press instead of letting the journal pursue an investigation on its own. It’s a point, but in the circumstances a weak one. The value of the press is to shine a light on a situation that has been kept in the dark, pernicious conflicts of interest in medical science. JAMA may say they have mended their ways and we should just trust them, but trust has to be earned, and the track record of major journals like JAMA, the New England Journal, and others doesn’t inspire it and the trust that was lost has yet to be earned back. That’s too bad, but JAMA will have to live with the consequences.

JAMA can try to make this poorly managed brouhaha a matter of principle, but the fact is that it is just plain, well . . . ewww.

Comments

#1 Orac
March 24, 2009

The record is muddy, so we won’t pass judgment on what happened here.

I will, as I think there’s more than enough information in the public record to draw some conclusions.

In February, I attended a talk by Dr. DeAngelis in which she bragged about her toughness, how she had stood up to big pharma at JAMA, and how she thought that no physician should ever take any money from pharmaceutical companies and playing her Italian mother schtick to a T, charming the audience. Then I find out that her bragging about having been educated by the Sisters of No Mercy seems to have led her to show no mercy on even people who are trying to bring an honest concern to her:

http://scienceblogs.com/insolence/2009/03/catherine_deangelis_and_jama.php

I’m sorry, but Dr. DeAngelis could have, as you pointed out (and so did I), let this die a quiet death. Given her history with regards to pharma influence, she would have been rapidly forgiven, and the blot on JAMA’s record would have been slight and quickly forgotten. Instead, she’s elevated this to a crusade against Dr. Leo. In a public editorial she has banned Dr. Leo from ever being published in JAMA and by doing so has in essence warned any academic physician who might ever want to publish in JAMA sometime that he or she had better toe the line or face being banned from JAMA forever. I’ve never seen such an incident of naked intimidation and retribution in a medical journal. Ever. It’s unforgivable.
#2 Thom
March 24, 2009

JAMA looks horrible in all of this. The complete arrogance that somebody must shut up until JAMA has made a ruling is simply ridiculous.

How is that a transparent and democratic process?
#3 CC2
March 24, 2009

The AMA has announced that scientific merit is NOT a criterion for publication in JAMA. Did I get that right?

I sure ain’t appending my name to this comment!
#4 leorising
March 24, 2009

Aggressively persecuting whistleblowers has got to have a chilling effect on the authors JAMA publishes. These editors have made a grave mistake, one that could damage JAMA’s reputation for objectivity for a long time to come.

Evidently, JAMA has decided peers cannot review each other in a realistic, real-world way any more. Researchers’ ties to industry and/or politics must be publicized at the sole discretion of JAMA editors — if ever, perhaps?

Why would any scientist/researcher/physician trust a supposedly objective, peer-reviewed journal with such an obvious subjective weakness?

I predict that, if JAMA’s editors pursue this course, JAMA will thrive as industry’s hottest new publicity organ. As such, it will make a lot of money and possibly unleash pharmaceutical horrors on an unsuspecting public. It’s days as a respected, peer-reviewed scientific journal are severely limited, though.

Soon, the only reason to read JAMA will be to find out what Big Pharma thinks. No one with two brain cells to rub together will trust any of the results.
#5 D. C. Sessions
March 24, 2009

This is reminiscent of the computer security “full disclosure” debate.

On the one hand, you have the software vendors who want security vulnerabilities disclosed privately to them, them alone, and kept secret until they go public with a notice and patch. Their argument is that going public prior to that point just invites attacks.

At the other extreme, you have the “full disclosure” security party who want vulnerabilities disclosed immediately, so that users (notably administrators) can implement workarounds, replace vulnerable software, whatever. One of their more compelling arguments is that vendors all too often sit on undisclosed vulnerabilities, treating them as PR problems rather than system problems.

In the open-source world, the convention is that researchers privately notify maintainers, who have a core team implement a fix, and distributors arrange to have simultaneous releases. All of this is happening under a short exclusivity period, so that if a fix isn’t released promptly it all goes public. Firefox’s record for rapid security fixes suggests that this approach works.

Make of the comparison what you will.
#6 Rich
March 24, 2009

People who teach at small regional institutions rarely have the opportunity to publish in JAMA or do the work in JAMA’s scope. Leo has a background in neuroanatomy and research on mental illness so, while he may not be at Harvard or have large research grants, he wasn’t just some random gadfly from no where. Banning someone like Leo is a petty arrogant act and I suspect JAMA wouldn’t yhave done it if he was a more “visible” person, but the policy is in place and the lack of accountability negates what the editor has tried to do in other areas. It’s also interesting that she chose to deal with the blatant ethical issue (financial conflict of interest), but not the scientific one (research-based psychotherapy provides comparable or better outcomes than anti-depresseant medication), which undermines her credibility as an editor even further.
#7 Sam Dawes
March 24, 2009

This is an excerpt from a PBS “Living On Earth” show dedicated to conflicts of interest in scientific publishing from December, 2006. It included Dr. Catherine DeAngelis, among others, and the interviewer was Bruce Gellerman:

“GELLERMAN: So what about your journal? I leaf through it and I see, you know, one third is articles and two thirds ads for drug companies.

DeANGELIS: Ok. Let me tell you about that. If it were up to me I would not allow any ads in our journal. However, it costs a lot of money to do what we do and putting out a journal like that. And so the one thing we do is we separate the advertising. That’s all done by the publisher. We have the strictest rules for advertising. One of our rules is you can never sell an ad to a specific issue. That happened while, during my first six months here and I got a call from the advertising group that sells and they said Dr D., this company wants to advertise this four-page ad. And I said wait a minute, and they said, cause it’s in one issue and I said wait a minute, we don’t do that. And they said no, no. They want it in that issue. So I looked at the issue. We work five six weeks ahead. Sure enough there was an article in there for a product by this company. And I thought about it for a minute and I said alright, we’ll take it. And they said really? I said sure. It was a big ad. And then I moved the article ahead two weeks.”
Orac seems to have heard the same story, as he relates in his “Respectful Insolence” post. Then there is this:

“CLAPP: Another approach is online and or what’s called open access journals where the, the person who’s submitting the article actually has to pay for it to be put online and, and actually pays for the peer review process.

DeANGELIS: Sure, but let me tell you what’s wrong with that. It’s unfair to those authors who don’t have money to pay for their article to go online. That is very poor practice.

CLAPP: Well, it’s a widely accepted and growing practice.

DeANGELIS: I know, I know. So is cancer. It’s widely growing and it’s, sometimes I look at it as the same effect. . .”

‘nuf said?
#8 revere
March 24, 2009

Sam: Yes, ’nuff said. The only thing I’d amend is that authors of OA journals don’t pay for peer review in the strict sense. Peer review is a practice covered by the processing fee, just like maintaining the server or the manuscript submission software. So no reviewer is paid and in many cases neither are the editors.
#9 Lea
March 24, 2009

Side tracking for a moment you do realize who owns the Wall Street Journal? News Corp. aka Fox News. One of the owner’s numerous mantra’s is “gossip”. While I’m not saying this is gossip I am saying consider WSJ’s role in this.

One other thought, it’s good this has come to light and anyone who slobbers over Big Pharma’s bribes should be exposed, repeatedly.

Little Miss Vanity, (out of control vanity apparently) has been exposed and I say Yipeee!
#10 revere
March 24, 2009

Lea: Yes, I know who owns the WSJ. They have always had an excellent news page, which is separated from the Editorial page, and so far I don’t think this has changed much. Moreover, the JAMA editors don’t seem to context the main facts.
#11 Orac
March 24, 2009

I forgot the link where I saw it, but it’s been reported by the WSJ that DeAngelis has not complained to the WSJ that she had been misquoted. Curious that.

As for the WSJ, its editorial page is wingnut central bordering on WorldNetDaily territory long before its current owner bought the paper. However, the WSJ news operation has always been superb, with an iron wall between news and editorial operations, and there is no evidence I’ve seen yet that it’s deteriorated.

revere’s right, too. Just read the JAMA editorial. Other than denying that DeAngelis callled Leo a “nothing and a nobody,” the editorial does not deny the essential account as reported in the WSJ; it just tries to spin it and pull the old (and lame) “we’re sorry if you interpreted our actions the wrong way” nonapology.
#12 pft
March 24, 2009

And peer review takes another hit. It is looking more and more like peer review is just a way to censor inconvenient truths, and promote papers having a political and/or business interest.

How is it that the editors do not confirm if the author has a financial interest in order to disclose it. Is it like Don’t Ask, Don’t Tell, and if someone catches the author covering it up you shut them up with threats of banishment or unemployment, not to mention that any funding for any research projects you might have, or endowments for your university might be at risk.
#13 revere
March 25, 2009

pft: Financial disclosure is not a function of peer review. Peer review is for the technical and interpretive aspects of a paper. It is the colleague you ask to read your paper for a second look, to catch errors or give an opinion. It is fallible but it has its place as one of the ways that scholarly work is made better or checked. Regarding editors “checking” on financial disclosure, how do you propose we do this? We depend on voluntary disclosure and if there are violations that come to light there should be public sanctions. The behavior of JAMA in this instance is reprehensible and they will take a hit for it. Calling them on it is a form of journal peer review.
#14 moneduloides
March 25, 2009

What mechanisms are in place to deal with individuals who do not disclose their financial ties? Do the journals simply retrospectively pull the article and call it a day? I feel as though it would be more than justified for the popular medical journals to get together and create a blacklist of individuals who do not disclose financial ties to pharmaceutical companies when they publish, for the public and other medical journals to see.

The actions of JAMA in this case are completely and utterly unforgivable. DeAngelis should be removed from her post.
#15 revere
March 25, 2009

monoduloides: It’s not quite so simple. Papers are only retracted if they are shown to be false or fraudulent regarding the science. Conflicts are often a matter of interpretation. The usual way to handle it is to issue a Correction, often with an apology for the omission. Once a paper is in the literature and has been cited, it is a problem to remove it. Better to brand it with a correction so readers can make their own judgments. Then it is no different than if the relationship were disclosed initially. Knee jerk technical fixes almost always have unintended consequences. In this case the financial ties occurred several years earlier when the author was part of a speaker’s bureau. At what point is there no conflict? 2 years? 5 years? Never? Are government scientists conflicted out? Those who work for advocacy organizations? These are not easy questions, so some standard for what constitutes a conflict has to be stated by the journal in an understandable way that is meaningful for the reader’s ability to judge.
#16 JJackson
March 25, 2009

Would it not be more sensible to ban an author for life, for failing to disclose they had a potential conflict of interest, rather than the person that bought it to light?
#17 JJackson
March 25, 2009

That should be brought, but then in this story perhaps they both work.
#18 revere
March 25, 2009

JJ: I agree it is reprehensible to kill the messenger. But banning for life doesn’t make sense, either. There are often disagreements as to what is a conflict or even at times somone forgets. Moreover, mores change. Zero tolerance policies are usually bad. For flagrant conflicts, there needs to be some kind of sanction. Usually loss of grant support or even loss of a position is a consequence, as has happened lately. But we can over react to these things, just as the JAMA editors over reacted in the other direction. Sould we ban JAMA for life?
#19 moneduloides
March 25, 2009

@revere: I agree that it should be clearly stated in the respective journal what the criteria are for meeting a conflict of interest, but just because you can’t answer what is and isn’t a conflict of interest for a few cases (You mentioned government employees, for instance), doesn’t mean you cannot do this for others.

For instance, if a physician or scientist believes it is so worthwhile to accept funding from pharmaceutical companies, then they should forfeit their ability to publish any work on products produced by said company (Yes, a lifetime ban). Of course, if they agreed to this they would never receive the funding offer in the first place, and I’m not so sure this is a bad thing. Actually, keeping the hands of pharma out of the academic peer-review process is exactly what we DO need.
#20 daedalus2u
March 25, 2009

The editors exert editorial control over content and that control reflects their personal biases, prejudices and COIs. In many ways editor COIs are much more serious than author COIs because The editor affects everything in the journal.

For an editor to discourage an author from submitting something is (in my opinion) completely unacceptable as an editor. It means the editor is running the journal as a personal fiefdom and basing content on personal and idiosyncratic whim and cronyism, and not based on scientific content, news worthiness or relevance to the journal’s readers or the broader scientific community.

Good scientific journals do not exist because there are good editors. Good scientific journals exist because good scientists submit good papers, and good scientific peer reviewers review them to confirm that they are good and good scientists read and cite those good articles in still more good papers. Good editors are important, but they are important like the grease in a gearbox that keeps all the parts moving efficiently.

Banning for life doesn’t make any sense. Neither does banning employees of big pharma or of anyone else. By instituting a prospective ban, what is being said is that this person will never in the rest of their life produce something that will be worthy of being published. If you believe that nothing that big pharma has funded is worthy of being published, then you should also believe that no product of big pharma is worthy of being used to treat a disease.
#21 anon
March 25, 2009

Mono. I don’t think you are listening.

Mono said
“For instance, if a physician or scientist believes it is so worthwhile to accept funding from pharmaceutical companies, then they should forfeit their ability to publish any work on products produced by said company (Yes, a lifetime ban).”

The funding is to do the work. Doing the work and not publishing it is far worse an ethical breach than anything we have talked about thusfar. In short this is one of the most ridiculous suggestions I’ve heard surrounding this debate so far. Drug companies bear the responsibility for showing that their drugs actually work. How the hell are they supposed to do this without contracting independent research?

One of the consequences of your idea is that negative studies never get published simnply because they were funded by a drug company. And if you think this never happens then I can assure you that I’m the 1st author of exactly such an industry-sponsored study with industry co-authors of a manuscript quite clearly telling physicians not to use their drug for purpose X.

Furthermore, Revere has already pointed out that this still HAS NOTHING TO DO WITH PEER REVIEW MECHANISMS.

The non-disclosure of a COI by an author is a bad thing. What is really the issue here is the very odd reaction of the senior JAMA editors.
#22 Marissa
March 26, 2009

Sad. This smacks of Merck’s tactics to smear researchers who contested the results of the VIGOR and other trials of Vioxx, one of the sorrier episodes in pharm history.
#23 moneduloides
March 26, 2009

@anon: Actually, if you would have refrained from selectively quoting me, and read merely a sentence further you would’ve seen the following:

“Of course, if they agreed to this they would never receive the funding offer in the first place, and I’m not so sure this is a bad thing.”

I never once implied it was ethical to take money for work and then refrain from doing it. What I did imply, however, was that funding from pharmaceutical companies provides too great an incentive to fudge numbers, an incentive that really isn’t necessary to execute the research necessary to get new drugs to sick people.

“How the hell are they supposed to do this without contracting independent research?”

Perhaps you have heard of this little group called the ‘Consumers Union’, which publishes a periodical entitled “Consumer Reports”? They thrive on this crazy philosophy that its possible to objectively test products without the influence of the corporations producing them. I know… it’s totally nuts to think this could happen in pharmaceutical research.

“Furthermore, Revere has already pointed out that this still HAS NOTHING TO DO WITH PEER REVIEW MECHANISMS.”

Actually, it does. Let me lay out a simple flow chart for you:

Funding goes from pharma to researcher to test drug -> Researcher now has an incentive to fudge the numbers on said drug (Most recently: Scott Reuben, the profs @ Harvard that lied outright to their students, etc…), although it certainly doesn’t follow that they WILL fudge the numbers -> Scott Reuben fudges the numbers -> Peers review his research, and find it to be acceptable, because they believe his research to have been objective and not influenced by a specific cashflow -> Reuben is published = Peer review process fails at its primary role, which is to filter out unscientific research.

A flaw in peer review itself? No. It’s more like a virus that infiltrates the process and renders it completely ineffective.

Is this as much about stating COI as straight-up lying? Well, I would like to think that a bit more sound proof is required of an individual that states a COI due to funding from the company that produces the drug he/she is researching. I know this is me living in an idealized world, because of course this doesn’t happen at all. Perhaps, instead, there should be a mechanism in place that requires outside observation or independent trial repetition before articles are published all willy-nilly?

Is this too much to ask? If your answer is “yes,” then perhaps you should be asking yourself if it is ethical for such research to be published when it has the very real likelihood of killing patients. One of the drugs Reuben published on was pulled from the market for causing strokes and a variety of other conditions. This could have been prevented with such a mechanism in place as I have outlined above.

Does fudging numbers happen with people who have no COI? Yes, all the time. We call these people “incompetent.” There, however, is no incentive for these researchers to lie, and as this incompetency is almost always a result of a flawed research model, there IS a mechanism in place to weed them out: peer review.
#24 Markk
March 26, 2009

Moneduloides, you want me as a taxpayer to pick up the tab for validating drugs that drug companies produce? No thanks, I think the drug companies themselves should pay for this. If they do, then who is supposed to do the research? You just said nobody should. You are putting on the public the cost of developing drugs so the drug companies expenses and liability just went way down. You just made a friend of all the drug companies. Are you a shill for them? (joke) But what you are saying is unrealistic. We all have influences. Better that things are disclosed. People will judge the work over time and know if something smells.

You say there is no incentive for independent researchers to lie, but that is not the case. There is ALWAYS an incentive. It need not be financial. In fact looking at the major academic fraud cases recently I can’t think of one that had a financial motive directly. Things would be pushed under the table and I think drug safety would actually get worse under your proposal.

Consumer Reports works only because what they do is not so expensive. Under your system places like UL which Consumer reports relies on would not be done because they are funded partly by the manufacturers. So that kind of research would be right out.
#25 moneduloides
March 26, 2009

—
Moneduloides, you want me as a taxpayer to pick up the tab for validating drugs that drug companies produce?
—

No, and I never mentioned this. And, as far as I understand Consumer Reports is not funded by tax dollars, so I’m not sure where you’re getting this idea from (I could be mistaken).

—
But what you are saying is unrealistic.
—

How is it unrealistic to expect innovation on the part of scientists to come up with some sort of mechanism or organization that can control this type of scientific fraud? No, it wouldn’t look exactly like Consumer Reports, because this is not an identical beast, but their model is something to start from. What is happening with COI and fraud these days is not acceptable, and the mechanisms we have in place obviously aren’t sufficient to neutralize the problem. Perhaps you’re okay with this reality, but I’m not.

—
You say there is no incentive for independent researchers to lie, but that is not the case.
—

Of course there is incentive in independent research; people publish, and in turn receive tenure. I suppose I wasn’t quite clear enough. I was referring to the Consumers Union model when I made that comment.

With a model like the Consumers Union, made up of researchers independent of pharma funding, the research burden would be spread more widely, and there would be a great deal of internal fact/method-checking before any research was published, as individual #1 has the incentive of covering their own ass by intensely analyzing the research of individual #2 before the work is published with the organization’s name on it (of which both individual #1 and #2 would be listed as contributers).

—
Consumer Reports works only because what they do is not so expensive.
—

Really? I’m pretty sure what they’re most known for is crashing cars. Crashing cars over, and over, and over again… cars that they themselves pay for… If it’s so cheap I would love to see a source for the numbers. I’m honestly interested in knowing how the expenses of the two compare.
#26 antipodean
March 26, 2009

Mono

It’s quite clear that you don’t understand either peer review or the drug registration process.

Consumer reports doing randomised controlled trials of drugs. That’s just ignorant. What you vaguely propose is that nobody does the research. Which is a system we already have for CAM and supplemental healthcare products (i.e. vitamin pills and the like). The current FDA system was set up to replace this sort of crap from happening with real drugs- we’ve seen what happens in your laissez faire dream world already. Drugs are not like consumer products. The cost of getting one patient through a long drug trial might in many cases be equivalent to your car example. There are many thousanbds of such patients in some trials. You are underestimating the cost of drug trials by orders of magnitude.

As for peer review. I could sit at my desk right now make a whole lot of shit up and get it published. This would still not be a failure of peer review. However, to do this and get caught means the end of my scientific career. Plus I really don’t like the idea of harming people even though according to you I am an evil agent of pharma who must be making stuff up. What I don’t like are clear conflicts where physicians are on advisory boards, speakers lists or own shares.

Peer review is not a fraud detection device. It can sometimes detect fraud but it’s primary utility is cleaning up manuscripts so that they present a coherent scientific argument. Honesty is an assumption of the system. It can be broken but when that has been detected you get the massive storm of outrage that we are witnessing repeatedly.

Once again you have proposed that it not be possible for professionally done research to be published. That is entirely unethical. Plus it’s a return to Victorian gentleman’s science- strictly for the men of leisure. And I’ll repeat this again since you won’t listen. It will stop the publishing of negative studies which show the drugs don’t work.
#27 maynet
April 29, 2009

Thank you Markk