Psych Problem #2: Cooking the Books

Lisa Bero

Critics of the FDA drug-trial process have often complained that the drug companies are free to publish only the trials that are flattering to their cause (that is, only those that show effects above placebo and relatively low side-effects). As explained in Wired Science, UC San Francisco health policy expert and Cochrane Collaboration co-director Lisa Bero has been picking this process apart:

The difference between what drug companies tell the government and doctors suggests that they're cooking the books, which could mislead doctors making prescriptions.

Of 33 new drugs approved by the Food and Drug Administration in 2001 and 2002, one-fifth of supporting clinical trials were not published in medical journals, according to a new study. And those results that were published were often more positive than what companies presented to the FDA in their applications. As a result, potentially unreliable data is being used to promote drugs on which billions of dollars and thousands of lives may ride.

"Some studies aren't published at all. Then, when they are, there are little changes that make the papers look more favorable towards the product," [said Bero].

Among the things Bero found:

Among the differences between results submitted to medical journals and to the FDA were trials that didn't favor a company's product, Bero found. Only half of 43 such outcomes were reported in the literature. More subtly, but just as importantly, key pieces of trial data vanished.

"The main thing that jumped out at me was the addition and deletion of primary outcomes. Those are the most important outcomes of a trial. To find that one disappeared from a paper, or just appeared in a paper, is pretty amazing to me," said Bero.

How to fix this? The FDA does things this way partly because the drugs companies are paying for the studies, and so get to control them. But there is a healthier model:

Bero calls for the FDA to be overhauled to run clinical studies itself, as is done by comparable agencies in Italy and Spain.

"The Italian FDA collects money from every drug company that sells drugs in Italy, pools that, and funds drug trials. They fund the sort of head-to-head drug comparisons that companies don't like to fund. And they have independent people peer-reviewing the trials. It's a great model," she said.

Bero's study is at PLOS Medicine.