The Frontal Cortex

I agree with what you're saying, David, but I would also argue that enough correlations do give you reason to suspect that, regardless of causation, if you're a drug company, it's in your best interests to fund as many studies as possible for more self-interested reasons that just scientific benefit. And time and again, we've seen such instances in various journals in various disciplines. It reminds me of those people that refuse to accept poverty as a legitimate cause of illness. "Sure," they say, "the studies all do point to that fact, but, there are more variables to consider, so we shouldn't list poverty as a cause yet." The same logic can be applied ad nauseum til the end of time, regardless of how much evidence is accumulated. And, of course, I'm not sure how you could get drug companies to go along with an experiment to test the hypothesis anyway.

Posted by: lilyang | August 5, 2006 12:58 AM

It's true that when I used the word "correlation" before, in the back of my mind was the way tobacco companies used to say smoking and lung cancer was "only" a correlation, as if there could be some other factor that caused so much cancer without it being smoking. I'm just saying that a good scientist or a good journalist needs to consider other possibilities.

One thing I didn't mention before was that a red flag went up in me when I read that financial incentives increased the chance of a positive study 5 times. 5 times?!!! Not 20%, which is what I might think unconscious fudging could do, but 500%? This is either outright fraud or someone is comparing apples and oranges. Now that's my intuition. I hope it's experienced intuition, but even if it's raw intuition, a scientist needs such prodding to look at the quality of any study. What were the design of these studies? There is an effect where drug studies based on community use don't do as well as more formal studies where someone stays on the subjects to take their meds, even if it is a placebo. I can't imagine that could account for a 500% difference, but it has to be something.

And this is an issue where long-term experience will show which studies were right. Who will look at it again in 5 years to see? The drug companies may not want to resurrect the issue, even if experience exonerates them. The authors of the larger study might have moved on.

I realize also that it's something of a moot point. I'm sure the drug companies will adapt to whatever perception is. If they can make more money controlling their studies, strictly from a matter of quality control, to get them done faster, and limit spurious incidents, they'll do that. If they have to do that through "clean" investigators, they'll do that. If they have to give up control of evaluation to government or academics, they'll do that.

I'm more interested in the general issue of reading something in the media where I have questions about the study, and no one raises them. I don't think it's good to have a reporter gushing over what a good result this is instead of asking the questions that a good scientist knows to ask.

Posted by: DavidD | August 5, 2006 8:22 PM