Posted by Steve Novella at Science-Based Medicine, a level headed and judicious advocate of better use of scientific evidence in clinical medicine, has written his own view of the BPA issue we covered in a post the other day. Orac pointed to it in the comments as “another take” on the issue. We aren’t sure if he meant it disagreed with the view we expressed or not. For the record, we have some differences, but not on the judgment about the BPA paper. Differences about that are mainly a matter of emphasis and that’s pretty subjective. “Real but small effects” may be very important but we don’t know that yet. Those judgments aside, we would like to take the opportunity to comment on some other parts of Steve’s post which we think bear some discussion, particularly the question of how one demonstrates causation. He opines that the only way to do this is through a randomized experiment. We disagree. Our view is that there is no way to prove causation but many ways to demonstrate it. Unfortunately this subject quickly gets us into deep water and it can’t be done in a single post. Indeed, since many books have been written on this subject and there is no consensus, many posts won’t do the trick either. So we’ll settle for making a couple of points.
The most important is that the question of causation and how to demonstrate it is not settled by philosophers of science. The only ones who think it’s settled are scientists and that’s because they aren’t experts in the subject. As one wag once said, expecting a scientist to understand scientific method is like expecting a fish to understand hydrodynamics. Scientist are experts in doing science. But they do not often understand exactly the logic of what they are doing.
Consider the role of deductive reasoning, which most scientists take to be one of the hallmarks of scientific method. Yet its use is fairly restricted, mainly to constructing mathematical tools. Beyond that it has limited relevance because deductive reasoning requires something we don’t have in empirical science, absolute certainty. Here’s an example from the late ET Jaynes’s book, Probability Theory:
Suppose some dark night a policeman walks down a street, apparently deserted. Suddenly he hears a burglar alarm, looks across the street, and sees a jewelry store with a broken window. Then a gentleman wearing a mask comes crawling out through the broken window, carrying a bag which turns out to be full of expensive jewelry. The policeman doesn’t hesitate at all in deciding that this gentleman is dishonest. but by what reasoning process does he arrive at this conclusion?
[snip]
A moment’s thought makes it clear that our policeman’s conclusion was not a logical deduction from the evidence; for there may have been a perfectly innocent explanation for everything. It might be, for example, that this gentleman was the owner of the jewelry store and he was coming home from a masquerade party, and didn’t have the key with him. However, just as he walked by his store, a passing truck threw a stone through the window, and he was only protecting his own property.
Now, while the policeman’s reasoning process was not logical deduction, we will grant that it had a certain degree of validity. The evidence did not make the gentleman’s dishonesty certain, but it did make it extremely plausible. This is an example of a kind of reasoning in which we have all become more or less proficient, necessarily, long before studying mathematical theories. We are hardly able to get through one waking hour without facing some situation (e.g., will it rain or won’t it?) where we do not have enough information to permit deductive reasoning; but still we must decide immediately what to do. [emphasis in the original]
This passage appears in the first two paragraphs (Chapter 1, p. 3) of Jaynes’s Probability Theory: The Logic of Science. It is the posthumous masterwork of one of the 20th century’s foremost mathematical probabilists, a graduate level text in mathematical probability theory. He was no lightweight and certainly not a crank. The 700 pages that follow contain Jaynes’s development of the seminal work of Jeffreys, Cox, Shannon and Polya, focusing not on deductive reasoning but on plausible reasoning and its rules. It’s a book in the logic of probability, not the logic of certainty, which is what deductive logic studies.
The product of deductive reasoning is proof. Judgments about causation are the product of plausible reasoning. “Randomized” experiments are one (important) tool of plausible reasoning in epidemiology and some parts of clinical medicine. They are rarely used in the physical and chemical sciences laboratory or in the animal studies. Animals are rarely randomized nor the researchers unaware of the independent variable. The purpose of randomization, when it’s done, is to reduce the chance of differences in two groups unrelated to the independent variable of interest. Randomization neither guarantees comparability nor demonstrates causation. There is a much deeper and more complicated argument that goes with each of those statements but for the moment let’s observe that if a randomized experiment were required we would still not know if cigarettes cause lung cancer, be unsure about the validity of relativity and not know much about chemistry, astronomy or geology, much less evolution and paleontology. All of these sciences use rigorous forms of plausible reasoning without requiring randomized experiments to demonstrate their basic premises or accepted truths. Steve acknowledges this in discussing lung cancer and smoking without, in our view, fully grasping the nettle.
Because whether required or not there are some deep questions about randomization and causation. The role of randomization was the subject of some fundamental disagreements between RA Fisher and Egon Pearson. There is nothing cut and dried about the subject and the consequences of the differences are important. But this is a whole other set of topics that epidemiologists and statisticians argue about, particularly causation, where the counterfactual school battles with more conventional methodologists in the realm of theoretical and practical methods. It’s an interesting subject of some interest to us, but there is too much to say within the confines of this post.
The bottom line here is this. We think laying down absolute markers on what is required to demonstrate causation in medicine is a fool’s errand and potentially dangerous. If accepted as the summa of scientific methods in toxicology (“the only direct evidence of causation”) it would zero out the bulk not only of toxicology results but of all science (because if it is a valid and required method for toxicology, why just for toxicology?) That doesn’t make much sense to us. There is no “direct evidence” of causation. Causation is a judgment about the available evidence using plausible reasoning. There is no privileged form of evidence. Indeed we consider this to be a scientific sin: methodolatry.
We think Steve is a terrific blogger (as is Orac). What fun is having a blog if you can’t start a food fight about ideas with someone you respect?
September 18, 2008
September 18, 2008
September 18, 2008
September 18, 2008
September 18, 2008
September 19, 2008
, I would be eternally grateful. Thank you very much in advance.