Cognitive Forensics

Nick Bostrom offers up a great suggestion for a new academic field:

Perhaps we need a new field of "cognitive forensics" for analyzing and investigating motivated scientific error, bias, and intellectual misconduct. The goal would be to develop a comprehensive toolkit of diagnostic indicators and statistical checks that could be used to detect acts of irrationality and to make it easier to apprehend the culprits. (Robin's recent post gives an example of one study that could be done.) Another goal would be to create a specialization, a community of scholars who had expertise in this subfield, who could apply it to various sciences, and who could train students taking advanced methodology classes.

Of what components would cognitive forensics be built? I'd think it would have a big chunk of applied statistics, but also contributions from cognitive and social psychology, epistemology, history and philosophy of science, sociology of science, maybe some economics, data mining, network analysis, etc.

The time might be ripe for this sort of endeavor. I have the impression that scattered articles on the problems of peer review and on possible statistical biases in scientific research (e.g. by funding source, file drawer effect etc.) are now appearing fairly regularly in Science and Nature.

I heartily agree. The psychological sciences have made tremendous progress identifying the consistent mistakes made by the human mind when working with numbers and making "judgments under uncertainty". And yet, this awareness hasn't really altered the way science is done. The same statistical errors and biases crop up again and again. A new scientific field, dedicated to exposing these mistakes, seems like a worthy undertaking.

That said, a professor of scientific forensics probably wouldn't be very popular on campus, considering their main academic aim would be criticizing the work of their colleagues.