Otis Dudley Duncan and Tim Lambert
The discussions on so many blogs on the Lott case have been invaluable not only in turning up new evidence but also in illustrating the wide variety of reactions to the evidence. One thing that strikes a professional researcher is that many who have commented do not look at such cases in the way that researchers are trained to do. What this case is about is the professional work of a a social scientist, and the question is whether that work meets the ethical standards of scientific inquiry and reporting. And commentators are not at liberty to define for their own purposes just what the relevant ethical precepts are. A great deal of thought has been given to the codification of the main ethical principles, violation of which constitutes scientific misconduct. We think that the quality of comments would be enhanced if the writers took a little time to look at some of the formulations that have been proposed. The American Society of Criminology has a page that gives links to several of them.
A Google search for "Office of Research Integrity" shows that many universities have such ethical policies. A particularly extensive one is at the University of Pittsburgh Office of Research Integrity. It begins:
"Fabrication and falsification of research results constitute serious misconduct. It is a primary responsibility of a researcher to avoid either a false statement or an omission that distorts the truth. In order to preserve accurate documentation of observed facts with which later reports or conclusions can be compared, every researcher has an obligation to maintain a clear and complete record of data acquired."
An especially rich source of actual cases in which scientific misconduct was alleged and in which action has taken place is the files of the U.S. Office of Research Integrity. An example in point is the case of a Dr Duan. He was charged with engaging "in scientific misconduct by reporting research that was inconsistent with original data or could not be supported because original data were not retained" and the report notes that "Dr Duan denies all allegations of scientific misconduct and contends that some of his original data is missing." The resolution of the case was:
Dr Duan has entered into a Voluntary Exclusion Agreement (Agreement) with PHS, in which Dr Duan has voluntarily agreed [among other things] .............(2) That for a period of one (1) year after the conclusion of the voluntary exclusion period, any institution that submits an application for PHS support for a research project on which his participation is proposed or that uses him in any capacity on PHS supported research, or that submits a report of PHS funded research in which Dr Duan is involved, must concurrently submit a plan for supervision of his duties to the funding agency for approval; the supervisory plan must be designed to ensure the scientific integrity of Dr. Duan's research contribution, and the institution must also submit a copy of the supervisory plan to ORI; .............
(4) That he will not oppose the submission to journals of a statement summarizing the current state of the science with respect to the scientific matters at issue relating to grant R01 AI36552, which has been jointly agreed to by Thomas Jefferson University and the United States of America.
The central issue, therefore is that of "fabrication and falsification". It is beyond question that Dr Lott published false statements attributing his 98% figure to various national surveys and to Kleck.
Was there fabrication and falsification in regard to the findings of the survey that Dr Lott has stated that he conducted? Even conclusive proof that he conducted a survey in 1997 would not, of itself, resolve this issue. The findings of that survey could have been fabricated even if a survey was actually carried out. The main reasons for suspecting that this happened are:
- Dr Lott claimed his survey was conducted over a three month period in 1997. He also claimed that survey data shows that 98% of defensive gun uses involve merely brandishing weapon as early as Feb. 6, 1997, in his testimony to the Nebraska legislature.
- Long before Dr Lott ever mentioned that the 98% figure came from his survey, he had (erroneously) attributed it to various other surveys.
- The statistics concerning incidence of defensive gun use, frequency of firing or brandishing, and firing target (warning shots vs. shooting at the offender) are such that it is hardly conceivable they could have been legitimately produced by a sample of the size Dr Lott has mentioned.
- Dr Lott has been unable to supply any documentation whatever of the particulars of his survey such as the exact wording of the questions asked, the sampling plan, calculations of statistics, and so on. Any text on survey research will make explicit the details of what is normally expected by way of documenting a survey. A formal listing can be found at the American Association for Public Opinion Research. And it should be borne in mind that loss of data because of a computer crash or any other reason does not relieve the investigator of the responsibility to provide such documentation. It is a clear breach of the ethics of science to promulgate a finding in the absence of data (see the Duan case, above).