The APA has an important rule that all authors of APA-sponsored journal articles must agree to before publication:
After research results are published, psychologists do not withhold the data on which
their conclusions are based from other competent professionals who seek to verify the
substantive claims through reanalysis and who intend to use such data only for that
purpose, provided that the confidentiality of the participants can be protected and
unless legal rights concerning proprietary data preclude their release.
The rule seems quite straightforward. But when data is requested, how many researchers actually comply? A group led by Jelte M. Wicherts has put that question to the test (PDF via BPS Research Digest). They asked the authors of articles in four prominent journals for copies of their datasets, a total of 141 studies. If it was demanded, they provided their academic credentials and institutional approval for their work. How many researchers complied? A chart of their results is below.
In the end, only 27 percent of researchers complied with the request for data. Through non-reaction, or outright refusal, the vast majority of researchers did not share their data.
There are a few points in the researchers’ defense. In some fields, sharing data may not be something that is typically required or done — researchers may have felt that everything a fellow scientist would need to replicate their results was included in the journal article itself. Also, when you work in a specific field, you know who your colleagues are around the world. There is only a very small set of researchers engaged in your particular line of work — perhaps fewer than a dozen. It may be that psychologists would be more willing to share data with someone when they were certain that person was working on the same problems they were. Getting an email out of the blue from a researcher you’ve never heard of is different from a request by a respected colleague.
On the other hand, the opposite might be true; researchers might view these colleagues as “competition,” and be even less willing to share with them. It would be fascinating to see a study where the requests for data came from close colleagues. This would be more difficult to do, but the results would be more telling as well.
In other news:
- Mind Hacks has links to cognitive scientists’ predictions for the next century
- Deric Bownds has an excellent explanation of how studies of “invisible” images work
- Jonah Lehrer tells us about new research on phantom limbs
- Nominate your favorite science blog for the Weblog Awards (I nominated Mixing Memory)