Predatory open access journals seem to be a hot topic these days. In fact, there seems to be kind of a moral panic surrounding them. I would like to counter the admittedly shocking and scary stories around that moral panic by pointing out that perhaps we shouldn't be worrying so much about a fairly small number of admittedly bad actors and that we should be more concerned with the larger issues around the limitations of peer review and how scientific error and fraud leak through that system.
I'm hoping my methodology here will be helpful. I hope to counter the predatory open access (OA) journal story with a different and hopefully just as compelling narrative. Fist of all, after gathering together some of the stories about predatory OA journals, I will present some of what's been written recently about issues in scientific peer review, it's problems and potential solutions.
Then I'll be presenting a more direct counter narrative to the predatory one. First of all, I'll present some information about the fantastic resource Retraction Watch. Then I'll present some concrete case studies on how traditional peer reviewed commercial publishing fails in all the same way that supposedly predatory publishing fails.
Finally, using the incredible work of Walt Crawford and others, I'll gather some resources that will further debunk the whole "predatory" open access moral panic and further suggest that perhaps it isn't the bogus OA journals that are the main source of "predatory" publishing, but rather that the big commercial and society publishers perhaps deserve that label more.
I want to be perfectly clear. My issue isn't with the necessity of peer review and it's importance in science. Issues like climate change and vaccination panics highlight why trusting in peer reviewed science is most responsible thing to do. After all, “Research misconduct accounts for a small percentage of total funding”. I think it's probably safe to say that at the end of the day, peer review and scientific publishing work fairly well as far as fraud and general quality levels go.
Both peer review in particular and the scholarly communications ecosystem in general are human systems with all the potential for the full range of human weaknesses that implies: folly, error, bias, fallibility and bad faith. This post will explore some of the dimensions of folly, error, bias and bad faith in scholarly communication.
Let's start our adventures with some media stories and cases studies of bad faith -- true predatory open access journals.
- Nov 2014. Fringe activists learn to use 'predatory' science journals
- Nov 2014. Sham Journal Accepts Totally Absurd But Completely Appropriate Paper (The Get Me Off Your Fucking Mailing List paper.)
- Dec 2014. A paper by Maggie Simpson and Edna Krabappel was accepted by two scientific journals (More here)
- Jan 2015. Why a fake article titled "Cuckoo for Cocoa Puff" was accepted by 17 medical journals
Predatory journals are a real problem, of course, as we can see from the list above. However, I think the moral panic about their extent and impact tends to be exaggerated. I would really love to see more balance in reporting about predatory journals that contrast the real issues with scam journals with what I think are the far more pressing issues in scholarly communications. In other words, the flaws and limitations in the peer review system and the far more "predatory" traditional system of scholarly publishing that's controlled by the big commercial and society publishers. It's those publishers that are the leeches affecting the system.
These stories and anecdotes about predatory journals tend to acquire the mythic stature of the stories and anecdotes about vaccination that drive the anti-vaccine movement. Those tragic, personal stories take on a weight and social impact that's disproportional to the actual scientific and statistical significance.
Time to explore bias and human fallibility a little bit. Here are some resources about the general state of peer review, talking in general about the issues around peer review and the potential for reform. This list is meant to contrast the moral panic about "predatory" open access journals with a sober discussion of the strengths and weaknesses of peer review across all of science publishing, not just some fairly specific issues with a limited number of open access journals.
General Resources on Quality in Scientific Publishing, particularly on Issues with and Reform of Peer Review
- Jul 98. Effect on the Quality of Peer Review of Blinding Reviewers and Asking Them to Sign Their Reports: A Randomized Controlled Trial
- Aug 2005. Why Most Published Research Findings Are False
- Dec 2008. Exploring Scientific Misconduct: Isolated Individuals, Impure Institutions, or an Inevitable Idiom of Modern Science?
- May 2009. Three myths about scientific peer review
- Aug 2011. High-profile titles lead the field in number of retractions
- Feb 2013. Opinion: Scientific Peer Review in Crisis
- Jul 2013. Why Has the Number of Scientific Retractions Increased?
- Apr 2014. What Can We Do About Junk Science?
- Jun 2014. The perils of Science by press release: ‘overly exaggerated presentation of research findings’
- Jul 2014. The Guardian view on the end of the peer review
- Oct 2014. Science fiction? Why the long-cherished peer-review system is under attack
- Nov 2014. The Open Science Peer Review Oath
- Nov 2014. Measuring the effectiveness of scientific gatekeeping
- Nov 2014. Impact Challenge Day 19: Establish your expertise with Open Peer Review
- Dec 2014. Peer review could reject breakthrough manuscripts, study shows
- Dec 2014. Peer review isn’t good at “dealing with exceptional or unconventional submissions,” says study
- Dec 2014. Peer review — reviewed: Top medical journals filter out poor papers but often reject future citation champions
- Dec 2014. Scholarly Context Not Found: One in Five Articles Suffers from Reference Rot
- Dec 2014. PLOS is anti-elitist! PLOS is elitist! The weird world of open access journalism.
- Feb 2015. The Ethics of Authorship: Is Ghostwriting Plagiarism?
- Feb 2015. Scandals prompt return to peer review and reproducible experiments
- Beyond Beall’s List: Better understanding predatory publishers
- Mar 2015. We need post-publication peer review of journals
- Mar 2015. Why you can't always believe what you read in scientific journals (Pubpeer & post-publication peer review)
- Mar 2015. Springer and Université Joseph Fourier release SciDetect to discover fake scientific papers (More info)
- Mar 2015. The Extent and Consequences of P-Hacking in Science ("One type of bias, known as “p-hacking,” occurs when researchers collect or select data or statistical analyses until nonsignificant results become significant. Here, we use text-mining to demonstrate that p-hacking is widespread throughout science.")
- Mar 2015. Amid a Sea of False Findings, the NIH Tries Reform
- 2015. Data fraud in clinical trials
- xxxx. COPE statement on inappropriate manipulation of peer review processes (Committee on Publication Ethics, undated)
(The Shit My Reviewers Say tumblr is a lighter-side exploration of some of these issues.)
More importantly perhaps, there is another set. There is no shortage of fairly well publicized cases of significant retractions or scientific fraud that got past the peer review process in traditionally published, peer reviewed journals, mostly from the big commercial or society publishers. In other words, where peer review was the issue, not the subscription model.
If you want to know about the failing of the big publishers when it comes to quality control or about researcher perpetrating scientific fraud, Retraction Watch is the definitive site on the web.
Resources by and about Retraction Watch
- Aug 2011. Is it time for a Retraction Index?
- Dec 2012. Elsevier editorial system hacked, reviews faked, 11 retractions follow
- Jun 2014. “Barriers to retraction may impede correction of the literature:” New study
- Jul 2014. Retractions are coming thick and fast: it's time for publishers to act
- Nov 2014. Publisher discovers 50 manuscripts involving fake peer reviewers
- Nov 2014. The Peer Review Scam: How authors are reviewing their own papers
- Nov 2014. Publishing: The peer-review scam
- Dec 2014. Science journals screw up hundreds of times each year. This guy keeps track of every mistake
- Dec 2014. Retraction Watch is growing, thanks to a $400,000 grant from the MacArthur Foundation
- Dec 2014. Are companies selling fake peer reviews to help papers get published?
- Dec 2014. Why MacArthur is Backing a Popular Blog About Flawed and Fraudulent Science
- Dec 2014. The Top 10 Retractions of 2014
- Jan 2015. Hall of Shame
- Mar 2015. Yes, we are seeing more attacks on academic freedom: guest post by historian of science and medicine
The site Science Fraud was taken down by various legal threats. While it existed, it was an amazing resource for uncovering practices such as falsified images or tables. Some posts are retrievable via the Internet Archive.
SCIgen is a website that allow anyone to automatically generate a bogus paper. It is often used to generate garbage papers for predatory open access journal stings. SCIgenDetection is one site that detects SCIgen papers. The SCIgen page has a number of examples and other resources related to automatically generated bogus scientific pages.
Springer has recently teamed up with Université Joseph Fourier to release the a generalized open source software package SciDetect which tries to detect fake scientific papers such as those generated by SCIGen.
And yes, the main event where we explore a different dimension of bad faith and human folly and weakness. This time on the side of the supposedly "good guys."
Bellow are examples where big commercial or society traditional, subscription-based peer review have fallen short, either due to careless or insufficient review or fraud on the part of scientists. Of course, peer review will rarely catch genuine fraud as the books are cooked. But even fraud cases demonstrate the limits of peer review across all scholarly communication, not just in "predatory" open access journals.
I would like to emphasize that this list is extremely selective. I'm mostly only highlighting particularly egregious examples that have made their way into the mass media or onto popular blogs. As above, for much much more, please visit Retraction Watch for more complete coverage. For example, The top 10 retractions of 2014.
This list is meant to contrast in number and severity to the list of examples of "predatory" open access publishing crisis and stings above.
Failure in Scholarly Communications Ecosystem through Stupidity, Error or Fraud
- 1989. Fleischmann–Pons experiment cold fusion controversy
- Apr 1994. Doctor Says He Falsified Cancer Data to Help Patients (Roger Poisson fakes trial data)
- Mar 1999. A Doctor's Drug Trials Turn Into Fraud (Robert Fiddes fakes lab data, gets published)
- Feb 2000. Breast Cancer Researcher Admits Falsifying Data (Werner Bezwoda fakes data & gets results published)
- Jun 2001. How a cancer trial ended in betrayal (Harry W Snyder Jr & Renee Peugot fake data for profit, get published)
- Sep 2002. Scientific fraud found at Bell Labs by Linda A. Johnson (Jan Hendrik Schon's fraud at Bell Labs, numerous papers involved)
- Jan 2006. Lancet study faked: Investigation to probe all research conducted by scientist accused of fabricating results from 900 research participants (Jon Sudbo fakes out The Lancet, gets published)
- Nov 2008. Elsevier Math Editor Controversy (November 2008 issue of Chaos, Solitons & Fractals has five articles by notorious editor Mohamed El Naschie. He has over 300 articles in that journal overall.)
- May 2009. Elsevier published 6 fake journals (Elsevier journals sponsored by pharma companies)
- Oct 2009. Disgraced Cloning Expert Convicted in South Korea (Hwang Woo-suk is a famous case of fraud in stem cell research)
- Feb 2010. The Lancet retracts Andrew Wakefield’s article (Notorious "vaccines cause autism" paper by Andrew Wakesfield is retracted by Lancet)
- May 2011. Updated Cell Phone Study Deeply Flawed, Say Experts (British Medical Journal publishes flawed paper saying cell phone radiation causes cancer)
- Jul 2011. Marc Hauser resigns from Harvard (Hauser resigns due to scientific misconduct)
- Nov 2011. Report finds massive fraud at Dutch universities (Diederik Stapel fakes data in dozens of social-psychology papers, gets published)
- Jul 2012. Anesthesiologist Fabricates 172 Papers (Yoshitaka Fujii gets an unbelievable number of papers past mostly traditional peer review)
- Sep 2012. Top Canadian scientist and award-winning student caught in 'blatant plagiarism' of text (Dongqing Li and Yasaman Daghighi publish plagiarized paper in Springer journal Microfluidics and Nanofluidic, which Li edits)
- Retraction count grows to 35 for scientist who faked emails to do his own peer review (Journal of Enzyme Inhibition and Medicinal Chemistry author Hyung-In Moon did his own peer review)
- Feb 2013. Arsenic-based life paper: peer review process comes to light #arseniclife (The journal Science publishes a paper they hype to the moon on arsenic-based life that is shredded in post-publication social media commentary)
- Jun 2013. A Personal Academic Journal: Why is a major academic publisher printing a journal that seems a lot like the newsletter of the editor’s fan club? (SAGE journal Nursing Science Quarterly seems to be mostly about it's editor, Rosemarie Rizzo Parse)
- Feb 2014. Publishers withdraw more than 120 gibberish papers (IEEE and Springer withdraw 120+ computer generated gibberish conference papers)
- Mar 2014. Want to make sure your paper gets published? Just do your own peer review like this researcher did (Yongdeng Lei faked peer review for paper in Environmental Management, among others from Springer etc.)
- May 2014. The So-Called Sting (The journal Science runs a sting on various open access journals to see how their peer review holds up. However, they design their experiment so poorly that it would have failed any reasonable peer review process if they had published it as an article rather than "news")
- Jun 2014. Facebook’s Unethical Experiment: It intentionally manipulated users’ emotions without their knowledge (PNAS publishes paper that should never have passed internal ethics/peer review)
- Jul 2014. ISU loses $1.4 million in fraud case (Grant to study HIV vaccine rescinded due to Dong-Pyou Han's misconduct in faking results)
- Jul 2014. STAP stem cell papers officially retracted as Nature argues peer review couldn’t have detected fatal problems (Nature acknowledges limits of peer review in rush to publish papers in hot field)
- Jul 2014. SAGE Publications busts “peer review and citation ring,” 60 papers retracted (Huge scandal at Journal of Vibration and Control)
- Nov 2014. This Is What Happens When No One Proofreads an Academic Paper (Wiley journal Ethology has embarrassing lapse in "value added" service)
- Nov 2014. University of Regina prof investigated for allegedly plagiarizing student's work (plagiarized paper slips into ICE journal Environmental Geotechnics)
- Dec 2014. Elsevier retracting 16 papers for faked peer review (Author Khalid Zaman orchestrated series of fake peer reviews in Elsevier journals Economic Modelling, Renewable Energy, and Renewable and Sustainable Energy Reviews)
- Dec 2014. For Sale: “Your Name Here” in a Prestigious Science Journal (Springer journal Diagnostic Pathology may have copied articles, researchers used "paper writing" service which copied & pasted)
- Feb 2015. Journals without editors: What is going on? (unusually high percentage of papers in Elsevier journal Research in Autism Spectrum Disorders are by editor Johnny Matson)
- Feb 2015. What pushes scientists to lie? The disturbing but familiar story of Haruko Obokata (Fraudulent papers published in Nature.)
- Mar 2015. Major publisher retracts 43 scientific papers amid wider fake peer-review scandal (Springer-owned BioMed Central retracts papers due to faked peer review)
- Mar 2015. The games we play: A troubling dark side in academic publishing (Elsevier journals Research in Developmental Disabilities & Research in Autism Spectrum Disorders accused of gaming citation counts)
As noted above, this is the tip of the iceberg. Please see Retraction Watch for the rest of the iceberg.
And here are some books about academic fraud.
- 1983. Betrayers of the Truth by William Broad, Nicholas Wade
- 2004. The Great Betrayal: Fraud in Science by Horace Freeland Judson
- 2009. Plastic Fantastic: How the Biggest Fraud in Physics Shook the Scientific World by Eugenie Samuel Reich
- 2010. On Fact and Fraud: Cautionary Tales from the Front Lines of Science by David Goodstein
- 2014. Faking Science: A True Story of Academic Fraud by Diederik Stapel
And as a bit of a desert, let's take a brief look at who we should perhaps be considering predatory journals, those big commercial and society journals that soak the library world for every penny of obscene profit.
Oh yes, some resources from this blog and beyond that highlight some of the issues with the big, traditional journals, some of which are society, some of which are commercial. And finally, some resources about the real predatory publishing, the big commercial and society publishers who control so much of scholarly publishing.
This list is extremely partial. Please feel free to add your own suggestions in the comments.
- Mar 2014. Elsevier: bumps on road to open access: Academic seeks to gather examples of cases where open access article fees have been paid but content remains behind a paywall
- Apr 2014. The Sad Case of Jeffrey Beall
- Jun 2014. Around the Web: Your university is definitely paying too much for journals
- Oct 2014. Around the Web: Big Deals ‘R Us, or, Libraries in the lobster pot
- Mar 2015. Know the Difference: Scientific Publications versus Scientific Reports (American Chemical Society takes a swipe at PLOS ONE style peer review, which assess scientific validity rather than attempting to judge impact)
- Mar 2015. Elsevier Appears To Be Slurping Up Open Access Research, And Charging People To Access It
- Mar 2015. Wrongly paywalled articles: a recap of what we now know
- Mar 2015. Wiley are charging for access to thousands of articles that should be free
- Apr 2015. The Economics of Open Access
Hearing complaints and panic about predatory open access journals? Send them here for a hopefully more complete and honest picture.
(As usual, if I've mis-characterized or misunderstood any of the incidents or if I'm missing any significant items for any of the lists above, please let me know in the comments or by email at jdupuis at yorku dot ca. Hey, think of this as post-publication open peer review on this blog post. The wave of the future!)
Great read! I would differ a bit: I don't think the peer review systems is as broken as people often say, and some of your links about "problems" with peer review I don't really think reveal problems at all. Iaonnidis' paper, for instance (Why Most Published Research Findings Are False), has always bothered me, because I don't think we should expect every published finding to be "True" in isolation - that's a naive view of how science operates. I have a few thoughts about that at http://bit.ly/1wVSkmk , and I'm working on a future blog post that will tackle it more directly.
Still, specific carps aside, thanks for a thorough and well-thought-out post!
Thanks, Steve. I guess where I'm coming from is that I see the issues with predatory OA journals more as symptoms of larger issues in the scholarly communications ecosystem, one of which is dealing with the human limitation of a system like peer review. I agree that overall peer review is very important to science (and all scholarlship) and is something we couldn't do without.
Hello, I wrote the "A Personal Academic Journal: Why is a major academic publisher printing a journal that seems a lot like the newsletter of the editor’s fan club?" piece.
l enjoyed this post, but I do think that predatory journals are a real problem, not a mere moral panic.
I don't think it makes much sense to debate whether they are a bigger or a smaller problem than the issues in traditional journals. It's a different kind of problem. But you only have to browse Jeffrey Beall's blog to see how utterly outrageous the behaviour of these predatory publishers can be - and yet hundreds or thousands of academics (many from low-income countries) are paying them good money. That is a serious problem in my view.
Hi Neuroskeptic. Point taken. I agree that predatory journals are a real problem and one we should all work on to limit and control. The point I was hoping to make (and perhaps could have made better) is that taken in perspective, predatory journals aren't as big a problem as larger issues in managing peer review in an exploding publication landscape with "traditional" publishers being as guilty of lax standards as anyone. That, and we can also see perhaps the true predators are those rapacious publishers who such up so much of the money that's flowing in the ecosystem, a number that's often quoted as being in excess of $ 10 billion per year.
These are all important issues around predatory open access, but the "moral panic" may miss another important issue noted by Neutroskeptic.
Why are the journals so filled with articles from developing countries? Certainly there is an absurd publish or don't-graduate model in India and elsewhere, and yes, that generates a lot of rally bad research.
But I also see a great deal of potentially useful data being collected with some integrity by universities trying to do research with very limited means. This data isn't glamorous, or uses the latest equipment or techniques, but comes close to what we might consider really good undergrad level research data. Data, for example, related to agricultural observations might be useful in some bigger context, if someone with more means could collate it and analyse it properly.
I can't help but feel that as university populations and reputations grow in developing countries, that many researchers who mean well and just can't get their modest data in mainstream journals, are throwing a lot of good data to the predatory wolves.
So long as Western researchers have access to more expensive gadgets and money for methods development than non-Western researchers, the latter are always going to find themselves scrambling to catch up with constantly changing standards - not just for what methods must be used to generate and analyze data, but for what details must be incorporated into manuscripts, how data must be presented, etc. If they send a high-quality Western journal a manuscript of a sort that would have been published without question a couple of decades previously, it will meet with contemptuous rejection - and by the time they can meet today's standards, those standards will no longer suffice.
The root cause is the feeling of university administrations and bureaucrats that international, preferably Western presentations and publications are the ultimate mark of status. But publication in Western journals is not the best place for many non-Western studies. Data related to agricultural observations in Africa, to use Thad's example, would be of infinitely more value to Africans than to the readers of an American or Indian journal. Much better to publish them in a cheap, low-status African journal than to send them to a foreign predatory publisher. But this requires a social adjustment.
India seems to be getting good at this. I know of an Indian journal that publishes mostly rather terse papers of the sort that would have been considered just fine when the same field was at the same stage of development in Europe. The paper quality is terrible. I bet that means it is printed cheaply enough that Indian academic libraries can afford it. Not a lot of Americans read this journal, but then they don't much care what is happening in India.
Thad, Jane, Very good points. Thank you very much.
Thanks for putting all of this together John. We really appreciate it as it comes at just the right time for us and our efforts with OA advocacy and our OA press.
News from Brazil:
"The almost absolute silence of Brazilian researchers in relation to predatory journals is due not only to the complicity or the risk of embarrassment with those who use this type of publication. The biggest problem is the holes that they reveal in the the Qualis [database of the federal agency CAPES, of the Ministry of Education].
These holes expose an important part of the performance evaluation system as a whole. And, based on this system, in recent years, not only academic careers and reputations were built, but also investments were made and institutional priorities were established."
From: "The Qualis and the silence of the Brazilian researchers"
Last update: "Brazilian graduate system counts now 235 predatory journals)
I recently ended my tenure as the chair of our college of education personnel committee. My public U has over 25,000 students, so we're decently sized. Perhaps predatory journal stories are being overblown as you claim, but it was the single biggest issue I dealt with when we decided to deny tenure and/or promotion. Faculty on the margins who are desperate for publications are increasingly turning to predatory publishers where articles can be published nearly instantly without any review at all. I tried multiple times to create a policy against predatory journals, but this is much more difficult than it sounds and my efforts failed each time. Now, we are trying again using something like the criteria that Jeffrey Beall provides on his site: https://scholarlyoa.files.wordpress.com/2015/01/criteria-2015.pdf - The actual criteria are fairly good even if you don't agree with Beall's efforts (and I suspect you don't based on the inclusion of the article criticizing him from 2013). Our librarians used to be in the same boat, but just a few months ago decided that they would promote the criteria that Beall publishes as good practice for selecting journals for publication. In the meantime, we have denied a few people who went the predatory route, but it's hard to do when there is no policy in place. Perhaps there are problems in the more traditional journals as well, but those issues seem to be more anomalies than the standard practice like we see from predatory publishers.
Sean, I'm very sensitive to your situation. I don't mean to imply that predatory publishers aren't a problem, they are. But in your case the predatory publishers that are preying on your colleagues are a symptom of a deeper problem rather than the disease itself.
AASCIT are complete spammers beware. Recently i sent an article to the American journal of science and technology. After a bogus review they replied they have accepted my article. they asked me to pay article processing charges. The bank transfer address is Hong Kong while the journal is American! I transferred USD 200. After acknowledging the receipt of the money they refused to publish my article. I pleaded with them but since then they refused to reply. The address firstname.lastname@example.org. there is no person full name, no address, no contact. Only Gmail address. These are complete spammers. Do not ever send your journal to AASCIT. I have all the evidence of the communication i had with them. If need be i can give you the details.
Just because Elsevier et al. aren't capable of preventing fraud, doesn't mean they are comparable to the likes of OMICS publishing and similarly spammy, scammy and predatory publishers. In the first case, fraudulent and low quality publications do not constitute the majority of content published, unlike in the second case.
Elsevier et al. sometimes publish crap despite the fact that they try to prevent that from happening; OMICS publishing et al. don't really care too much about what they are publishing at all, as long as they get paid.
Notice that you've never seen Elsevier et al. accept blatantly ridiculous papers such as the widely-known "Get me off your fucking mailing list." one. The same cannot be said of *actually* predatory publishers, which seem to have even automated the manuscript acceptance process.
Also, the fact that you (a librarian whose native language is, I guess, English) can't seem to distinguish between "its" and "it's" does not work in your favor, I'll tell you that...
Elsevier may not have been so obviously pranked, but Springer and IEEE were: https://www.nature.com/news/publishers-withdraw-more-than-120-gibberish…
Not to mention that Elsevier also published fake journals: https://scholarlykitchen.sspnet.org/2009/05/07/elsevier-published-fake-…
Scam publishers are a problem. I've never denied that. But the real problem is the academic evaluation and incentive system that favours CV padding rather than quality research. That's why authors are such easy prey for these publishers.
Who is really harmed by predatory publishers? http://www.triple-c.at/index.php/tripleC/article/view/867