Posted by By email, following on the heels of my post about the Merck-commissioned, Elsevier-published fake journal Australasian Journal of Bone and Joint Medicine, a reader asked whether the Journal of American Physicians and Surgeons (JPandS) also counts as a fake journal.
I have the distinct impression that folks around these parts do not hold JPandS in high esteem. However, it seems like there’s an important distinction between a fake journal and a bad one.
Kathleen Seidel of the Neurodiversity Weblog wrote a meticulous examination of JPandS and of the professional society, the Association of American Physicians and Surgeons (AAPS), whose journal it is. While you should read the whole thing, here I’m going to focus on just a few sections of her post that I think are germane to the question of whether JPandS is a real medical journal (albeit perhaps one with a quality control problem) or a fake one.
First, about the Association of American Physicians and Surgeons:
A political advocacy organization comprised of doctors, lawyers and others who support free markets and limited government, the Association of American Physicians and Surgeons defines itself as:
…a national association of physicians dedicated to preserving freedom in the one-on-one patient-physician relationship. AAPS members believe this patient-physician relationship must be protected from all forms of third-party intervention. Since its founding in 1943, AAPS has been the only national organization consistently supporting the principles of the free market in medical practice.
Some of the consequences of the organization’s commitment to free-market principles and to the primacy of the patient-physician relationship are not obvious at first:
AAPS opposes the concept of evidence-based medicine, warning its members that:
Physicians must beware of accepting the concept of a standard of care that is itself evidence-based, threatening the autonomy of physicians and subjugating the patient’s interest to that of the collective. (AAPS Newsletter)
[Evidence-based guidelines] are a divisive force, creating uncertainty and mistrust, and undermining confidence in physicians and our medical system. EBGs can be used either to accuse physicians of withholding therapy, or of prescribing unnecessary or unproven treatments. Behind the façade of EBGs, [managed care organizations] can determine medical policy with impunity. (Norman Latov MD)
Practice guidelines and treatment protocols are regarded as impediments to the unfettered practice of medicine.
There is a lot of pressure to restrict physicians’ treatments to practice guidelines and to methods that have been shown to be both safe and effective in double-blind controlled trials. If we were to insist on this across the board, a huge number of medical treatments that physicians rely on would be ruled out… If everything can be done with treatment protocols, then perhaps we should do without physicians altogether. Of course, we recognize that there are varying abilities, but I think it is a mistake to say that one small group, politically appointed, can sit in infallible judgment on just who the real physicians might be…. (Jane Orient, MD, Executive Director, AAPS)
That a professional organization would take political positions is not in itself problematic.
When the profession being organized is one like medicine, where there is supposed to be some involvement of the scientific method in generating the knowledge that guides practice, political lobbying may raise concerns if it is not kept distinct from the process of generating and evaluating knowledge.
This is where the journal comes in.
The original publication of the AAPS was Medical Sentinel. From the looks of it, Medical Sentinel was always presented as an outlet for AAPS positions rather than a disinterested collection of articles reporting medical findings.
Medical Sentinel contained a preponderance of political commentary on issues pertaining to medical practice. In 2002, AAPS decided to revamp their journal — giving it a new name, a more professional appearance, and shifting its editorial focus from political commentary to “more articles of a scientific nature, particularly if they are relevant to contemporary policy debates.” Sporting a new masthead, AAPS would devote an increasing amount of space to scientific articles reflecting the organization’s politics and philosophy.
The refurbished publication — Journal of American Physicians and Surgeons — was unveiled in March 2003. JPandS promptly distinguished itself for featuring articles offering scientific justification for the AAPS opposition to abortion. Assertions made in Karen Malec’s The Abortion-Breast Cancer Link: How Politics Trumped Science and Informed Consent, Brent Rooney’s Induced Abortion and Risk of Later Premature Births and Joel Brind’s Induced Abortion as an Individual Risk Factor for Breast Cancer have since have been disputed by numerous cancer researchers. JPandS continues Medical Sentinel‘s tradition of conservative political commentary on medical topics, and has also published articles on hyperbaric oxygen therapy, including its controversial use in the treatment of multiple sclerosis; pain management, opioids, and physician prosecutions; hospital pricing transparency; sham peer review; nutritional treatments for ADHD and herpes; low dose radiation therapy for cancer; and the effects of diagnostic and environmental radiation.
Looking to science for findings that might have relevance to policy debates is, one would think, a good thing — a step forward from latching onto particular policy decisions regardless of whether they are supported or undermined by out best scientific knowledge.
However, to Seidel’s eye, JPandS has a striking lack of articles with results that might complicate AAPS policy positions or call them into question.
Another pattern that Seidel notes is that JPandS authors and JPandS articles played prominent roles in litigation which turned on the question of whether there was a good scientific basis for claims of causation. Indeed, some JPandS authors were already involved in litigation when their articles were received; meanwhile, that an article was published in JPandS (a peer-reviewed medical journal) was used as a basis for asserting its credibility as a source of scientific evidence in a court of law. Seidel points out this tangle of agendas as it plays out in litigation around autism:
Initiation of legal action against pharmaceutical companies and government agencies in the absence of conclusive scientific evidence to support a causal connection between vaccines and autism — in the words of Mark Blaxill –
The blame game continues, as does the scientific inquiry. Numerous large-scale epidemiological studies have dismissed a causal connection between autism and vaccines. Clinicians with expertise in autism and toxicology have clearly distinguished between symptoms of autism and those of mercury toxicity. Concurrently, parent-advocates and sympathetic scientists under pressure to present legal proof of “general causation” — proof that did not exist at the time that any now-suspect vaccines were administered — have produced a multitude of articles positing a connection between autism and vaccine injury, for use as evidence in an adversarial judicial process, and have disputed any evidence that would undermine the attainment of their litigation agenda. An aggressive public relations campaign is also underway aimed at cultivating widespread public belief that autism is a consequence of poisoning, and thereby gaining political support for litigants.
AAPS has wholeheartedly facilitated this activity, which furthers its longstanding political agenda of undermining support for public health programs and promoting suspicion of vaccines. In the three years since its refurbishment in 2003, JPandS has published 21 articles promoting a link between vaccines and autism or death, some heralded by press releases widely-distributed by the AAPS and other organizations.
While there seems to be no shortage of scientific findings that challenge the hypothesis that autism is caused by vaccines or by mercury, there is a shortage of such findings published in JPandS. (One also gets the impression that there is a shortage of papers whose conclusions agree with those appearing in JPandS published in “mainstream” medical and scientific journals; I leave it to those with a better familiarity with this literature to chime in about whether this impression is correct or mistaken.) And, strikingly, the use of results published in JPandS in litigation, if successful, would further the AAPS political agenda. Meanwhile, the JPandS articles did not disclose the potential conflict of authors who were also litigants or paid expert witnesses.
Indeed, JPandS even published an article arguing that the law might be better the scientific method for the purposes of evaluating knowledge claims:
In their article, On Evidence, Medical and Legal Dr. Donald Miller and lawyer Clifford Miller (an active participant in the MMR controversy in the U.K.) assert the superiority of legal standards of proof over evidence-based standards of proof, causation, efficacy and practice generally accepted in medicine and science. They seem to imply that a single case study should be sufficient to prove causation of harm — and, by extension, legal liability — in cases involving many thousands of claimants with widely varying circumstances and documentation.
…scientific standards of proof are not uniform and well defined, in contrast to legal standards. Standards of measurement, ways of reporting and evaluating results, and particular types of experimental practices vary. As a result, there is no simple and reducible algorithm against which “good” science can be evaluated. There is another aspect of the scientific standard of proof that particularly impacts medicine. Science’s quest for objective certainty admits only a narrow range of evidence… With regard to uncommonly occurring and rare events like adverse drug reactions and vaccine-induced autism, judges need to realize that a CDR [child death review] case report and CD case series alone can prove causation to a very high standard… medicine needs to develop a better understanding of the nature of evidence and of evidentiary proof, by emulating law’s approach to evidence. Law in turn needs a better understanding of the shortcomings of medicine’s current approach to evidence….
Relaxed editorial standards and the haste to produce “science” that supports predetermined conclusions and offers a scientific rationale for a political and legal agenda, have been accompanied by some notable irregularities in articles published in JPandS.
At this point, let’s pull back from Seidel’s examination of JPandS and lay out our expectations for scientific and medical journals. (I understand that some will argue that medical research ought to be bound by somewhat different standards than scientific research, but I think it requires a positive argument to support the assertion that medical research shouldn’t live up to something like scientific standards of evidence and reasoning.)
- Manuscripts are subjected to peer review to ensure that the research reported was guided by sound scientific methodology. Minimally, this means that manuscripts include information about how the experiments were conducted (and how they might be reproduced by other researchers), a consideration of potential sources of error and a discussion of how these were dealt with, and probably some discussion about how the results fit into (or change our understanding of) the existing body of knowledge in the field.
- Serious efforts are made to identify, disclose, and/or manage potential sources of bias by the researchers (and the peer-reviewers), whether these stem from financial or other considerations.
- There is documentation that any research with human subjects or non-human animals was conducted according to recognized ethical standards and prevailing laws and regulation. (Among other things, this means that the IRB or IACUC at the researchers’ institution has approved the protocol and oversaw the research.)
- The journal’s policy with regard to which submissions are accepted for publication reflects a commitment to follow the evidence, even of that evidence should pull in the opposite direction from political ideology or standing policy recommendations.
There’s another expectation that I think is implicit: a good scientific journal will provide a good microcosm of the knowledge built by the scientific community in the particular area the journal takes as its focus. This means that if a matter is a source of legitimate controversy for scientists committed to using scientific methodology to understand it, the articles appearing in the journal will reflect this controversy. If, on the other hand, the scientific community reaches a point where a scientific matter is essentially settled (except for some of the details around the edges, perhaps), the articles in the journal will reflect that consensus.
Does JPandS live up to these expectations?
While articles accepted by JPandS undergo peer review, Kathleen Seidel and Orac have noted serious methodological shortcoming in them. If the peer review is not focused on the scientific quality of the manuscripts under review, on what basis are the peer reviewers evaluating the manuscripts?
To the extent that peer review is supposed to provide a level of scientific quality control, if the way it is implemented by JPandS fails to provide that quality control, there might as well not be peer review. Broken peer review can break the whole medical journal.
The failure of JPandS authors and editors to disclose potential conflicts of interest might lead the casual reader believing that JPandS articles are more objective and disinterested than they really are. This is at least misleading, if not outright deceptive.
Finally, the range of articles included in JPandS strongly suggests that submissions are subjected to an ideological filter that plays a significant role in which will be published. The microcosm of scientific knowledge presented in the pages of JPandS departs significantly from that presented by other medical journals. However, it closely tracks the preexisting ideological commitments of AAPS.
On this basis, it certainly looks like the credibility of JPandS, and of the articles published within it, is very shaky.
Operationally, I’m inclined to say that this makes JPandS no better than a fake journal. However, I can’t help but wonder if the people editing JPandS and submitting articles to it realize that this credibility gap exists.
If they are completely committed to a particular view of the world, fully expect that the evidence will support that view, and blame mistaken methodology for results that don’t support that view — indeed, to the point of openly rejecting evidence-based medicine and scientific standards of proof — the problem isn’t one of lying so much as arguing for a different standard of credibility.
The danger is that the outward appearance of JPandS might not alert the casual reader that this journal does not accept the standards that other medical and scientific journals try to set for themselves. If AAPS knowingly trades on this confusion to get unearned credibility, I think the case can be made for fakery. But even if no conscious effort is made to fool the reader, the outwards similarities may be enough to do harm.
May 6, 2009
May 6, 2009
May 6, 2009
May 13, 2009
June 21, 2009
August 16, 2009
September 21, 2009
October 14, 2009