You may have been following the saga of intelligent design proponent Casey Luskin's use of the ResearchBlogging.org "Blogging on Peer-Reviewed Research" icon in a way that didn't conform to the official guidelines for its use.
The short description on ResearchBlogging's mission says:
Research Blogging helps you locate and share academic blog posts about peer-reviewed research. Bloggers use our icon to identify their thoughtful posts about serious research, and those posts are collected here for easy reference.
The guidelines for using the spiffy icon include registering with ResearchBlogging, something Luskin did not do at first in the post for which he used the icon. However, Luskin made changes that, in his estimation, brought his post into compliance with the guidelines. Did he succeed? And, is there any effective way for a community to enforce compliance to the spirit of its rules, rather than simply to the letter?
The main source of controversy about Luskin's post ("Leslie Orgel: Metabolic Origin of Life 'Unlikely'; Complexity Requires 'A Skilled Synthetic Chemist' ") seems to be about the extent to which he makes it clear in his posts which claims come from the peer reviewed paper he discuss and which claims are Luskin's. The ResearchBlogging guidelines include:
4. The post author should have read and understood the entire work cited.
5. The blog post should report accurately and thoughtfully on the research it presents. ...
7. The post should contain original work by the post author -- while some quoting of others is acceptable, the majority of the post should be the author's own work.
The idea here is that a blog post on peer reviewed research ought to reflect an accurate understanding of that research. The post should also add value to that peer reviewed research -- whether by explaining it clearly to a lay audience, or weighing in on how this research relates to other research, or saying something about the larger implications of the research. The "value added" might also come in the form of a critique of methodology, a disagreement with the conclusions the author draws from the research, or an argument in favor of alternative conclusions that could be drawn from the reported results.
There is no prohibition against the author of the blog post injecting his or her opinion into the blog post. However, the guidelines seem to indicate that this should be done in a way that doesn't undermine an accurate representation of what the author of the peer reviewed research actually says.
The tension between these two is where things get sticky in the Luskin case. Dave Munger walks us through it:
Does the post report accurately and thoughtfully on the research it cites?
Commenter Doc Bill gets to the heart of the matter:
Clearly, Luskin doesn't understand the purpose of Orgel's essay, that is, to discuss the plausibility of hypothetical nonenzymatic cycles.
Instead Luskin just makes stuff up, like this: "Again, Orgel essentially assumes that cyclic metabolic pathways are irreducibly complex systems that require a large number of parts in order to function"
Luskin replies:
Orgel states, "At the very least, six different catalytic activities would have been needed to complete the reverse citric acid cycle. It could be argued, but with questionable plausibility, that different sites on the primitive Earth offered an enormous combinatorial library of mineral assemblies, and that among them a collection of the six or more required catalysts could have coexisted." That seems to meet the definition of irreducible complexity.
Doc Bill then points out that Orgel doesn't specifically state that his objections to these processes means they are irreducibly complex, and that Orgel in fact never uses the term. I might add that Orgel's essay only addresses a few possible mechanism for these cycles; it doesn't argue that no other mechanism could possibly work, only that these proposed mechanisms probably wouldn't.
DiGz discusses another portion of the post:
"Just like the case of the ribosome, the evidence shows that the complexity of life requires an intelligent cause."
That's a conclusion based on his own pre-conceived notions and is not mentioned anywhere in the paper, nor could it be extrapolated scientifically from its contents. That's simply Casey making something up and as such it misrepresents the content and conclusions of the paper.
Luskin contends that that's his own opinion and not relevant to his accurate reporting on the facts of the article. He is correct in pointing out that the guidelines do allow bloggers to state their own opinions -- we don't restrict bloggers to a bland reporting of the facts. However, I'm troubled by the statement DiGz quotes because it's unclear that it in fact represents only Luskin's opinion.
I talked privately with science ethicist Janet Stemwedel, and she agrees that the blurring of the distinction between what's supported by the article itself and what constitutes the blogger's personal opinion is problematic. In this case, when Luskin refers to "the evidence," what evidence could he be referring to, if not the evidence supposedly offered in Orgel's article? Yet Orgel does not present any evidence that "the complexity of life requires an intelligent cause." The blog post itself should make it clear that this final assertion is Casey's alone, not Orgel's. We shouldn't have to wait for Luskin's assertion that this was only his personal opinion, especially when he still hasn't modified his original post to make that clear.
Given that Dave mentions a conversation he and I had about this, I figured I should expand on that a bit.
There seems to be blurring of the lines in Luskin's post between "what the article says", "what the data from the article demonstrates", and "what I make of the data and conclusions presented in the article." In general, these are lines that we want to make as sharp as possible, although in practice this may be easier said than done.
As I understand it, a central goal of the ResearchBlogging project is to clarify for the reader of the icon-bearing and aggregated blog posts what bits of knowledge and interpretation were "certified" by peer review. These are distinct from the opinion, critique, and alternative interpretation that the blogger brings in as his or her original work.
Sometimes an author's explicit conclusions seem to fit so well with our own (slightly different) conclusions that we don't realize that we've arrived at slightly different conclusions from what the author said. Sometimes particular conclusion from a given set of results seem so obvious to us that we don't notice that the author whose results we draw them from didn't arrive at those conclusions. Which is to say, it's possible unintentionally to blur the line between what the peer reviewed article actual says and what we got out of it.
On the other hand, the circumstantial evidence (of the Discovery Institute's track record and of the ID/creation science crowd's dismissive attitude toward peer review) suggests that the blurring in this case is intentional rather than merely sloppy -- that it's not Luskin clearly putting forth his own view as his view, but trying to smuggle his view under the umbrella of the knowledge and interpretation certified by the peer review of the article on which he's opining.
The big question, for the ResearchBlogging community, is how to distinguish unintentional ambiguity from arguing in bad faith.
I doubt that this is the kind of thing that can be accomplished with a clarification of rules. The more precisely you formulate the rules, the more likely it is you rule out a post that argues in good faith while simultaneously admitting bad faith arguments that have found a loophole in the rules.
Rather, I think a healthy and continuing discussion within the community, while imperfect, is the best option. The community is defined by a common goal, but members of the community seem capable of voicing vigorous (and reasoned) disagreements. Members of the community seem ready to hold people to account for living up to the spirit of the rules as well as the letter.
People who try to associate with the community and partake of the credibility that comes from its standards won't be able to weasel out of those standards without being called on it publicly. At that point, these people will have to decide whether to become real members of the community who have to engage with the rest of the community seriously, or whether to slink off to the communities whose values they really endorse.
There's no automatic way to keep the interlopers out. But a community with values worth defending will have members willing to poke the interlopers with sticks until they come along or get out.
- Log in to post comments
One of the common aspects of Evolution Denial and HIV Denial is a pattern of deliberate misrepresentation of "orthodox" papers in an attempt to support their own agenda.
AIDS Denialism and the Misuse of Scientific Studies
I think the litmus test is to ask the question: "Would the authors of the study agree with my interpretation?".
Several authors of "orthodox" HIV papers have been forced to denounce misinterpretations of their papers. Of course in Casey's case he chose an author who is no longer capable of doing this.
If the authors of the paper would not agree with your interpretation then you are obliged to clearly differentiate your conclusions from those of the authors and justify exactly why you reach different conclusions.
HIV Denialists generally counter by claiming that they are simply looking at the evidence and that the author's conclusions do not necessarily follow from the evidence. However, the author of the paper is much more familiar with the evidence in the paper than the person writing about it. It is up to the critic to demonstrate that their conclusions are better than those of the author.
I'm not going to disagree with the analysis of Luskin's post, but I think this entire issue should be viewed from an entirely different angle. The act of analyzing Luskin's post to see if it meets the guidelines is based on the premise that the post is worthy of consideration at all given the very obvious elephant-in-the-room context of who write it, why (we presume) and where it was posted. To analyze the value of the post itself is to remain willfully ignorant of this context.
Luskin is a creationist, the DI web site is a creationist web site, and the method de jour of creationists is to pretend like they are not creationists but rather just more scientists looking at this whole evolution thing at a different angle.
BPR3 is a ... what is it ... a thing on the web that links peer reviewed research in science journals with science bloggers who are writing about that literature. Luskin is a creationist writing on a creationist, anti-science web site pretending to "fit in." The Luskin post begins and ends with an utter lack of credibility, and with a proven track record (amongst the broader ilk of DI operatives) of repeated ethical breaches and overall dishonesty. The Luskin post need be given no further consideration than deletion from the BPR3 database and a letter from a lawyer asking him to not use the symbol (which may or may not mean anything, but letters from lawyers should be sent to DI whenever possible in order to run up their legal fees and help shut them down).
Although I don't necessarily disagree with the analysis, I'm rather disturbed that we are even having this conversation! Just tell them to screw off, I say!
I have further babbling on the topic here:
TinyURL was created!
The following URL:
http://scienceblogs.com/gregladen/2008/02/luskin_and_the_dis
covery_insti.php
has a length of 76 characters and resulted in the following TinyURL which has a length of 25 characters:
http://tinyurl.com/35cokb
[Open in new window]
Or, give your recipients confidence with a preview TinyURL:
http://preview.tinyurl.com/35cokb
[Open in new window]
How to copy and paste the TinyURL: To copy the TinyURL to your clipboard, right click the link under the TinyURL and select the copy link location option. To paste the TinyURL into a document, press Ctrl and V on your keyboard, or select "paste" from the edit menu of the program you are using.
http://tinyurl.com/35cokb
Litmus test for what? Willingness to toe the received line?
Bah. What are the BP3 people afraid of? If a blog post is commenting on a peer-reviewed article, then it's fair game for their icon -- I don't remember them mentioning anything about "only posts by people we select as worthy", or "only commentaries we agree with".
I think Luskin is an idiot in general, and completely wrong in this instance. But unless BP3 wants to become another invite-only circle-jerk, I think they ought to let Luskin in and then do the work -- take public issue with his errors, make sure he can't claim that people agree with him who don't, and so on.
Janet wrote
That implicitly endorses suggestions that posts displaying the icon should allow reader comments and trackbacks. Absent the ability to comment on a post, critical discussion is inhibited and invisible to readers of the post. ID Web venues like Luskin's are notorious for disallowing critical comments. Luskins' venue allows none.
Luskins' venue allows none.
This is hardly an issue when the BPR3 thingy is all about aggregation in the first place.
These are two symbiotic groups of self-promoting loudmouths. The moment I read about this "peer-reviewed" icon business, it was obvious that exactly this scenario was going to play out. I'd be amazed if that wasn't the plan to begin with.
a community with values worth defending will have members willing to poke the interlopers with sticks until they come along or get out.
The problem with that scenario is that it assumes getting poked with sticks will deter the interlopers. If their agenda is A) to diminish the value of the BPR3 logo and/or B) to play the "Expelled" card: loudly and publicly claim that any contrary view is automatically excluded----then being poked with sticks is a reward rather than a deterrent.
For pubjacking. There's a difference between making valid critisms of a paper and blatantly misrepresenting it. Sometimes there is a fine line between the two but Casey's case it's more of a wedge.
It is not possible to post comments on EN&V. The only way Casey's readers, if they were inclined to, could find these comments would be to visit ResearchBlogging.org and search for some comments. Most of his readers would remain ignorant that there was any criticism.
The implication of such a litmus test has nothing to do with whether one agrees or disagrees with the study's methods or outcome. Rather, it has to do with whether one understands the study and describes the authors' presentation accurately. I don't think such a standard must be absolute, but certainly gross misrepresentation of the authors' words or presentation of ones own conclusions as those of the authors falls well outside the line.
I think the litmus test is to ask the question: "Would the authors of the study agree with my interpretation?".
Chris, I think that is a deeply problematic proposal, simply because it would preclude genuine, scientifically-knowledgable bloggers from being able to expose bad work. Bad work does get published sometimes, and authors will bend over backward to defend it sometimes, so not being able to "legitimately" blog about exactly how and why work is wrong would actually remove one tool that the science blogging community has to raise standards of work and police themselves.
But unless BP3 wants to become another invite-only circle-jerk
When you buy a piece of electronics you have a CSA stamp, or equivalent, that says that the design and manufacturing have met a minimum number of standards and won't set your house on fire. Would you allow anybody to use the stamp at will?
Sometimes an "invite only circle jerk" is a good thing. If you have no standards, or fail to enforce the ones you supposedly have, then your imprimatur is completely worthless.
You are correct. Any sort of demarcation problem is problematic.As a single criterion it would preclude much genuine criticism.
I really meant it as a first step. If the answer to the question is no then you should, if you are honest, ensure that.
a) You honestly describe the aims, context, data, and conclusion of the paper.
b) You clearly separate your own views, prejudices and interpretations from those of the author.
c) You demonstrate that you have a reasonable knowledge of the field
d) You justify exactly why you think the author is wrong and why somebody should take your interpretation over that of the authors.
I think these are fairly self evident to most working scientists but not to Evolution Denialists and HIV Denialists.
Denialists are typically so self deluded that they fail to conceive how anyone could possibly not see things through their own prejudices. Denialists such as Casey Luskin typically delude themselves that they have competence in fields that they don't.
I think that the question "would the authors of the paper agree with my interpretation" should serve as a reality check for would be pubjackers.
The expanded version covers the problems admirably, I think; I could go with that. But the thumbnail version, even as a shorthand, would leave it open to the exact objection I made, being made by Luskin and that ilk. I would suggest not using the phrase at all; just, perhaps, saying
--If you choose to reject the conclusions of the paper, you should:
...and then include the four steps you describe. More wordy, but considerably less open to abuse.
Brief isn't always better. ;-P