A little science blogging quality control

Dave Munger has been spearheading a useful tool: Blogging on Peer-Reviewed Research, an aggregator and set of icons to be used on blog posts that are summaries of actual, genuine, peer-reviewed research. Read the guidelines; the idea is that when you see the icon, you'll know that the blog article is something more than an opinion piece, but is specifically an attempt to explain some specific technical research papers for the general reader.

This is not just an effort for those of us at scienceblogs.com — if you are an unaffiliated science person who explains research to the citizenry, use the icons. It'll help people recognize what you are doing, and when the aggregator is in place (I think the plan is to get it done in the next month or so) it will help readers find you.

More like this

Another plug in Uncommon Descent wont need...

And yet you just know it is going to start appearing on all sorts of creationist sites where it doesn't belong. Although if it got some pseudoscientists to actually read peer-reviewed scholarship, that would be a major accomplishment...

As a layperson fascinated with science, I think that's a fantastic idea, to flag articles meant to summarize technical articles for the general reader. James McGrath is right, however, that the icons will be abused by creationists and other nuts. I dunno how that can be prevented other than somehow make sure general readers understand beforehand what sites are legit and which are kooks. Y'don't want to license use of the icons - it wouldn't work anyway - so ultimately there has to be an intelligent word of mouth campaign behind this.

I think the point of the icon is not to ensure quality of the blog and blogger, but rather as a clear statement by people you're already going to trust, that this post is a review of peer-reviewed literature.

Would it make sense to differentiate this a bit? Like having a different icon for:

a. Reviews of well-established results in a field.

b. Reviews of new results that look good and got published, but aren't as solid as (a).

c. Critiques of new results that got published.

d. Summaries or reviews of an area of the field, referencing many peer-reviewed papers.

or some such thing? It's often useful to distinguish between research that's hot off the presses, and research that's well-established and solid, because from outside a field, it's often hard to tell the difference.

In my own field (cryptography), you might do something like

(a) Review van Oorschot and Wiener's paper on parallel collision search, which is solid and widely used.

(b) Review Wang's MD5 cryptanalysis from Eurocrypt a couple years ago.

(c) Critique Courtois and Pieprzyk's paper on algebraic attacks on block ciphers.

(d) Review the broad area of visual cryptography, starting with Naor and Shamir's paper that kicked the field off.

By albatross (not verified) on 29 Oct 2007 #permalink

You're all right, it's not about quality control actually, but about simply making it easier to find the specifically science-focused articles in the chaos of blogs. I suspect it will have to evolve somehow, though, because there will be abuse by creationists, and I'd hate to see the aggregator turned into advertising for bullshit.

That's a concern for sometime farther down the road, though.

We've thought a lot about the potential for abuse by creationists/other cranks. Once the aggregation system is in place, it will be a simple matter to remove posts and/or blogs that promote pseudoscience.

We will also have a system whereby readers can report abuses.

A secondary question is how much we'll crack down on actual use of the icon. If you use the icon to fraudulently label a post as covering "peer-reviewed" research, do we crack down on you for trademark/copyright violation?

Too bad there isn't one for:

Blogging on Idiot-Spewed Nonsense

By CalGeorge (not verified) on 29 Oct 2007 #permalink

I was rereading an old article on this site the other day and saw that. I think it's a good idea. Kudos.

unaffiliated science person who explains research to the citizenry

uh... how about humanities? there's peer reviewed research here too...

Dear Pharyngula,

I wrote your friend Coturnix (PLoS) about the benefits of public forums for publishing "anomalous" scientific data.

True to form, he responded as follows: "Peer review is not perfect, but it effectively keeps out pseudoscientists, quacks, and those who are then forced to peddle their wares on YouTube and horrendously designed websites." I think he was angry, doctor!

Re-reading your comments and the outburst of Dr. Coturnix, names like Semmelweis, Oliver Wendall Holmes, Wegener, Anna Williams, etc. enter my obviously empty head, rest awhile, and then remind me of the lessons of history.

It's not about having some silly icon that people will misinterpret. It's about having the data about the review in a structured form that the aggregator can recognize. Dave and those guys have mentioned nothing about this, and are letting misunderstanding spread.

In order to aggregate, you need to have some markup in the HTML of your post that identifies your post as a review, and possibly contains the citation info of the papers you are reviewing. For an example, see my reviews of stem cell papers in pubmed.

I still don't understand what makes this a better solution than the existing services provided by Postgenomic.