The usefulness of commenting on scientific papers

Here is a great example by Cameron Neylon:

It's a little embarrassing...

...but being straightforward is always the best approach. Since we published our paper in PLoS ONE a few months back I haven't been as happy as I was about the activity of our Sortase. What this means is that we are now using a higher concentration of the enzyme to do our ligation reactions. They seem to be working well and with high yields, but we need to put in more enzyme. If you don't understand that don't worry - just imagine you posted a carefully thought out recipe and then discovered you couldn't get that same taste again unless you added ten times as much saffron.

None of this prevents the method being useful and doesn't change the fundamental point of our paper, but if people are following our methods, particularly if they only go to the paper and don't get in contact, they may run into trouble. Traditionally this would be a problem, and would probably lead to our results being regarded as unreliable. However in our case we can do a simple fix. Because the paper is in PLoS ONE which has some good commenting features, I can add a note to the paper itself, right where we give the concentration of enzyme (scroll down to note 3 in results) that we used. I can also add a note to direct people to where we have put more of our methodology online, at OpenWetWare. As we get more of this work into our online lab notebooks we will also be able to point directly back to example experiments to show how the reaction rate varies, and hopefully in the longer term sort it out. All easily done on the web, but impossible on paper, and in an awful lot (but not all!) of the other journals around.

Or we could just let people find out for themselves...

Yes. If you find out after publication that you need to tweak your experimental protocol, the only place where other people interested in using your technique are guaranteed to find the new information is if it is attached to the paper itself.