After talking to Cameron Neylon last week, I'm strongly considering setting up an online lab notebook for my research lab. Not so much for the philosophical reasons having to do with openness and the like-- as a practical matter, I still don't think my data will do anybody any good-- but for reasons of sheer convenience. Having the lab notebook on the web will allow me to keep tabs on what's going on during the next few months when I'm going to be spending a lot of time at home with SteelyKid.
The one catch is, the system Neylon uses for his lab blog is optimized for, well, a biochemist-- someone who has a fixed set of instruments, and mostly just needs to keep track of what samples were fed into what instruments. The work I have going on at the moment is largely apparatus construction, and thus the existing lab-blogging tools don't work terribly well to capture what's going on.
Specifically, I would like a way to easily include small drawings-- block diagrams of the apparatus, beam line sketches, and so on. I posted pictures of pages from the lab book a couple of years ago, and that's fairly typical stuff.
So, a question for the audience here, many of whom have more knowledge of this sort of thing than I do: Are there tools out there that do what I need already? Specifically, I would like the ability to make small electronic drawings of the lab apparatus (basically, the tool set available in Paint or Word would be fine), and post them to a lab-book blog in a relatively seamless way (where "relatively seamless" involves fewer steps than making the drawing in Paint, exporting it as a .jpg, uploading it to a blog server, and pasting the link into a file).
I'm looking into the possibility of getting a local CS student to put something together as a student project, but if there's some cheap and easy pre-existing solution that would work as an interim measure, until I can get something custom made, I'd love to know about it.
- Log in to post comments
More like this
You could also take this up on the forum at Nature Network. There has been quite a bit of discussion about these things over there.
Come over to the Dark Side. It is your Destiny.
A whiteboard in AJAX or Flash is probably the kind of thing you need but while the technology exists I'm not sure it's available in a way that would fit your needs. As an example there is a nifty tool at http://imaginationcubed.com/ but you can't save the drawings on your own web site, you don't own them, and there's no assurance they'll be available at some future date.
Assuming you can find that CS student to do the coding there are some examples out there - a search for "whiteboard" paired with "AJAX" or "Flash" will turn them up. I'm sure that someday soon there will be a nice open-source AJAX whiteboard available, maybe your CS student will write it.
Why not MediaWiki? Then you can typeset your math in LaTeX as well.
I've been keeping my labbook in LaTeX, but seriously considering switching to keeping looseleaf by hand and scanning it to djvu, but my handwriting is excellent. The only downside is you can't search for terms in the pdf, but now that I think about it, I've never actually done that...
Hi Chad, this looks serious! I think very few of the things out there at the moment really have sketching built in - which inevitably leads to the 'save as image, upload image, link to image' nightmare. It will no doubt come as different interfaces become more common but currently it's pretty ropey.
It occurs to me that one thing that could well fit a lot of your needs is Microsoft's OneNote which is available in different versions of Office. Its set up as a journaling system but enables you to type, sketch, drag and drop stuff from web/other programmes pretty effectively.
The one thing it does not do is slap it up on the web but you might find that it allows an 'export as html' button conceivably. I have a vague recollection that LiveJournal had some nifty drag and drop functionality that might at least allow you to copy and paste the OneNote file as an image into a blog-like thing.
Still think your best bet may be a scanner though...the problem with any sketching doo-dah is that it doesn't really _feel_ like paper and pen... :-)
I suggest a collaboratively edited wiki. There are many. TWiki, moinmoin, dokuwiki, etc. I'm a fan of TiddlyWiki but collaborative editing might be hard there. Check out the WikiMatrix (www.wikimatrix.org) for a mind-numbingly large tour of all the different kinds of wikis. I would avoid MediaWiki. While popular it's a pain to set up and overkill for most purposes.
Meh, forget the Dark Side. ;)
Adobe's Flex/Flash makes constructing the whiteboard fairly easy as previous commentors have mentioned. ActionScript 3 has a couple of functions that allow one to grab a snapshot of the whiteboard, convert it to a byte array and send it to the a server-side process -- no messy file uploading!
Once at the server, the image(s) can be stored in the usual manner.
FlexBuilder is free for .edu folks and the SDK is open.
Chad --
The one I use and love was developed in TiddlyWiki and was created as an online notebook of theoretical physics by Garrett Lisi http://deferentialgeometry.org/ It comes loaded with latex and lots of additions. -- Try out his latest addition, an elementary particle explorer. Best of all, it fits easily on a thumb drive so I take mine with me everywhere.
I'd like to second the suggestion of a wiki, especially as most wikis create page histories and track editing by various users. I particularly like PmWiki: easy to use, can use either hierarchical or free-form structures, and uses text files rather than a database to store its content. I set one up for our group about a year ago and it's been fairly useful.
It also has a plugin called PmWikiDraw, a Java application which allows collaborative image editing and attachment to pages. It's not perfect (vector-based only, I think) but I think it could produce the kinds of diagrams you need. See http://www.pmwiki.org/ and http://www.pmwiki.org/wiki/Cookbook/PmWikiDraw .
Chad --
Oops. Should have read a few posts back:-( As far as drawing objects goes try Autodesk's DRAW http://draw.labs.autodesk.com/ADDraw/draw.html You can edit and save images locally, right to your wiki or keep them on the draw site. There may be some other AJAX drawing programs around, but I don't think many are as mature.
Actually a nice whiteboard app that talked nicely to the MediaWiki API would be very cool. The people over at OpenWetWare would probably also be interested in such a thing to integrate into their lab book system. Doubt there is any money in it but would provide the putative CS student with a wide range of potential users.
If the wiki approach appeals then OWW is a good place to start in any case. TiddlyWiki (otherwise known as FiddlyWiki) I wouldn't really recommend because if you're starting to store graphics heavy content they get very big very fast - which means they slow down and you lose the one major advantage (which is carrying it a round on a pen drive).
MediaWiki has the best API I believe which if you want to do something generally useful is valuable. Others are easier to use but probably less powerful. Depends on the pain:gain ratio really. If you do it through an API theit ought to be relatively painless and the same whiteboard could talk to other services with a minimum of modifications.
I second the suggestion for OneNote. I use it routinely and it is amazing for physics notes and collaboration.
Windows Live Writer lets you copy/paste images into your blog posts, if it'll post to your blog software.
Agreed with the others that a Wiki seems to be the right tool for the task, moreso than a blog. I'll refrain from commenting on which specific wiki software, as my advice would be useless to you. (I am a computer scientist, so I'm more interested in how "hackable" a wiki is rather than ease of use for normal users... :))
Chad,
I've been looking for such a beast for about 5 years, both in connection to my graduate work as well as to use in my own lab now. I've found nothing so far. I do, however, have a list of things I want:
1) Sketches (preferably handwriting input, hand-drawn graphics, and handwriting recognition)
2) Indexing/searching- one of the beauties of a digital lab notebook is that it could keep all the data organized and searchable
3) Flexability- like you, I don't have a set "instrument" that I use for my research- my needs change, expand, and grow and I need to be able to add components to the notebook without too much hastle
4) File connections- like iTunes manages the music files on a computer's hard drive, I'd like a digital notebook that did the same for all my data files (of a variety of types from Excel, LoggerPro, jpg, LabView, specialized CCD camera driver pictures, etc)
5) Typesetting- some kind of integrated LaTeX driver to read my scribbles, interpret it, typeset it, and save it (preferably in some xml-like markup language with metadata).
6) An on-line interface with multiple users so I can teach students to use the notebook and be able to monitor changes to the notebook on a regular basis
7) Portability. Whatever gets used must be as portable and reliable as my sturdy brown lab notebooks. Anything less and I'd be spending more time on the notebook and not on the science.
So far, there is nothing that will do all of this. The technology exists in various forms to do all of the pieces, but I can't get anyone to put together the big package to make it work for me. Good luck.
I should note that I'm thinking blog rather than wiki for two reasons:
1) Ease of set-up: Kate can easily install WordPress on web space that we already own. And the WordPress interface is simple enough that I'm not overly worried about my students being able to use it, or my ability to explain how to use it.
2) The nature of the material: I think of a lab book as a historical record, which strikes me as better suited to the blog format than the wiki format. I don't want to edit a single page showing what the apparatus looks like now, I want a set of pages showing that it had this configuration on this date, and what was done between that upgrade and the next.
You can get that out of a wiki set-up from change tracking, but it seems to be a more natural result of the blog format.
I'm going to start out with a WordPress blog on a steelypips site, and see what I can find by way of add-ons or work-arounds to do diagrams.
Just one word of warning about the standard Wordpress install. I think in the latest version while there is some versining of posts it isn't very robust - this means you can do things like change the dates on items. Now this may not be an issue if you're not going to be going into court with it anytime soon - but it rubs some people up the wrong way.
If you're trying to pin down when a student did something, the fact that dates can be changed by the user may be a worry. But you may be able to at least remove this from UI on your own install.
The nature of what I'm working on makes it very unlikely that I'll be involved in any priority debates any time soon, but that's a good point about the dates. I'll have to check up on that.
Thanks for starting an interesting discussion. It seems like many people are using (or considering using) electronic lab notebooks. I can see some reasons why, but I wonder what some of the pitfalls might be. For example:
How long will they last compared to pen and paper notebooks? (Or, what would I need to do to ensure an electronic notebook will survive disaster?)
What legal issues might arise from a multiuser wiki based notebook?
Can electronic notebooks be used in patent applications?
I'm sure there are others, and that folks who use electronic notebooks have answers to these questions and ones I haven't asked. I'd be interested to learn more.
Hi Brian
Couple of quick answers:
"How long will they last compared to pen and paper notebooks?" - As long as you look after them, much like a paper notebook. It is also worth asking how useful a paper notebook is after a few years - can you actually find anything in it? If so you keep a better record than most. The key issues for ELNs are backups and checking the file formats can still be read. Backing up a static html dump should be a pretty good strategy for the forseeable future. Storing data is a bigger problem but that is a problem anyway as the majority of it is already digital.
"What legal issues might arise from a multiuser wiki based notebook?" - None as long as all users are from the same organisation. Or conversely as long as their is a robust record of who wrote what. Let me put it this way 'What are the legal issues that arise from a multi-user paper notebook?'. Really exactly the same question.
"Can electronic notebooks be used in patent applications?" - Many big pharma now exclusively use ELNs in some parts of the organisation. I am sure they wouldn't do that unless the lawyers were happy. The important point is that to be solid in court you've got to have absolutely water tight provenance, date stamps, and be able to prove that no-one fiddled it. Depositing MD5 hashed files on a third party server with a third party time stamp goes a long way to achieving this.
One point, particularly for academics, is that the usual paper notebook, as it is kept in most research laboratories is of almost no use whatsoever in a patent dispute. They are not in most cases part of a robust enough system to survive the discovery phase of a patent case. particular issues are the types of notebooks used, the signing off procedure (and evidence that this is part of a clear written policy which is actually followed), and the completeness of the record.
Hi, Chad
My friend sent me a link to this discussion. I think we may have a solution for you- web based electronic notebook software, which has been designed to resolve most of the questions discussed here. If the interest still exists, please let me know, so we could discuss it further.
Ramil Menzeleev
Kalliste Systems, Inc.
Commercial Electronic Laboratory notebooks (ELNs) are becoming increasingly popular and important because of government regulations such as 21CFR11, which specify how records must be created, digitally authenticated and archived in order to be admissible in legal or regulatory proceedings. It's true that there are open and free solutions out there, but the commercial ones have a far greater interest in making sure that your research is recorded in a fully compliant way. One example of a modern ELN that is optimized for life-science research is CERF by Rescentris Inc. CERF is both Mac and Windows compatible and fully 21CFR11 compliant. It includes advanced features such as semantic metadata searches and customizable access permissions that help PIs to manage and track all the activities in a modern lab staffed by many scientists working on different problems. Long term storage is in .PDF format - guaranteed by adobe to be supported for at least 75 years. CERF recently added software-as-a service remote server hosting, so it is no longer required for labs to maintain their own dedicated server in house. Sign up for a free webcast demo of CERF at www.rescentris.com.
Quite impressed with the open lab notebook idea its good to have one like this cuz when i was doing my research i used eNovator electronic lab notebook & its good too.
We use at our lab free e-notebook, and it work well to our group.
check at www.sparklix.com