• An important biology dataset is losing NSF funding and may fold. Nor (as the article explains) is it the only one. It is impossible to overstate the desperate gravity of the data-sustainability question. Academic libraries, if we are not the white knights here—and we certainly have been in the past; witness arXiv—who is?
• On a similar theme, Yahoo pulls the plug on GeoCities. O ye researchers relying on consumer-grade web services, or new startups, have an exit strategy! Consumer-grade services die when they lose money. Jason Scott may not come charging to your rescue.
• H1N1 science depends on a public database of flu immunity data. "As the researchers acknowledge in their paper, the work couldn't have taken place if it weren't for extensive data sharing within the community of flu virus researchers." Data sharing makes possible better, faster science.
• Data and the journal article. First: if you are saving your data as PDF, stop it. Second: as I suggested to Chris on FriendFeed, there's a serious structural issue with expecting journal publishers to cope with appropriate data archiving: by the time a researcher chooses a journal to publish in, all the decisions about data gathering and representation have already been made—and they may well have been made badly. The poor journal publisher can't go back in time and fix bad decisions! In our not-yet-standardized data age, early data interventions have to happen close to the researcher, which to me means they need to happen at the institution where the research happens.
• The need for clear data licenses. I haven't talked about data licensing here, partly because the current state of intellectual-property law makes me sick at heart, but there's no question that it's an important piece of the data puzzle.
• Peer-to-peer technology used for the forces of good: BioTorrents. Datasets vary in size; for the large ones, network latency becomes a sharing problem. Torrenting won't precisely solve the problem, but it certainly increases the size range within which datasets are portable.
• Fascinating data project of the week: National Center for Ecological Analysis and Synthesis. What caught my attention is that as I read the project description, it takes public data sharing for granted. NCEAS researchers are not generating data; they are mining existing data. I'm inordinately curious about the disciplinary culture that makes this a feasible thing: what price scooping?

Whew. I have a lot more, but it's Friday.

You have digital data. You think it's important. We'll start from there.

• Your grant runs out… and then what?
• The graduate student who's been doing all the data-management chores leaves with Ph.D in hand… and then what?
• Your favorite grant agency institutes a data-sustainability requirement for all grants… and then what?
• Your lab's PI retires… and then what?
• Your instrument manufacturer or favorite software's developer goes out of business… and then what?
• Your whomped-up next-door data center burns up, falls down, then sinks into the swamp… and then what?

You get the idea. No far-fetched catastrophizing, just all-too-plausible scenarios that researchers really ought to have thought about already but usually haven't. If your service can position itself as the "… and then what," you're on to something.

What I read in the area and think is worthwhile enough to keep around ends up in a few places, all of which have RSS feeds:

• the Data Curation folder in my Zotero (you may also be interested in the Digital Humanities or Digital Preservation folders)
• the toblog and datacuration tags in my del.icio.us (items in the "toblog" tag end up in tidbits posts here—usually)

Happy to share these, and also happy to start up a Zotero group if anyone else is interested in contributing items thereto!

(By the way, one rather annoying thing about the Zotero feed—I almost always save copies of the item along with the item record, and Zotero dumps both into the RSS feed, which from the consuming end looks like a lot of unnecessary duplication. I apologize for this, and wish Zotero would fix it.)

Stripped to its essentials, this article is about carrots for data sharing. Acknowledging that cultural inertia (some of it well-founded) militates against spontaneous data sharing, the authors suggest a way forward.

I'm calling this one out because it has implications for storage-system design. The authors want three things for their public data: persistent identifiers, citation mechanisms, and data usage information.

(For once, I feel good about institutional repositories: they swing two out of three at the minimum, and some manage all three!)

Persistent identifiers seem simple but aren't, necessarily. For example, does a constantly-changing dataset get a persistent identifier? How does that identifier know what it's identifying, in that case? Should a persistent identifier be just a URL? What if the domain name goes away or changes? (This is not an idle concern; the University of Illinois, for example, just changed its top-level domain, and the institutional repository I run is eventually going to lose its separate domain entirely.) What, exactly, gets a persistent identifier? The entire dataset? Files within it? Should a query performable on that dataset also be persistently identifiable? How does that work, exactly? And when does something get its persistent identifier? As soon as it hits the system? Or after it's done and blessed, if it ever is?

Anyway. All of this needs to be hashed out (so to speak). It's not optional, system designers.

Once that's sorted out, citation isn't actually a huge hurdle from where I'm sitting. It's not a technical problem; it's kicking the style manuals into acknowledging data and making citation formats for it.

Usage, now, that's a hurdle. It, too, is utterly necessary for cultural reasons, however. The culture of academia looks kindly on impact measurements, even hopelessly faulty ones. Somehow or other, research impact has to be measured for researchers' careers to advance. Data are no exception.

(In my professional neck of the woods, systems designers ignored the need for usage documentation entirely too long, which has made my life as an IR manager extraordinarily difficult. I make this post in hopes of avoiding the same mistake in this new arena.)

What counts as a "use" exactly? How does "use" get harmonized over different kinds of access schemes? How does an API "use" compare with an entire download?

I don't know. I encourage systems designers not to get too hung up on such questions. Record all accesses and make the best decisions you can right now about how to present them. Yes, you'll have to rewrite the event-analysis code, probably more than once, so comment it well.

Do not, however, wait until you have all the answers to write an analyzer. If you do that, you're strangling the open-data movement in its crib. BMC Bioinformatics explains why.

Chris Rusbridge has blogged the program.

(If I seem more scatterbrained than usual, it's because most of my spare time and brainspace is currently devoted to building a course I will be teaching online in the spring for Illinois's GSLIS. It's a "Topics in Collection Development" course, which means I have to view things through a lens I'm almost completely unfamiliar with—I don't do normal collection development, and most of what I know about it is that it scares me to death! I am designing my version to be "how coll-dev is currently changing and may continue to change." Data curation will be included, as will scholarly communication and the serials crisis, institutional repositories, digital collections, digital preservation, and similar things that I actually do know something about. Wish me luck. I will need it.)

I've gotten some good comments to yesterday's poll. Please keep them coming. I know there's more out there!

I want to understand what's going on where with data curation specifically at the institutional level (no NOAA, no ICPSR, none of that) Stateside. Grant-funded is fine, though I'm doubly curious about programs that have been weaned (or are weaning themselves) off the grant money. Here are the programs I know about offhand:

• Institutional data curation: San Diego Supercomputer Center (right? I'm not entirely sure what they offer vis-a-vis long-term data stewardship), Purdue's D2C2, Cornell's DataStaR.
• Subject-specific but still (mostly?) institution-focused: Cornell's CUGIR (there must be a lot more GIS out there, mustn't there?), North Carolina's DRYAD
• Data-curation training: Illinois, North Carolina.

Tell me what I'm missing, please and thank you.

Is there an XML verification tool not subject to this particular tail-chase? I don't know of one offhand.

Perhaps this collaboration instinct will help libraries respond to the challenge of domain expertise for data curation. Do I need to know cheminformatics, or do I just need to buy a cheminformaticist conference potations until I secure her business card?

Formalizing expertise-sharing arrangements strikes me as rather difficult. Nobody wants to be the person everybody across the country calls with questions about ChemML; when would there be time to get any work done? Still, I would have thought that collaborative collection development had too many moving parts to be practical, and it's being done.

In any case… I have "develop network of domain experts" in the back of my head as a wise thing to do.

• An extremely nerdy but (for nerds) fascinating examination of XML and its implications for data modeling. Do we have to reduce everything to a relational model? Really? Perhaps not… Notably, it seems to me, this article describes fairly nicely how Fedora works. (For more beating on the humble RDBMS, see this blog post.)
• White Dielectric Substance in Library Metadata. "Understanding the noise turned out to be more important than understanding the signal." What does that mean for efforts to decide which data to preserve? "I've observed that most people trying to collect metadata go through an early period of thinking it's easy, and then gradually gain understanding of the real challenges." So have I observed this, Mr. Hellman, so have I.
• An empirical study of data sharing by authors publishing in PLoS journals. My reaction split neatly in half: half "data are doomed because no one who makes them will lift a finger to save them" and half "surely this could be easier?"
• Steve Hitchcock on how data will change library-managed repositories. My best wishes to Steve as he makes his vision real.
• The Fourth Paradigm (PDF). Microsoft's toe-dip into the data waters. Useful case studies for those thrashing about in planning processes.
• Writing math on the web. Because it still makes my head explode that the Web was nominally designed to exchange physics papers, but the math-display problem didn't seem to occur to any of its architects until years later.
• And finally, an amazing data project: the Ocean Observatories Initiative.

That should keep everyone out of trouble a while…

A warning to avoid preconceptions: If you pay too much attention to all the cyberinfrastructure and e-science hype, it's very easy to fall prey to the erroneous notion that most of science is crunching massive numbers via grid computing and throwing out terabytes of data per second.

It ain't so. It never was so. Will it be so in future? Not any time soon, I'm thinking.

The report-writers don't try to soften their error (and much love to them for it): "There is much talk of ‘big science’, and our initial research design presumed that we would be studying large-scale formal collaborations. But we found that most research groups in the life sciences continue to operate on a relatively small scale, and we revised our plans accordingly."

Again, we don't have hard evidence for numbers or weight of small science versus Big Science. If we plan for nothing but Big Science, however, we're making an enormous error in judgment.

There's a good bit of attitude mining in the report; I have little to add to it, so I will merely recommend that you read it. The lack of carrots for data-sharing is a deal-breaker, just as it was for self-archiving, and I agree with RIN that using sticks only will cause fairly serious backlash.

Skipping to the end of the report, then, we find out what researchers want by way of data-curation support. Namely, everything and the kitchen sink. At zero cost to them or their grant agencies, of course. I don't know why any other response would have been expected; it costs researchers nothing to say in a focus group that they want a pony, so why would they not say they want a pony?

At some point, someone will have to tell them they can't have a pony. I don't envy that person or agency one bit. Even so.

I believe that individuals and institutions planning data-curation services should take researchers' wants as expressed in this report with a generous dash of salt. No institution can give them what they want, because what they really want is for the problem to be taken care of for them without their involvement. What should be aimed for is giving them what they need.

They're projects that, no matter their scholarly or design merit, are completely unpreservable because they were built from unsustainable tools, techniques, and materials. What's worse, even a cursory examination with an eye to sustainability would have at least signaled a problem.

It's not the unpreservability so much. It's the obliviousness that makes me hurt inside.

For various reasons, the digital humanities are particularly prone to this sort of thing. Scientists do use unsustainable tools, but often they haven't a choice (thank you for the lock-in, instrument manufacturers) and most times they're at least aware of the problem.

Humanists, on the other hand, will pick up whatever tool seems good to them without even asking themselves whether the result will last past the lifespan of the tool. Then they bring the resulting binary CD-ROM or Flash-based website or whatever to the library with beaming smiles, and are shocked to find out that the library can't help them.

Proprietary tools and formats are often quite shiny. I remember HyperCard well, and so may you. In its day, there wasn't anything shinier. The problem is, following the shiny to the exclusion of all other considerations dooms a shiny project to be less shiny a year later, hardly shiny at all five years later, and completely inaccessible and unusable five years after that.

(I do not kid. Historians and sociologists of early digital culture are deeply distressed at how much "HyperCard art" nearly fell out of reach forever, though there are now emulators capable of dealing with much of it.)

There are better ways to proceed. They may well be less shiny at first, but the secret is that shiny can almost always be added to solid sustainable data later on, through mashups or interface redesign or whatever takes your fancy. Once its platform is thoroughly obsolete, though, a project may well not be rescuable in any form. Worse yet, piling otherwise-sustainable raw materials into an unsustainable platform destroys the sustainability of those raw materials, too. I've seen it happen!

So please, step away from the shiny and think.

(Thanks to @pseudonymTrevor and other Twitter friends for inspiring this post, and possibly other ones—I am still pondering the intersection of "never done"-ness and sustainability.)

Be that as it may, standards development and compliance is one way to make everybody's data play nicely with everybody else's data. It's not the only way, to be sure; one very important way that I'm sure we'll also see more of is Being The Only Game In Town. ICPSR manages this quite successfully, and so does the Digital Sky Survey. If you want to be important in the data spaces dominated by either of these large players, you play by their rules, just that simple.

When there's no big player to lay down the law, though, standards development becomes more attractive. How do you make a standard, then? More to the point, how do you make a good standard, a standard that works, a usable standard, a standard that will last?

I liked this blog post by Adam Bosworth about standards development very much. I think it captures much of the excellence that goes into successful standards as well as the dysfunction attending failed ones. I do want to add a fillip of my own, though, based on my own experience helping to build standards and trying to use standards built by other people.

When you're in a roomful of people tasked with building a standard, make sure the room contains representation from every group of people who will be asked or required to use it. That emphatically includes the non-technical and the non-specialist. It goes double or triple if the standard will affect existing technology installations: you must have someone in that standards room who uses the existing technology! No, a developer of the existing technology does not fulfill this requirement, because the distance between developers' understanding and users' understanding is often vast.

If the non-technical, non-specialist representative in the room can't understand the standard, it will fail. If that representative can't produce data that fit the standard, likewise. I agree with Bosworth's reservations about RDF; I myself have trouble understanding it and putting it to use, despite a decade's experience with markup, and I believe the tribulations such folk as I face when trying to deal with it have retarded its adoption significantly.

What happens when this rule about representation is flouted, but standards are published anyway, is standards that fall apart under real-world use. I will adduce OAI-PMH as an example. It follows quite a few of Bosworth's recommendations: it's simple (I have explained it in twenty minutes to library-school students), largely human-readable, focused, precise about encodings, in possession of real implementations, and free on the web.

It is also flawed. Huge projects built on it have found its flaws impossible to bypass and expensive to work around (see Lagoze et al. 2006 for how NSDL ran aground on OAI-PMH's inadequacies).

The major flaw, to my mind, isn't difficult to explain or to understand: OAI-PMH has no error-reporting built in. In a protocol standard built for communication of and about metadata, nobody in the standards-design process ever seems to have asked the (to me) simple and obvious question, "What happens if the metadata is malformed or otherwise wrong?"

Anyone who's worked on the ground with repositories of any stripe knows that metadata problems, sometimes gross problems, are par for the course. For that matter, any librarian can explain the pitfalls of metadata and citation creation at great length. I honestly can't tell you why OAI doesn't seem to have on-the-ground repository managers and other librarians capable of raising such practical issues working on its standards bodies.

I can, however, tell you that they should. The latest OAI development, OAI-ORE, contains exactly the same no-error-reporting weakness I just pointed out in OAI-PMH. Yes, some of the underlying technologies that OAI-ORE is built on contain certain kinds of error reporting, but the aggregation of those errors that can be reported is only a subset of the errors that I believe will crop up.

To make standards that work, include people on the standard-design team who work with the processes underlying the standard. Now that you know this—go forth and standardize!

Chris Rusbridge (of, among other things, this thought-provoking meditation on digital preservation) has been spotted here, and whenever he pops up he makes me think about things. This time, I was thinking about disciplinary expertise, and how I need to make a better case that less of it is necessary for data curation than generally admitted.

I hope we can at least admit that data curators don't have to be researchers themselves. Do researchers have to be involved in the curation of their own data? Absolutely! Data curation starts at the beginning of the study-design process, and continues all the way through and past publication. But that doesn't mean that researchers have to do everything. The exact division of labor is still being sorted out; that's partly what this blog is about. That the labor must and will be divided appears to be beyond dispute.

The corollary to this is that a data curator will almost always know less about the data, viewed from certain axes, than the researcher does. She may well know more about it viewed from some other axes—file format details, metadata crosswalking, whatever. Some things, though, she won't know and presumably won't have to.

So what does she have to know about the research and the discipline in order to be a responsible data steward? And does she have to walk into the process with that knowledge pre-existing, or can she learn it as she works on the research project? How much of what she needs to know will transfer from other projects she's worked on?

Cards on the table: in the absence of much evidence either way, I think that someone with the intelligence, disciplinary background, and intellectual curiosity of a good subject-specialist librarian can learn enough "on the job" to hit the 80/20 point pretty easily—and 80/20 is more than good enough for a successful campus data-curation program in my book. The other 20% of edge cases can hire specially.

I'll use a True Story about myself as an anecdote. Feel free to quarrel with me (civilly, please) in the comments.

Some years ago I did a small contract job for the ACLS E-book project. They were working on rekeying and marking up an art-history book with extended segments of polytonic Greek text. Their keying vendor took one look and said "no way do we key polytonic Greek." So ACLS told them to key the rest of it and leave placeholders for the Greek. They came to me asking whether I could key the Greek in proper Unicode without snarling up the markup.

I have never studied Greek. I do not speak Greek. I do not write Greek. I do not read Greek, except in the sense that I recognize the letters and can laboriously sound them out. Don't ask me what in the world the accents and squiggly bits in polytonic Greek mean; I haven't the slightest clue.

Not snarling up markup? That I can manage. After an hour or so of research, I found fonts and tools that could enable me to do the keying job correctly and with reasonable efficiency. ACLS and I agreed on a price, and off I went. I didn't know what the squiggles meant, but I could reproduce them, and that was plenty good enough.

When it came time to proof my work, I didn't rely just on my own eyes; that would have been stupid. I called in my classics-major husband. He found typos and the odd homeoteleuton, which I duly fixed up. I sent the result back to ACLS, and they were happy enough to pay me, so there that is.

And there we have it: a partnership between a tech geek and a reasonably well-trained domain specialist (kindly note that my husband was an undergraduate classics major) took care of a data job. I think this can happen more often in more fields.

The chief barrier is the belief that it can't.

So wince and flinch at the design (black Arial on white? really? in 2009?), but read the slideshow anyway.

I do want to pick apart the slide from which I took the title of this post. I reproduce the said slide's text in full:

Can we just give the problem to the libraries?

As a professor in a library school, I wish I could say that libraries were the obvious organization to take care of data. They understand keeping things for a long time and arranging to find them later. It would be a sensible new activity to balance a decrease in foot traffic into book collections. But...

• They have not been ambitious in this area; libraries feel under budget pressure and don't want new tasks.
• They lack the subject area knowledge to deal with complex data sets in scientific areas.
• They often lack the technical skills for advanced data handling.

I have no quarrel whatever with Lesk's first point. Libraries have absolutely been timid about this, and they still are—not without reason, either! This, to me, is the buck-stopper, the Berlin Wall, the concrete bollards. If library administrators shy away from this, or give it lip-service only, Lesk is right and there's nothing to be done. It won't matter how many librarians are ready and willing to do this work, if they're not allowed to or not given sufficient resources and authority to.

How likely is this outcome? In my estimation, more likely than not. My estimation is admittedly colored by this being very early days yet, but as I've remarked before, the longer any interested group dithers, the more likely it is that the action will be elsewhere. The more the action moves away from libraries, the more likely library administrators are to breathe a quiet sigh of relief and turn away from the problem altogether.

So what is a librarian who wants this work to do? Well, one answer is to keep an eye on discipline-specific projects, those that are larger than any single institution, the up-and-coming ICPSRs and Sloan Digital Sky Surveys. For those interested in data curation inside an institution, I think the answer may well be to learn enough to insinuate oneself onto research teams directly through their in-house IT arms. I may revisit this answer later; in-house IT is starting to become just cost-ineffective enough that some recentralization may happen. In that case, the would-be data curator has more options. Either way, though—the wise data curator does not attach himself limpetlike to the library. The action may well be elsewhere.

What is a researcher or funding agency or think-tank that wants libraries to take on this work to do? Researchers need to ask. Nothing gets library priority so fast as a well-articulated request from faculty; that goes double in disciplines where physical library spaces are waning in importance. Agencies and think-tanks: I'd recommend being an awful lot clearer about what the services provided look like and how they need to be staffed. Laundry-lists of skills are useless without an estimate of FTE and budget; such an estimate is noticeably lacking in every single discussion of this problem I've ever read.

I half-agree, half-disagree with Lesk's second point. There's a lot of disciplinary knowledge in academic librarianship. We don't select books blindly! We do it by taking heed of what our local researchers are doing. Many selectors and liaisons assigned to particular disciplines have degrees, sometimes advanced degrees, in that discipline. In the social sciences, by the way, data librarians with appropriate disciplinary knowledge already exist.

The problem isn't the non-existence of disciplinary knowledge; it's the uneven spread of it. For any given discipline at a research university, I'd guess it's a better-than-even bet that the library has a librarian somewhere with appropriate disciplinary expertise—but it's not a certainty.

Of course, there's also a question of how much disciplinary expertise is actually necessary for this work. Diane Hillmann remarked to me at ALA this summer that "[researchers] all think they're special snowflakes," but in her experience the basic sustainability questions don't differ all that much from dataset to dataset. That's what I think, too, with the added wrinkle that disciplinary specialists may actually be too close to their data to have a good read on how others will want to use and query it. An outsider perspective may well be useful!

(The real problem is one of first impressions and secret handshakes, as my SciBling Christina adroitly points out in the context of reference interviews.)

I could very nearly recycle the answers I just gave for Dr. Lesk's second question for his third. In aggregate, research libraries have quite a lot of technology expertise. How much any given library has isn't predictable, and may well not be sufficient.

If we cross the answer to the second question with the answer to the third, we approach the real conundrum: sufficient disciplinary expertise and sufficient technical expertise tend not to coexist within the same librarian. Take me, for example: if it's textual or linguistic data, I'm your librarian—that's my educational background! I can apply common sense and well-honed data-management expertise to numeric or instrument data, but I can't apply disciplinary knowledge because I don't have it. Selectors and liaisons, conversely, likely understand quite a lot about local research in the disciplines they serve, but they mostly don't sling Python and XSLT, nor do they tend to have the digital-preservation knowhow that I do.

John Saylor of Cornell gave what I believe to be the appropriate answer to this problem in his talk at ALA Annual: a technical team dedicated to data needs to work with librarians who have disciplinary expertise in order to solve problems. The disciplinary coverage achievable with this staffing model won't reach 100%, but it'll get as close as seems feasible. Nota bene: without broad participation by disciplinary specialists across the library, a data-curation service suffers and may well fail!

Lesk's objections are serious, pertinent, and pointed. They are not, I believe, unanswerable, but answering them will take considerable vision and will on the part of research-library administrators. Time will tell.

• The classical librarian
• The humble index
• Classification

Today I'll finish my discussion of classification, and distinguish it from subject analysis, since that distinction often seems to confuse, especially in our digital age.

So if we'll recall, the goal we set for ourselves was to collocate physical books on shelves in such fashion that their arrangement would be useful to information-seekers. With most non-fiction, that means collocation by subject, by what the books are about.

(There are lengthy philosophical discussions of "aboutness" in the information science literature. I recommend avoiding them with all your strength. They make my eyes bleed.)

To make this work, we have to map knowledge-space onto physical space: divide up human knowledge into convenient slots to assign books to. This is, you might say, a tall order: an ontology of infinite domain, but where each item can only fit in one place.

In the States, most libraries use one of two such maps: the Dewey Decimal System or the Library of Congress Classification. About the kindest thing one can say for Dewey Decimal is that it was a product of its peculiar time; for today's purposes, it is heavily overnumbered in religion, for example, and undernumbered in science. Perhaps worse, its sense of the world is not exactly immediately intuitive to the modern eye: why the long separation of geography from the so-called "social sciences," of which psychology is apparently not one?

This is one danger of any would-be universal classification. Our sense of the world and its knowledge changes over time, sometimes quite a lot and quite suddenly. If our ontology doesn't keep up, it serves its purposes less and less well. How easy is it, really, to find the right shelf in a library of any size organized by Dewey Decimal? Considerations such as these no doubt informed the shift of one library (and later others) to the BISAC codes typically found in large bookstores.

Another danger of the universal classification is that its specificity is of necessity somewhat limited. Many medical libraries, for example, ditch Library of Congress Classification because it just doesn't drill down far enough into medical minutiae for their needs. The NLM Classification fills the gap.

With physical books, we cannot escape the constraint that each book must go in one and only one place on the shelf. Once we're away from the physical item, that constraint disappears. The card catalogue was the first desperately clever escape from the tyranny of the physical item: in a card catalogue, the same book could be "shelved" by author, title, and one or more (usually three to five, to avoid overproliferation of cards) subjects assigned to it by the cataloguer.

This meant the addition of a subject-heading system to the classification vocabulary. You can't just add more classification numbers to the physical item; you then imply that it goes in more than one place! This is the difference between Library of Congress Classification and Library of Congress Subject Headings. Under most circumstances, the LCC number assigned to a book will correspond closely in meaning to the first LCSH assigned in the book's catalogue record. They are still distinct systems, however! Don't confuse them. Librarians chuckle behind their hands.

Of course, digital items don't have to live in just one space. Classification is therefore slowly giving way to subject analysis and similar ways of relating items to each other as digital libraries develop.

And that, in a remarkably simplified nutshell, is how books are arranged on shelves in libraries. It doesn't happen by magic!