In most of the discussions of using usage as a metric of scholarly impact, the example of the clinician is given.  The example goes that medical articles might be heavily used and indeed have a huge impact on practice (saving lives), but be uncited. There are other fields that have practitioners who pull from the literature, but do not contribute to it.

So it was with interest that I read this new article by the MacRoberts:

MacRoberts, M., & MacRoberts, B. (2009). Problems of citation analysis: A study of uncited and seldom-cited influences Journal of the American Society for Information Science and Technology, 61, 1-12 DOI: 10.1002/asi.21228

The article provides great examples from the field of biogeography (the distribution of plants and animals over an area – they tell me). It is typical for researchers in this field, when writing articles in peer-reviewed journals, to not cite their data sources.  Some of the data sources are flora  – “a list of plant species known to occur within a region of interest.” The flora might be books, government reports, notes in journals or some other sort of gray literature.

The authors give a couple examples – one is their article – and show how these articles are uncited according to Web of Science, but heavily used and well incorporated into databases, books, and pamphlets. As they say, the purpose of the article has been achieved.

Not only are these things used directly, but once their contents are incorporated into databases, the database then goes on to serve maybe thousands of people. The sources are often listed in notes or in an appendix but no citation. This content that is sucked up into books or databases provides no traceable usage link – as far as I can tell. If we can’t even determine the impact of the article – a container for an idea – how can we understand or evaluate the impact of the author and his or her knowledge contributions?

It’s been noted elsewhere (see the article for citations/discussion) that the largest influence on a scientist often comes from informal communication partners – colleagues and co-workers. This is not cited, either.  So the question becomes, if we are truly interested in evaluating a scientist on his or her influence, we have to come up with new methods that look at how their ideas have been used – it is not enough to look at article citations or downloads.

(as an aside: the authors quote a website that bemoans the difficulty of locating floras. Certainly if they were cited, that would help!)