Thomson Reuters, Nobel Prize predictions and correlation vs. causation

It's time for my annual post taking issue with Thomson Reuters (TR) Nobel Prize predictions.

(2002, 2006, 2007a, 2007b, 2008, 2009, 2010)

Because, yes, they're at it again.

Can the winners of the Nobel Prize be correctly predicted? Since 1989, Thomson Reuters has developed a list of likely winners in medicine, chemistry, physics, and economics. Those chosen are named Thomson Reuters Citation Laureates -- researchers likely to be in contention for Nobel honors based on the citation impact of their published research.

Reading this you would reasonably assume that TR thinks there is at least a little bit of a causal link between citation counts, or as they call it "citation impact," and winning the Nobel Prize. Sure, they say "likely to be in contention" as a way of softening the link they'd like to draw. But they make such a big deal of the whole thing that it's hard not to imagine that they see drawing a strong link between the two as a great way to promote their citation reporting and analysis products.

However, in their Process Essay, they make make explicit that they understand that the link between citation counts and true scientific impact is only correlation:

Numerous studies in the past three decades have shown a strong correlation between citations in the literature and peer esteem, often reflected in professional awards, such as the Nobel Prize. This should cause no surprise. Citations have been likened to repayments of intellectual debts, so persons who have accumulated such credits from their peers are often those whom these peers nominate for prizes and other honors.

*snip*

It is clear that the choices of the Nobel Committees are more complex than simply identifying highly cited or most-cited scientists.

Last year there was an article Globe and Mail covering the TR predictions:

"We choose our citation laureates by assessing citation counts and the number of high-impact papers while identifying discoveries or themes that may be considered worthy of recognition by the Nobel committee," said David Pendlebury of Thomson Reuters.

"A strong correlation exists between citations in literature and peer esteem. Professional awards, like the Nobel Prize, are a reflection of this peer esteem."

And Pendlebury again from a comment in my 2009 post:

Many in our lists rank much higher than the top .1%

The reason others suggested the same names we have, in blogs and news stories, is that they have studied our selections in this and past years.

By the way, Blackburn, Greider and Szostak won this morning, and were picked by us this year. Through citation analysis we focused on Blackburn as long ago as 1993: http://archive.sciencewatch.com/interviews/elizabeth_blackburn1.htm

Note that this was before the receipt of the Gairdner Award (1998) and Lasker (2006).

Again, not causality, just a strong correlation between citations and peer esteem.

We don't disagree that Nobel Prizes are not chosen on the basis of citation counts.

From Toronto's Dr. James Till, one of the citation laureates that TR chose last year(from the G&M article):

Dr. Till, reached Tuesday at his Toronto home, said he was told by Thomson Reuters that he and Dr. McCulloch are among the top picks for a Nobel. But Dr. Till, known for his scientific rigour, was reluctant to say much about the prediction.

"I'm skeptical," he said. "This is just speculation based on data that Thomson Reuters gathers, citation data."

"This kind of speculation is not something I'd like to comment on."

TR has certainly changed their tune over the last number of years in the way they frame their predictions.

That being said, however, I'm still not a fan of the exercise. Citation counts aren't what's important in science and aren't the best way to measure impact. The Alt-Metrics project and many other initiatives have sprung up over the last few years looking for better ways to measure scientific impact than merely using citations.

Basically my position is that citation is a narrow way to gauge true impact and any project that relies primarily on citation data to predict prizes such as the Nobel is fundamentally flawed. I'm sincerely hoping that TR will soon reconsider their annual project.

So, let's see who they've predicted for this year. Note: I'm indicating with letters the teams of scientists that TR have grouped together in their predictions.

Chemistry

• Allen J. Bard
• Jean M. J. Fréchet (a)
• Martin Karplus
• Donald A. Tomalia (a)
• Fritz Vögtle (a)

Physics

• Alain Aspect (a)
• John F. Clauser (a)
• Sajeev John (b)
• Hideo Ohno
• Eli Yablonovitch (b)
• Anton Zeilinger (a)

Physiology or Medicine

• Robert L. Coffman (a)
• Brian J. Druker (b)
• Robert S. Langer (c)
• Nicholas B. Lydon (b)
• Jacques F. A. P. Miller
• Timothy R. Mosmann (a)
• Charles L. Sawyers (b)
• Joseph P. Vacanti (c)

Economics

• Douglas W. Diamond
• Jerry A. Hausman (a)
• Anne O. Krueger (b)
• Gordon Tullock (b)
• Halbert L. White, Jr. (a)

For the four prizes, that's 23 different people chosen in 13 groupings of one or more.

Let's see how they do this year. I predict about the same as previous years, in other words, some right and most wrong. Some of the people picked this year will be chosen for a Nobel this year. Some will get picked in a later year. Probably one or two people who get the Nobel this year will have been picked by TR in a previous year.

After all, over the years TR have picked so many different people that every year their odds improve that the Nobel committees will select someone TR has chosen in the past.

Once again, I would like to emphasize that I have nothing against the scholars whom TR has "nominated" and wish them well. I certainly don't mean to cast a negative light on their contributions to their fields at all. My beef is not with them, but with TR's misuse of their citation data.