In a comment to my previous post on Benny Peiser's claim that Naomi Oreskes article on the scientific consensus was wrong, Meyrick made a good case that Peiser had conducted a different search than Oreskes:
Think I've finally worked out how to replicate Oreskes' search. There are 2 fundemental differences between Peiser search and Oreskes.
1. Oreskes excluded the "Social Sciences Citation Index" and the "Arts & Humanities Citation Index", Peiser does not.
2. Oreskes set the search limits to include only "Article"s, whereas Peiser set the search limits to include "All document types".
Using Oreskes search you get 929 documents (her article says 928, close enough?), where as with Peiser's search you get 1247 documents.
I can confirm these findings. I suspect the 928 vs 929 difference is because an article has been added to the database since Oreskes ran her search.
Peiser is well aware of this because he was involved in the ensuing discussion. Now Brian Schmidt has discovered that after this discussion Peiser claimed: "I have analysed the same set of abstracts." He lays out the case in detail here and here.
Also, Lars has discovered that Peiser peddling his bogus study in Canada's Financial Post:
Dr. Peiser has a hand-wringer in today's Financial Post (business section of Canada's National Post, sorry but no link, it is in today's hard copy edition [scoffed from a recycling bin, I won't pay for this rag] but not posted on today's website edition, subscription anyway and really not worth it even for Canadians) in which he bemoans "the stifling of dissent and the curtailing of scientific skepticism..." which is "bringing climate research into disrepute [sic]". It appears to be a rehashing of the points that he makes above and in his Science letter, and Oreskes (and her pernicious influence) comes in for another drubbing. Dr. Peiser seems to have made his getaway from Deltoid with his hat shot full of criticisms but his head unscathed; as far as I can see, none of the points made by correspondents here have been addressed in this op-ed piece. It is three columns and over half the page---the Post always gives prominent display to contrascience work.
All right, who's up for some distributed abstract checking? I'm thinking of setting up a system with all 929 papers in it so that everyone can classify a few papers. That way we can check who is correct about the percentage that support the consensus.
Update: John finds the link to Peiser's National Post piece. Notice that Peiser has dropped his easily refuted "34 articles reject the consensus" claim and switched to
An unbiased analysis of the peer-reviewed literature on global warming will find hundreds of papers (many of them written by the world's leading experts in the field) that have raised serious reservations and outright rejection of the concept of a "scientific consensus on climate change." The truth is, there is no such thing.
Presumably a reference to Timo Hameranta's list, that includes papers that explicitly endorse the consensus. And his other big piece of evidence is Bray's useless survey. Canadian readers might want to send a letter to the National Post.
wth, it makes a break from reviewing proposals...:)
Actually I think the difference is that the 929 includes the AAPG piece.
I might add, the categories to be used for classifying abstracts need to be decided upon. Do we use Oreskes' 6 or Peiser's 8 categories?
An online system for abstract classification sounds like fun. Can you set it up so that it delivers you a random abstract and you vote it into 6 and 8 categories? (no reason not to do both). It needs to track some login id, o/wise you'll suffer as Bray did.
I say 6 categories. The Oreskes paper needs to be replicated, not the Peiser. But William's idea is a good 'un too. Count me in.D
Pledgebank is your man on this one.
I'm up for it as well, depending on the deadline for the work (got a lot going on in May).
May need to do some quality control on the resulting work, or restrict who's allowed to classify the abstracts.
Count me in as well. However would it be reasonable to run this by Oreskes to see if we have replicated her abstract list?
On thinking it over maybe not. If we use a different list and come to the same conclusions then we can call her results robust ;-) . So either way, carry on.
My attempt at replication was a "best guess".
I've emailed Dr Oreskes about 10 days ago to confirm whether my guess was correct or not, but no reply yet. I'm not an expert in climatology so the lack of response is not suprising, so I've ask for help from someone who is (he knows who he is).
I emailed Oreskes as well with the same question yesterday, and got an auto-reply saying she was gone until May 23.
OK, the article appears to be on line now and can be found here:
While I am a Canadian and I love this country, some of my countrymen can be so blind!
Given that we have pretty much established that the debate is not really about science, I'm not sure that what is proposed would actually "resolve" the issue. But it could still be an interesting exercise that could also produce something worthwhile if we structure it right. Since this is outside of the formal scientific peer review process, we might do it instead as a citizens jury? or blog jury? I'm thinking this one through as I type. But instead of just voting the abstracts up or down, could we annotate each of them with a sentence saying why they do or don't accept the consensus? It would be even more interesting if we could state for each, the kind of uncertainty addressed, so as to make clear the distinctions between uncertainty about AGW, and, say, uncertainty about climate "policy." Then, we could actually produce an article about this and submit it to the next Skeptics Circle - a circle of Real Skeptics! As I said in a follow-up post at www.postnormaltimes.net - they have just entered the Malebolge with their 8th edition, posted at Pharyngula). As a conclusion, we can decide which of the bolge these people belong in. The folks in the (real) Skeptics Circle are too busy at the moment with evolution issues at the present - the next edition will be out May 26th, for which links can be sent to saint_nate at hotmail.com. If that is too soon, maybe the next one after that? We could invite Peiser et al to bring up any issues they have in that forum, and let the Real Skeptics decide if their arguments have any merit whatsoever. I also volunteer the Post-Normal Times to host this as a special edition - many of those listed on the PNT advisory board are well known for research on issues pertaining to data quality and extended peer review, and participatory assessment wrt climate change. So they may also be able to provide some useful input. The notion of extended peer review is not to second guess scientific judgment on technical quality of information, but to recognize importance of input from stakeholders/users of scientific information, as to whether information is even relevant or not.
Thanks for the link, John. Don't know how I missed this before.
Peiser repeated his dishonest statement on May 12, four days after he learned it was untrue:
On May 12 at 4:26 p.m., Dr. Peiser wrote "I have analysed the same set of abstracts" as Dr. Oreske.
I wrote about it here. http://www.opendemocracy.net/forums/thread.jspa?forumID=179&threadID=44…
Had a look at the file formats for ISI Web of Science. My God, talk about antiquated, have these people never heard of XML?
Tim, you might find the following software useful for converting the ISI format to XML:
I don't think it was there before. I searched earlier in the day and couldn't find anything. In fact I had scanned the article and prepared to submit it to interested parties (after I had obtained the requisite copyright release of course ;-) )
Even found a XML Schema for the XML format:
Assuming you want to move the abstracts into a database, I would:
1. Save the abstracts from ISI Web of Science into a single text file
2. Use isi2xml.exe to convert from the text format to the XML format
3. Use XMLBeans (using the schema) to copy from the XML format into the database of your choice (MySQL?)
Thanks for the Peiser linky John. Benny forgot to mention his paper would have been rejected anyway if someone had read it, because it is a POS. This should be his argument: yes, my paper can't stand scrutiny, but no one will scrutinize it because I'm an amateur and the professionals won't let an amateur come to the table. Much more powerful than his whining in the National Post.D
Stating the obvious but have any of you read Oreske's paper?
Of course. Is there a point here somewhere?D
Oh dear, I smell an attempt to claim Dr Oreskes' work is sloppy. What "they" actually mean is the Dr Oreskes didn't state every single setting and filtering rule that she used.
Startling new greenhouse developments on the home front can be found over at The City
Got this reply from Oreskes:
Thank you for your recent correspondence. Unfortunately, due to the
overwhelming quantity of email I am now receiving on climate change
issues, I am unable to reply personally. I am currently working on a
long essay, which answers many questions that have been put to me in
recent months. I will be happy to send you a pdf when it is available.
In the hope that we can all work together to ensure the best possible
future for humans and other living things,
I'm not sure whether this should affect Tim's idea of replicating her original study, although it would be nice to have more details of what she plans to do, or of the filtering rules she used.
Oreskes is missing the point. Peiser is all over the media, probably has a booker on loan from AEI or the equivalent, with a legion of flaks setting him up for interviews, op eds and whatever. Oreskes huddles with her data bases. By the time she is through and has a definitive whatever ready for publication, it will be too late. The lady needs to get herself on the offensive here or she has lost.
Sadly, all too true Eli. We have ample evidence of this happening in the past, and there is no reason to doubt it is happening again. The flip side is if she were to have a network like the contrascientists, she would be accused of having a network. Science needs to figger out how to best explain its findings to decision-makers and the public - hopefully not using the corporate model.D
On the 8 May I emailed Dr Oreskes inquiring into her search methodology. In particular I included the following:
I have used the following criteria for the search, that is to say those criteria that differ from the default values of the search engine:
1. Search only in "Science Citation Index Expanded", excluding the "Social Sciences Citation Index", and the "Arts & Humanities Citation Index".
2. Search from years 1993 to 2003.
3. Topic field = "global climate change".
4. Document types = "Article" instead of "All document types".
5. The following article is dropped from the sample since it is not a peer-reviewed research publication, but an statement by the American Association of Petroleum Geologists:
Gerhard LC, Hanson BM, "Ad Hoc Committee on Global Climate Issues: Annual report", AAPG BULLETIN, 84 (4): 466-471 APR 2000
This produces 928 documents, the same as your article stated. Is this the same criteria as you used, or have I made a mistake?
With some help I've received a reply that included the following:
"Yes I only looked at scientific journals and only articles, not all document types"
I figure this is a confirmation of the first 4 parts of the search criteria.
Dano, being on the offensive means never having to explain how you got there.
Perhaps instead of settling this argument, a simpler response (given Oreskes lack of time to confirm the abstracts) would be to do a NEW study; say from Jan 1 2000 to End 2005? (I suggest either waiting to begin, or adding the last few later as they come in).
And rather than examine the abstracts, it would be useful to contact each of the Authors involved (at least the corresponding authors, but preferentially all) with a questionnaire.
Does your article Support the idea that the earth is warming?
Does it support the idea that this is due to increased warming gases (Co2, methane...)
Does it support the idea that this is Anthopogenic warming?
3 boxes to tick; supports, rejects, neither.
You could also include a box asking whether the authors opinions (as opposed to the article findings) are in favor/against the questions posed.
That should provide a real vindication of Orestes work.