Annals of Possible Cherry-Picking

A very interesting issue--discussed in comments here and here--has arisen over one aspect of Roger Pielke, Jr.'s testimony yesterday. In that testimony (PDF), Pielke suggested that Waxman's committee had cherry-picked science with the following statement in a memorandum (the original of which I have not been able to locate):

". . . recently published studies have suggested that the impacts [of global warming] include increases in the intensity of hurricanes and tropical storms, increases in wildfires, and loss of wildlife, such as polar bears and walruses."

The above statement on hurricanes, Pielke says, was backed up with a reference to three supporting studies, but nowhere did Waxman's committee mention that a still more recent consensus statement from the World Meteorological Organization (PDF) had observed that a substantial debate continues over the results of these studies. Thus, according to Pielke, Waxman's committee was cherry-picking.

Is this a valid criticism? Well, Waxman's committee certainly could have noted that this hurricane-climate issue is a subject of ongoing debate. If I was writing the passage quoted above, I would have done so.

On the other hand, there is nothing literally incorrect about the passage. "Recently published studies have suggested..." That's absolutely true and, moreover, a fairly weak statement over all. The committee does not claim to judge whether those "recently published studies" are valid, it merely says they've "suggested" a particular outcome, which they certainly have.

So, was Waxman's committee guilty of cherry-picking? Perhaps a little. But Pielke could have found a much stronger example to use to make his point, because at least in this case, Waxman's committee was fairly cautious in its language. Moreover, this example shouldn't be used to minimize the much more egregious behavior of the Bush administration with respect to science.

Categories

More like this

Thanks Chris for the link ... a few responses:

1. "this example shouldn't be used to minimize the much more egregious behavior of the Bush administration with respect to science." Nowhere did I suggest this, and I think your implication that I did is more than unfair. In fact I said in my testimony:

"The Bush Administration has engaged in hyper-controlling strategies for the management of information."

and I also discuss "heavy-handed Bush administration information management strategies which include editing government reports and overbearing management of agency press releases and media contacts with scientists. Inevitably, such ham-handed information management will backfire, because people will notice and demand accountability."

So, at least please properly represent what I actually said..

2. The discussion about Phil Cooney's edits at the hearing were not that they were technically incorrect in most instances (citing Drew Shindell's comments) but that they didn't tell the whole story. That is simply the case here with the memo, and indeed your defense of it.

My point is about the dynamics of selective use of information, not a comparison of intent or impacts of Bush Administration and the (irrelevant) Committee memo.

When there is a consensus statement that cites all relevant literature, presumably that is best to cite rather than a few papers, no? And shouldn't that approach be followed for the IPCC as well?

1. Okay, fair enough...I agree, I was there in the room, and you did describe the Bush admin behavior very strongly. So I don't mean to suggest you're creating a false equivalence here, although others might use the example to do so.

2. When there's a consensus statement, absolutely it should be cited. Without a doubt. But when it comes to misrepresenting hurricane-climate science, Kucinich's opening statement was vastly more egregious than the Waxman memo....I'd say use that one next time!

I agree with Roger here. Loss of the polar bear, is a pretty uncontroversial statement, and I doubt there is much in the way of literature that would contradict this. Placing tropical storms (about which there is certainly debate) in the same context as something that is fairly clear cut, provides the wrong impression.

Steve,
Yeah all the more reason that Waxman's committee should have cited the WMO statement. But I don't know anything about the polar bear literature and had at least vaguely heard that there might be controversy here as well.

Chris,

I just had a quick search on Web of Science and the results of this extremely informal survey (to be taken with the appropriate dosage of salt) suggest a degree of consensus. Rather worryingly:

"All ursids show behavioural plasticity but given the rapid pace of ecological change in the Arctic, the long generation time, and the highly specialised nature of polar bears, it is unlikely that polar bears will survive as a species if the sea ice disappears completely as has been predicted by some."

From the following (which seems to provide a good overview):

Derocher, A.E. et al. (2004) Polar bears in a warming climate. Intergrative and Comparative Biology, 44, 163-176.

Anyway, this is an aside. I think we are in agreement on the central point.

Correction. Counter to his statement above, Pielke did use this example to minimize the egregious behavior of the Bush Administration.

He writes "What has occurred in this memorandum is exactly the same sort of thing that we have seen with heavy-handed Bush administration information management strategies...."

http://sciencepolicy.colorado.edu/prometheus/archives/PielkeTestimony30…

If he'd like to disavow this statement, fair enough, but it is in his testimony.

Jim-

This came up in the testimony Q&A. I will admit that "sort of behavior" is not the most precise language. But yes I am talking about the dynamics. If you take a look at my oral testimony I did make this even more clear by adding "in microcosm":

"What has occurred in the preparation of this memorandum is in microcosm exactly the same sort of thing that we have seen with heavy-handed Bush administration information management strategies which include editing government reports and overbearing management of agency press releases and media contacts with scientists."

I do recognize that blog discussions often do not allow nuance (sort of like congressional hearings;-), but let me be absolutely clear on this point -- I am discussing the dynamics of how science is used in political argument - NOT equating the significance of an irrelevant memo with actions by the Bush Administration.

Clear enough?

Thanks!

Roger Pielke said: "I am discussing the dynamics of how science is used in political argument - NOT equating the significance of an irrelevant memo with actions by the Bush Administration."

If that is the case, why did you choose that particular example, of all the examples from all the years that you could have selected from?

Whether it was what you intended or not, most people (and particular those defending Bush) would read that as "Waxman's comittee is engaging in the very same behavior that they are criticizing the Bush administration for."

In other words : "Everyone does it".

Everyone may do it, but not everyone does it to the same degree -- not even close.

The degree is all important and merely to point out that the "political process is political" adds little (if anything) of value to the conversation. Indeed, it's a vaccuous statement if ever there was one.

So, once again: Why did you select the particular example of the memorandum to illustrate your point? Surekly, it did not happen by accident (you know, just pop randomly into your head).

By Dark Tent (not verified) on 31 Jan 2007 #permalink

1.) I think this matter is one of sleight of hand on the part of Pielke. Cherry picking as I understand usually involves selecting passages within one piece of writing that distort the original author's meaning or intent, and to advance an agenda different from the authors'.

To select a discussion of the findings of three articles all antecedant to a fourth, and to say the selection is biased by wanting a particular conclusion is to ignore the role of time. If that summary is written in October, what are we talking about--that Senate Committee reports should be subject to a level of peer review that even scientific journals cannot aspire to?

It is the weakest of arguments to suggest that a study is flawed, because a particular study was not considered in the literature review, through the thrust of the analysis stands.

2. Pielke's non-distinction between scientists and policy makers is nonsensical. It is not incumbent on policy makers to consider all scientific input equally, or to weigh just the scientific information when there's uncertainty (notice how this is glossed over by term 'politics'). Consensus can only indicate whether you are justified in taking a course of action, not its tempo and intensity.

Mike

I think there's another possible explanation (of probably several) for the reaction to some of Roger's oral remarks (I've not read his written testimony - but will soon - and I'd wager most of the Representatives that objected to his statements didn't read it either).

They may have exploited his choice of words, and of example, to re-emphasize the outrageousness (perceived and actual) of the Bush Administration's conduct. They considered his characterization of the committee memo as asserting a smaller example (microcosm implies this, even if Roger didn't intend it as such) of the alleged intimidation. Suggesting such cherry picking was the start of a slippery slope may have been more apt (though likely no less inflammatory). Have both the committee and the Administration cherry picked? Yes. Have they both engaged in initimidation worthy of reviewing whistleblower statutes? Probably not. For better or worse, the committee was interested in the latter - which has probably been overstated by the scientific community, given some of the data from the UCS survey. In other words, they thought Roger equated the conclusions jumped to in the statement equivalent to claims of scientists being silenced by zealous political appointees.

This was not the hearing, the setting, or even the people before which one should try and make a general argument about the selective use of scientific evidence. This was a relatively tightly focused oversight hearing. I'm not sure any climate science hearing would be the right place, because the application of the evidence is as much, if not more important to the policy discussion than the process by which it's developed. I doubt there's a relevant hearing for that kind of discussion scheduled any time soon.

By David Bruggeman (not verified) on 31 Jan 2007 #permalink

I would suggest that everyone listen to the hearing from 2:47 when Rep Wells starts to speak until 2:58 after Drew Shindell has said that Roger Pielke's understanding about where the consensus lies is not so good. You can advance the timer by hand to get to that place very quickly. At 2:56 Wells bluntly asks if Clooney's editing of scientific conclusions and the Committee staff statement are equivalent. Pilke's response at 2:57+ is that "there is NOT MUCH DIFFERENCE".

Clooney's behavior was previously defended by Roger Pielke Jr. Pot kettle and all that.

For those looking for some nefarious or evil intent in my comments, I am sure they will find it no matter what I say here;-) But once again:

On the dynamics of the selective presentation of scientific information in Committee memo and the Cooney edits there is indeed not much difference, as I said in the response to Mr. Welch -- obviously both selectively presented information to advance an agenda. Both misrepresented the science.

On the impact/importance of a minor memo in a minor hearing from a major report from the Administration, there is clearly a lot of difference. Those trying to place words in my mouth are stretching pretty far for something to complain about. If that is the biggest complaint about my testimony then it obviously must be pretty good . . .

My last comments on that, feel free to accept them or not;-)

Thanks!

Sounds to me like there are some sour grapes mixed in with the cherries.

By Dark Tent (not verified) on 01 Feb 2007 #permalink

It's well worth viewing the actual testimony.

Dr. Peilke was under oath and presumably told the whole truth and nothing but the truth. He said "there is not much difference" in how they edited.

He didn't get asked whether there was much difference in _what_ they edited, or _how_ it was used, or _what_ the effect was --- no one asked him the followup questions. He says he saw "clearly a lot of difference" in the effect of the two edits.

But nobody asked him that. The "whole truth" doesn't require providing answers to unasked questions (in politics). It does, of course, when you're doing science editing.

That may be the single clear difference between the two activities, eh? That's how I tend to tell scientists from politicians --- feed the scientist something that contradicts what's on the table and she's pleased and interested; feed the politician something that contradicts what's on the table, and he leaves the room in a hurry.

Since then, we've heard the political editors of the IPCC were getting serious pressure from the US side to minimize the hurricane-climate issue. They may just not want it talked about.

By Hank Roberts (not verified) on 04 Feb 2007 #permalink