John Mashey on how to learn about science

Another post on John Mashey's virtual blog. Everything that follows is from comments posted here by Mashey, lightly edited.

This long essay grew from a dialog in this thread into something that may be a more general resource than just some answers to Mr Manny.

There are 3 parts so far:
Part 1 Motivation & Approach to Science
Part 2 Relevant Personal Background
Part 3 Answers to Questions, Sources

Part 1 Motivation & Approach to Science

1.1 Why This?

I'm always curious when people with decent-or-better educational backgrounds strongly espouse conclusions directly opposite that of mainstream science. Is the mainstream wrong? Have they not yet done sufficient study? Or are there extra-science reasons?

Walter Manny teaches Calculus & English (since 1996) at Millbrook School, which seems a nice preparatory school in a lovely rural area ~90 miles North of New York City.

As I owe much to fine high school teachers, especially of math, science, and history, I was curious that a Yale-educated teacher at a credible school, would take such a strong position as Mr. Manny does, including evident contempt for serious climate scientists. Hence, I asked the usual sorts of questions.

Mr Manny answered my set of questions, mostly, and asked for my answers, which appear in Part 3. This part gives necessary general background, and my relevant background appears in Part 2.

1.2 Ideas, Hypotheses, Theories in Science

To paraphrase Stanford Professor Stephen Schneider, in any scientific discipline, ideas can be roughly categorized as:

(S3) Some things are well-established - strong proof is required to overturn [strong theory]

(S2) Some things (especially measured effects) have competing explanations. [hypotheses]

(S1) Some things are speculation. [ideas]

S3 includes:
(a) simple statements
(b) statements about probability distributions of some measured or projected quantity, often expressed with confidence intervals. Many people are far more comfortable with single numbers than distributions, and are not accustomed to error bars on measurements.

Sometimes, S2 hypotheses gain strength because new measurements shrink confidence intervals small enough that competing hypotheses can be ruled out.

Example: (a) cigarette smoking increases the risk of disease, now sufficiently understood in the US that few adults start smoking. (b) Numerous studies provide statistical measures of the likelihood of developing various diseases. Of course, controlled lab experiments on humans are not feasible.

Example: (a) CO2 is a greenhouse gas and the recent increase in CO2 is substantially due to human activities, (b) The temperature sensitivity to doubling CO2 (from 280ppm to 560ppm) is believed to be in range ~1.5C to ~4.5C, not exactly 3C.

While some technical disciplines allow/require exact yes/no answers, this is rare among complex observational disciplines for which simple lab experiments or mathematical proofs do not exist. Natural scientists in particular require substantial ambiguity tolerance.

People new to a specific topic are often confused by fierce arguments on S2 or S1 and think they are about S3, especially if just reading abstracts of research papers. Benny Peiser and KM Schulte both displayed especially severe cases of this error in attacking Naomi Oreskes' 2004 essay in Science.

It is important for a newcomer to read, not just the current mainstream view, but enough history to understand its development, and understand where a given idea fits and how long it has been there.

If one studies science histories, one can find obvious progressions, which might be roughly categorized as follows, and based only on peer-reviewed science, since nothing else counts for much:

Case 1 S1 -> nowhere
Case 2 S1 -> S2, but refuted fairly quickly, via errors, new data
Case 3 S1 -> S2..., but competing explanations persist, long battles
Case 4 S1 -> S2 -> S3 wins over competing explanations

Case 1: someone publishes something that is not very interesting, seems to have no way to be falsified, or turns out to have serious errors.

Peer-review only says "We didn't find obvious errors in a quick review and this might be worth reading".
Passing peer-review does not prove correctness or importance. Not being able to get papers through peer-review should be a major red flag for the reader.
Scientists with new major results do not just write OpEds or web pages, or publish them in Energy&Environment or Journal for Scientific Exploration. They send their results to Science, Nature, or other credible journals.

Case 2: an interesting new idea appears and gets attention, but then fairly quickly (within a few years) gets refuted, or claims of strong effects get weakened.

Example: Richard Lindzen's "Iris" hypothesis attracted interest, but did not gain widespread scientific support, as substantial conflicting evidence existed. There may still be some interest, but this paper did not suddenly invalidate AGW as some wanted to think.

Case 3: multiple hypotheses arise and persist for some time, gathering support, being modified, sometimes combining, or failing to accumulate evidence. An issue can stay open decades, and then quickly be resolved if the right new data or explanation appears.

Example:Geologists argued fiercely for many decades over Alfred Wegener's hypothesis of continental drift, but when enough new kinds of data appeared following World War II, most geologists quickly accepted it.

Case 4: some hypothesis has gained additional supporting data from multiple research efforts, and is accepted as a strong theory. This may well take decades, and there is often a continuous transition from hypothesis to well-supported theory, not a sudden jump, although the latter occasionally happens. Any theory is just an approximation to reality, and a good new theory must explain everything a previous good theory did, plus be a better approximation.

Example: it took many years to accumulate data that showed the health effects of tobacco. Some chemicals in cigarette smoke are known to be carcinogens without knowing the exact biochemical processes that cause them to be so.

Example: Newton's laws of motion work pretty well on Earth, well enough to launch satellites. Einstein's work better, and are needed for GPS satellites. Relativity is often revered, not just because it explained existing awkward data, but because it made many correct predictions of effects that were not yet observable.

Example: the idea that H. Pylori bacteria caused some peptic ulcers went from an odd idea to being well-accepted in few years, and of course, Warren and Marshall got a Nobel for their work. Important wacko ideas that turn out to be true are big wins.

The publication cycle of the most credible peer-reviewed journals is long enough that a non-expert should be prepared to be wary of any paper only 1-2 years old, especially if it has novel implications counter to mainstream established science. It normally takes at least several years to reach Case 3, and many more for Case 4.

Some people fasten on any new paper without understanding this, and in some cases, persist for years in referencing papers that have long since been refuted.

Sometimes, a good, or even great, scientist will become fixated on some idea, and will fight on in its behalf ... forever. Most scientists change their minds when the balance of evidence becomes clear, but some do not.

Example: Sir Fred Hoyle was a great astrophysicist, but fought for the "steady-state universe" long after overpowering evidence had accumulated for the Big Bang. Halton Arp has done fine observational astronomy, but also has not accepted the Big Bang.

Example: Sir Ronald Fisher was a great statistician, but never accepted the statistical evidence for the smoking-cancer connection.

Current example of real science one can watch happening: my favorite example of scientific process in visible action can be found in William Ruddiman's "Plows, Plagues, and Petroleum" plus surrounding papers, arguments, counter-arguments, modifications, to-and-fro-ing. Bill offers several somewhat surprising hypotheses (early CO2, early CH4, and more recent plague effects on CO2), with enough evidence from a highly-regarded researcher to make it to S2, but not yet (and maybe never) part of S3.

Unlike many arguments, this set is actually understandable to non-experts (like me), and one can actually watch science in progress. These hypotheses may linger with insufficient evidence to confirm or deny (Case 3), may get refuted later, or may turn out to be brilliant multidisciplinary theories accepted as the best explanations for otherwise puzzling data.

1.3 Metaphor: The Great Wall of Science

Think of progress in a scientific discipline as building a large structure, like the Great Wall of China (well, Olympics has been on :-), with multiple segments (sub-disciplines) each building upwards, but also trying to connect to form one consistent, connected whole. The Wall is so big that no one can see the whole thing. Quite often really exciting work happens in the gaps between well-established segments. Following are examples of the various Cases.

Case 1: Someone places a brick (S1), but no one else cares, so it never connects with the Wall.

Case 2: Someone else quickly appears and kicks the brick away (Case 2).

Case 3: The brick looks OK, and others start working with it, perhaps moving it around, or placing more bricks and mortar atop it (S2). Perhaps other groups do the same thing with competing alternative segments of the Wall.

Case 4: Sooner or later, some segment acquires enough bricks, and mortar, and even steel rebars, at which point it is enough stronger than the alternates, that the latter are abandoned.

A new brick anywhere is not yet mortared in, and probably takes a few years, even if it's atop the existing wall, i.e., a refinement of the mainstream. If a new brick is a bit wobbly, that doesn't mean the Wall collapses. Nobody cares very much about a brick until it has been tested, and mortared with others. In particular, a tall stack of bricks erected by one worker alone, with no connections, may carry very little interest, and falls over easily. Important papers in science get cited positively by other people, not just by the authors and colleagues, and not just to refute.

Measurement errors happen. ARGO buoys or weather balloons are found to have calibration problems. After years of use, computer programs for satellite temperature calculations are found to have simple sign errors. That's life.

A new brick placed far away from the Wall has to be very compelling to pull efforts in that direction (H. pylori and peptic ulcers). Scientists are strongly motivated to establish such new directions, not just add another brick to the Wall, as it's a good way to get a Nobel, as happened in that case.

Sometimes a well-established part of a Wall runs into a height limit, and needs a whole new level, i.e., Newton -> Einstein. The lower level is fine as far as I goes, but the second level is a better approximation. People have many hypotheses for the next level, but there is as yet no agreement.

For some people, usually not those directly involved in Wall-building, the appearance of a single brick anywhere else is enough to declare its collapse. Some may cite collections of old discarded bricks, cherry-pick specific bricks, or ignore inconvenient recent bricks, mortar and even steel, and claim the whole Wall is down. People routinely claim to have disproved long-established major laws of physics, which might be considered steel-reinforced concrete (like laws of Thermodynamics), and others then publicize such claims as proof of collapse.

It is very rare for long-established, rebarred Wall segments to be torn down or even reworked in major ways. When it does happen, it is almost always done by people experienced in the field, not by amateurs. Long-established AGW Walls are not demolished in a few months of part-time effort by 15-year-old students without much knowledge of physics and statistics. It is sad but true that most scientific breakthroughs are not generated by unknown lone scientists working alone in their basements.

It is easy for a non-expert, starting with the wrong book, website, or blog, to become convinced that AGW is all wrong, especially with a snapshot at one point in time, and especially if they get pulled into a self-reinforcing group that knows this.
(Ruddiman calls this an amazing "alternate universe" in which "most of the basic findings of mainstream science are rejected or ignored.")

It takes time for a non-expert to assess authors. If someone claims the Wall is wrong, but relies mostly on workers who contributed little, or cites long-discarded bricks, or changes their reasons every few years, then one can assume they are doing anti-science. It is well worth going back 5-20 years and seeing how people did or did not change their views. It is worth checking the publication records of those referenced. One must be especially careful when an expert builder on one segment retires, and then suddenly starts opining on a completely different segment in directions totally opposite the current workers there.

Real scientists would describe the Wall by how well-established each element was. If a non-expert backtracks what good scientists say, they'll find that wrong ideas get discarded, good ones progress and gain support, and understanding improves.

The IPCC is especially explicit about its confidence levels.

1.4 Extraordinary and Non-extraordinary Claims

Extraordinary Claims

Carl Sagan was known for saying "Extraordinary claims require extraordinary evidence", although this was more likely occasioned by his thoughts about parapsychology and other "interesting" ideas. I've read plenty of these for fun, and because every once in a while, something crazy turns out to be a better approximation ... but hardly ever.

Non-extraordinary Position On AGW

I subscribe to the mainstream science position that global warming is real, is substantially caused by humans, and will very likely cause serious problems under Business-As-Usual (BAU) assumptions.

Clouds, ocean heat exchange, and aerosols, especially contribute uncertainty, but I believe the IPCC's uncertainty bounds are reasonable, and I know smart people are working hard to tighten the bounds. Like many, I do worry about inherently-conservative IPCC forecasts in the presence of potentially non-linear effects/tipping points and I think there is easily enough evidence to require action. (I say "inherently" given the nature of the IPCC process, as discussed with a useful number of IPCC authors.)

Any non-expert (like me) could arrive at that same non-extraordinary position in two different ways:

(a) Default Acceptance of Mainstream

One either doesn't know enough physics/math/statistics, or doesn't want to spend the time to study AGW deeply, so one assumes the professional mainstream opinion is the best approximation of reality available.

Modern science is so huge that nobody can know everything, even in a specific discipline, so (a) is what most people have to do most of the time on most topics. One would find a few credible sources, understand the position, perhaps talk to a few experts, and that would be sufficient. [In Part 3, see Peter Darbee for an example of a smart non-technical person's approach.]

(b) Deeper Inspection by Interested (Classical) Skeptic

One conditionally assumes the mainstream, but really wants to study the topic to be sure, to be able to discuss the topic intelligently, and to give the objections every reasonable chance.

One generates a list of concerns about the mainstream position, studies each one in depth, and see whether the list grows or shrinks. Scientists think about preponderance of evidence, and are usually alert to contradictory data that might be right, since when found, such often lead to advances. Quite often, contradictory data turns out to be erroneous, or when fixed, is well within the plausible range.

This requires studying a selection of those disagreeing with the mainstream, and carefully assessing what they say, and giving them every possible chance to prove their cases.

However, it also requires some familiarity with well-established non-science methods of attacking science, and ability to assess credibility of sources' biases in any direction, possibly from ideology or economics,

One might go even further into studying anti-science memes, how they spread, who spreads them, how they work, their psychology and demographics, etc. (For me, this is a continuation of a long interest in science vs non-science issues.)

(c) Anti-Mainstream Position

Suppose an educated person takes the position:

"Even though I'm no expert, I'm almost certain mainstream scientists are wrong and these others are right."

I'd call that an extraordinary claim, in which case I ask my usual questions to try to understand why someone takes this position. It might be that they know more than the professionals, and thus vastly more than I do, in which case I could learn something new ... but, HARDLY EVER.

Since AGW is non-extraordinary mainstream science, I'm not sure it needs a lot of justification...

...but my answers may be useful as an example of the way a person in (b) approaches an area outside their immediate profession, and may record some useful information for others.

Part 2 Relevant Personal Background

Many can skip this, but it may be helpful to understand the starting point and approach.

2.1 Overview

Since "John Mashey" is a very rare, and likely, a unique name, I am trivial to locate: Google: john Mashey hits a Wikipedia entry that also points to a short bio as Computer History Museum Trustee. I'm a "half-retired" computer scientist/ex-corporate executive who consults for venture capitalists and technology companies, advises start ups, does private equity investing, attends lectures and conferences, does community work, travels for fun, skis, bicycles, i.e., what people do here when they want to taper off after years of intense Silicon Valley hard-drive and world business travel.

At DeSmogBlog, as a reply to Viscount Monckton's comment I gave some more background in my Reply, in part as an explanation of the 40-pager I'd written on the Monckton+Schulte vs Oreskes silliness in 2007.

Briefly, I got a BS Mathematics (+ 1 course short of BS Physics as well), then MS & PhD Computer Science, at Penn State, followed by 10 years (1973-1983) at Bell Laboratories as Member of Technical Staff, then Supervisor. Since then, I spent 17 years at Silicon Valley computer companies, mostly as a Director / VP / Chief Scientist. Of course, "computer scientist" is a broad label, and many of us do more engineering than science, although some topics (like computer performance analysis) share more methods with the natural sciences. I've also done troubleshooting / evangelism / competitive analysis / marketing / business-alliance work that is not so easy to categorize.

Google Scholar: JR Mashey will find a few publications, some of which are modestly well-known. Industry folks tend not to write as many papers as do academics. I was busy giving other talks (500+) and sales pitches (1000+).

2.2. [Mashey of Portola Valley ... wmanny asked]

It is easy to find that we live in Portola Valley, a quiet small town next to Stanford University on the opposite side from well-known Palo Alto, i.e., PV is an edge of Silicon Valley. PV is an intensely environment-conscious town (even for SF Bay Area) with unusual educational demographics, populated mostly by corporate executives, entrepreneurs, venture capitalists, lawyers, doctors, senior technical people.

Nearby Sand Hill Road is the world's center of venture capital, next to Stanford's SLAC. Many surrounding towns are working hard on plans for reducing carbon footprints or dealing with expected sea-level rise. Plug-in car conferences are popular. Not commonly found in S.V. are "left-wing pinko socialist using AGW as an excuse to destroy American capitalism." (or something like that) :-)

An easy bike-ride away is Stanford, of course a very strong research university in many relevant disciplines, including climate, environment, energy, economics, computing. Many lectures are open to the public.

2.3. A Few More Relevant Background Items

• I grew up on a 100-year-old family farm North of Pittsburgh, PA (i.e., sort of mid-West). Farm kids learn early about taking care of fixed resources, about Liebig's Law of the Minimum, and about not over-interpreting day-to-day weather changes, i.e., we grow up dealing with "noisy time series" and knowing that sometimes averages matter, and sometimes variability matters more.

• As an undergraduate, I worked summers as a mathematician/programmer at the US Bureau of Mines in Pittsburgh doing data analysis for coal research, building statistics software, etc. Western Pennsylvania includes Appalachian coal country, relevant to assessing coal industry practices.

• Besides undergraduate math & physics (& computer science), I took several psychology courses, including statistics for experiments and some cognitive psychology. This was handy later at Bell Labs, when I built a group that was half cognitive psychologists. It is useful in talking to psychology professor friends about why people believe weird things, anchoring effects, Dunning-Kruger Effect, all-or-none personalities, psychology/sociology of scientific research. Such friends have been very helpful.

• In grad school, I did some operations research, stochastic processes, statistics, etc along with computer science.

• When I worked there, at its height, Bell Laboratories was ~25,000 people, mostly engineers and scientists. An MS or PhD was required to be a Member of Technical Staff. Besides Nobel prize winners, some of the world's best computer scientists worked there, as did some of the world's finest statisticians, like John Tukey and Joseph Kruskal. The Bell System collected vast amounts of data, and many Bell Labs people used statistical tools to drive major decisions.

Internal review of papers proposed for outside publication was usually tougher than most external peer reviews, and ones with serious statistics tended to be examined by the best, which enforced some discipline.

Such an environment tends to improve the accuracy of one's assessment of one's own abilities, because there is almost always someone around who knows much more or is much smarter, or both. In addition, bold, but ill-informed, statements get slaughtered quickly, so one tends to learn to ask questions.

• Wikipedia mentions me as a founder of SPEC, an organization technical computer people started in 1988 to downplay meaningless, counterproductive marketing "mips ratings" in favor more scientific measures, with careful data collection and disclosure. The "microprocessor wars" of the 1980s and 1990s included considerable marketing and anti-marketing efforts, but this helped improve the situation.

• While at Silicon Graphics, when not designing chips, software, or supercomputers, I spent 50% of my time talking to customers, who included many senior scientists and engineers, CTOs, sometimes CEOs or senior government people. I visited many universities and research labs around the world, covering many science and engineering disciplines.

I probably helped sell $500M of computers to petroleum geologists. I spent some time with climate modelers, some of whom still use members of an architecture family I helped design long ago. [SGI Origin 3000 / Altix supercomputers]. At this level, one may well spend a day with senior technical people, discussing the problems they are trying to solve, what they know, what they don't know, what they would do with more compute power, etc. To do this, one has to be able to at least talk the domain-specific language somewhat.

• These days, being half-retired, I do things like due diligence for venture capitalists, i.e., they pay me to be able to quickly assess the credibility of a team looking for money, and be skeptical of technology and business plans, and dig in quickly to understand them. Such work has a high level of ambiguity, and decisions must be made without always having clearly right and wrong answers. Of course, such work was internal business-as-usual in all 4 companies that employed me.

2.4 Climate Science Self-Calibration

Just for calibration, I'd rate my climate science expertise as perhaps 2 on a scale of 10, i.e., that of a technical professional with some relevant background who can read primary research, who understands the basics of statistics, physics, math and computing, and could probably keep up with undergraduates studying Earth Systems Science, although my knowledge would be spottier than someone completing a formal B.S. program. I'm probably more experienced at the general issues of skeptical analysis of misinformation.

This is nowhere near close to someone doing a PhD in climate science, much less established professional researchers, and of course, the top professionals are far above that. Ray Ladbury (who certainly knows far more than I on this) often writes of this calibration, as at RealClimate.

So, on to the questions.

Part 3 - Answers to Questions, Sources

Remember, my self-assessment is that I'm about a 2 on a scale of 10, i.e., nowhere close to being a climate science expert. This is just my set of answers, not claimed to be an optimal set that would be recommended by experts. Also, this set is mostly about climate science, not about energy, economics, and policy. That's a mostly separate set of references. First, one has to get the science.

3.1 What sources do you use for learning about AGW? Which do you trust?

Trusted Ones:

IPCC TAR, and then later AR4.

Books/reports by relevant national research bodies, like US National Academies, NRC.

Primary research articles in peer-reviewed journals like Science.

Websites of NASA GISS, NOAA, NCAR, UK Met/Hadley, US NAS, Australia's CSIRO, GFDL, NSIDC, credible scientific organizations, many of which I've had professional contact with. Given Swiss ancestry, I'm also fond of the Swiss Glacier Monitoring Network's website.

Books by real climate scientists [i.e. who publish peer-reviewed research, preferably current]

Lectures by climate scientists, other senior scientists & discussions with them. Direct exposure really helps, but of course, is only routinely possible in some places. BUT if you get any chance at all to see such people talk, it's worth it, especially for people who have long experience in quickly evaluating people's credibility by seeing how they speak and answer questions.

Blogs (primarily to keep up with current events, sometimes to learn; resources are way better than in 2001-2002, but of course, Blogs should never be primary places to learn science. Good ones help.).

RealClimate (clearly #1, since each starts with an article from a real climate scientist, and they actually answer reasonable questions, and have more patience than I would.)

Open Mind (for great data analysis and statistics tutorials)

Rabett Run, Deltoid, Atmoz and others often carry useful analyses of science an anti-science, and there are more.

Skeptical Science is a very useful resource. It lists commonly-repeated, long-refuted arguments on one page, explains each one accessibly for non-experts, but (crucially) cites peer-reviewed research.

Others, Websites (and later) Blogs

In 2001, 2002, or when a new one appears, these were/are used in the mode "What do they say? Are there any reasonable concerns that should be added to my list?"

(Deceased) Tasmanian John Daly's Waiting for Greenhouse was my favorite website in this category - he at least had panache and nice pictures, and was indefatigable. I learned a lot about AGW cherry-picking from studying his work and pawing through many surface station records off and on for a a year or so.

These days, I look in on such sites only occasionally, because I came to believe they weren't doing science. However, anyone new to this should take a look and do their own calibration, after they're seen the real science.

SPPI, ClimateAudit, Watts, IceCap, ponderthemaunder, JenniferMarohasy, SEPP, EPW & Marc Morano, many others once or twice. Somewhere I have a list of ~60 that took ponderthemaunder seriously, before I gave up looking at them.

It is actually useful to read such, but without substantial exposure to real science, one can fall into an "alternate universe" that has very different physics [as per Bill Ruddiman]. Some people's primary experience is such blogs, although it is difficult to get many to admit that. There appear to be certain ideological / economic / personality properties that encourage and support this, but that's another whole discussion for the future.

At this point, my main interest is to understand the flow of disinformation memes, the players, their motivations, and calibration of credibility (or incredibility) levels. I dig into specific cases as they come up, some of which I'll mention later.

a) Do you attend lectures by real climate scientists?

Typically several times a month at Stanford, sometimes elsewhere, at Bay Area government meetings, local town meetings, etc.

b) Do you have (or have had) personal contact & discussion of this topic with {top-notch real scientists}?

Yes, frequently on this topic, and many times on many topics over decades.

Here's a sample. See their biographies to understand what top-notch people are like, ranging from good researchers to long-established people at the top of the profession.

Needless to say, I'm very far below this league, but am lucky to talk to such people regularly and even ask a non-dumb question now and then. Of course, I subscribe to the ordinary mainstream position because I've studied the science to my satisfaction, not just because a few experts say so, i.e., the following is not an "argument to authority", it's a simply self-calibration.

People can write anything they want on blogs, especially anonymously.

It is a different experience to sit with a small group of neighbors 5 feet from a Nobel physicist who gives a clear, compelling talk about AGW, and then straightforwardly, easily answers every "What about this?" question from the audience and sometimes identifies disinformation by its origin from specific people he knows personally.

People who are really, really sure ... are stunned to suddenly realize they might have been sure of the wrong thing, and their expressions show it.

Here are a few of the people that have especially impressed me, who I've heard speak, seen answer questions, and (mostly) been able to speak with. They all subscribe to the mainstream science position.

• Professor Stephen Schneider, Stanford, NAS (member National Academy of Sciences).

I've heard him talk a handful of times, was lucky to be the "discussant" for one of his seminars. (I'd just read Lomborg's TSE and was still working through it, so I was supposed to be the "skeptic.") I learn something every time, as he's an excellent communicator as well as a strong multidisciplinary scientist. He is widely misquoted by certain people.

• Professor Burton Richter, Stanford, Nobel Physicist, NAS, past president of APS
  He gave a short version of this talk, Gambling with the Future at a little local town meeting. The abstract of that talk says: "We are already in a regime that has no precedent in the last 400,000 years, and these consequences are almost certainly bad if greenhouse gas concentrations increase unabated." His remarks were firmer.

• UCSD Professor Naomi Oreskes is a geoscientist/science historian, recently promoted July 2008 to Provost of the Sixth School at UCSD.

I've read one of her books, reviewed a few chapters of her forthcoming book, talked to her a few times, exchange email now and then. I've attended two of her talks, both of which had NAS members there:

The American Denial of Global Warming and
You can argue with the facts.

The Scientific Consensus on Climate Change: How Do We Know We're Not Wrong? is well worth reading, especially the general discussion of scientific processes.

The next three gave climate talks at a several-day Imperial College, London alumni conference last year in Cambridge. IC is the "MIT" of the UK, and we've attended many IC events with their faculty over the years.

• Professor Sir Peter Knight, Physicist, Fellow of the Royal Society (UK), Principal of Natural Science at Imperial College London.

• Professor Joanna Haigh of Atmospheric Physics at IC, a Lead Author on IPCC TAR.

• Professor Ronald Prinn, MIT Atmospheric Science, a Lead Author on IPCC AR4. He heads the MIT Center for Global Change Science, which includes Richard Lindzen. I was especially glad to finally hear him, as he moderated a famous debate between Stephen Schneider and Richard Lindzen in 1990, and about 10 years ago, he thought the scientific evidence on AGW was still "equivocal". MIT World references a video akin to the talk I heard. He describes changing his mind, as he is certainly no longer equivocal. I recommend his video especially for his discussions on uncertainty.

• Lord Ron Oxburgh, geoscientist, ex-Rector (head) of Imperial College, ex-Chairman of Shell. He is also a cross-bencher in the House of Lords. He's an old friend from many meetings, and as Shell Chairman said he's worried about climate and the planet.

• Dr Bert Metz, Netherlands Environmental Assessment Agency, co-chair IPCC WG III for TAR & AR4. I've heard him talk, talked to him some, exchanged some email.

• Professor Mark Z. Jacobson, Stanford Civil & Environmental Engineering. I've attended several of his seminars. He is very good researcher & modeler, testifies to Congress. His website has many interesting publications.

• Professor Rob Dunbar, Professor-Geological and Environmental Sciences, Director-Earth Systems Program, Stanford.

• Dr Dan Cayan, UCSD, Scripps, USGS. I heard him give this talk at a good local government conference on sea-level rise.

• Dr. Bruce Molnia, glaciologist at USGS, studies Alaskan glaciers, as they (mostly) retreat. He gave a nice talk at the Menlo Park USGS, with great before-and-now pictures like this.

• I've heard more talks by IPCC authors and other climate scientists, but I lose track, especially with panel sessions, and of course, have had occasional email conversations with others. Of course, during my years at SGI, I've spoken to/with many other climate scientists, as described in Part 2.

Following are a few more, who are not geoscientists/climate scientists, but top-notch people who are certainly smarter than I, have looked at climate science, can of course talk to top climate scientists as they wish.

• Dr Arno Penzias, Nobel Physicist, used to run Bell Labs Research division.

We have lunch/dinner now and then, and he's no climate scientist, but he's certainly looked at the topic, and works on investments in energy technology at a big venture capital firm.

• Dr John Hennessy, President of Stanford, NAS. John is an old colleague and friend of 20+ years, and has definite ideas about the importance of dealing with climate change, for example here, which certainly reflects one-to-one conversations.

• Peter Darbee, CEO of Pacific Gas and Electric (very large utility in Central/Northern California).
  This describes the way a nontechnical, but smart, person learns climate issues.

There are of course, many well-educated people around Silicon Valley who are working on climate and energy issues.

All these people could be wrong, but personally, I'd have to be a lot smarter and more knowledgable about climate science to tell them so, even if that';s what I thought.

Someone who takes the extraordinary position (mainstream is wrong) must be prepared to say to themselves:

"I'm right and they're wrong. I know more than these people do about climate science, or I'm smarter, or they're all in a conspiracy, or they're getting paid to lie, or something, because I'm sure they are all wrong, not I."

That's a different position than (legitimately) saying "I don't know much, not yet enough to know", but some say "I'm not sure" after years of reading blogs, which probably means something else. The people above who aren't climate scientists spent much less time to reach their conclusions.

c) Are you a member of any relevant scientific societies?

AAAS (American Association for the Advancement of Science)

I joined AGU for a while to track down articles, including MBH98/99 to better understand the "hockey-stick wars."

[ACM; IEEE CS - occasionally relevant for computational science issues.]

e) Do you read any primary research literature in the field?

AAAS's Science every week, and it often carries relevant articles.

I usually average 1-2/month from other sources, i.e., typically published papers on author's websites. I sometimes get preprints from authors. Right now, emphasis is on energy & economics, so not reading the pure climate science papers quite so often.

f) Have you read any books on this?

Yes. Following are some classifications, a reader's guide, and then an alphabetized list of some of those I own, with comments and some editorial opinions.

A On critical thinking, classical skepticism, weird ideas, self-defense against disinformation

B Popular-oriented books

C General books on climate by scientists

D Serious textbooks on climate science by climate scientists, which tend to be more technical, and sometimes are similar to E or to peer-reviewed papers.

E Major compendiums by official teams, IPCC, NAS, i.e., secondary compilations based on primary research generally found in peer-reviewed publications.

F Climate+politics, or science+politics, or structure of science, or other

Z Disinformation, or silly theories believed passionately, or careless exaggeration. Most of these started in some other category, but then I decided had too much distortion to stay there.

Note: this list generally avoids books whose main emphasis is intersection of climate, energy, economics, policies, solutions, i.e., where there is far more room for real argument.

Reader's Guide

Critical Thinking:

For general defense against disinformation of various sorts: BES2001, CAP1987, HUF1954, JON1995, KUR2001, MON1991, PAU1998, TUF1983.

Scientists can believe strange things and stick with them: ARP1998, EHR2001, EHR2003

Many people can believe really strange things FRA1986, GAR1981, GAR2000, PLA2002, RAN1986, SCH1994, some of which the originators believe, and some of which are hoaxes. Some retain belief even after the hoaxers show them how they did it.

Starting from Scratch on Climate Science (B & C)

If I had to pick one book to read, it would be RUD2005.

Useful popular books are GOR2006, MAN2008, REV2006. Normally, I wouldn't recommend a politician's description of climate science, but in this case, it's a well-presented, mostly-accurate equivalent of talks by many climate scientists. MAN2008 is a nice recent addition.

One might go on to GRA1997.

At some point, one should learn more of the history of this topic, via WEA2003, or through the first half of Naomi Oreskes's video "The American Denial of Global Warming." mentioned above. Many key basics of climate science are actually quite old.

Learning More (D & E)

Start reading more technical materials, such as HOU1997, HAR2000, MAC2003, KRA2003.

Then, it may be worth looking at US National Research Council reports and similar work, like {USGC2000, USNRC2000, USNRC2001.

An of course, sooner or later, if one is serious, one has to look at the latest IPCC reports {IPCC2007}, which can be found online - read the SPM, the TS, and then sample the full technical reports. Of course, by then, one might be reading primary research articles, at least occasionally.

Category E entries are especially authoritative and useful, as serious scientists evaluate and summarize current knowledge. Of course, they also get out of date, and in the IPCC's case, there is always a cutoff date for inclusion, which means the latest research is not there. As noted in Part 1, sometimes research that is 1-2 years' old may not survive challenge, so this is probably OK. The NRC reports tend to be much shorter and more topical.

Understanding Politics and Anti-Science (F)

Organized anti-science in the US grew up around tobacco, for which BRA2007 is an excellent and illuminating history. Most anti-science campaigns since have used similar tactics, some of which are described in MOO2005, MIC2008, WAG2006

It is common to ascribe anti-AGW funding to Big Oil, but IMHO, I think it is more due to Big Coal GOO2006 and family foundations whose fortunes were built at least in part on fossil fuels. I grew up near coal country and worked for the US agency whose job in part was regulating them. As a group, coal companies, at least in Appalachia, make oil companies look like the most enlightened and environmentally-conscious on the planet, although of course there is substantial individual variability. We'll burn all the oil we can, and natural gas is way better on CO2 than coal, which makes the gas folks happy. Coal folks know that CO2 restrictions are not good for their business, and some have funded extensive disinformation efforts.

Of course, most people who have strong anti-AGW beliefs have reasons for doing so other than getting paid.

For a detailed analysis of the tactics of anti-AGW non-science motivated by ideology or economics, a great source will be Naomi Oreskes' forthcoming book, but until then, start with the last 30 minutes from each of her two talks mentioned earlier.

For discussion of progressions in real science and real controversies, see KUH1996, ORE1999.

For an example of a scientist committed to an idea, see SVE2007, ARP1998, of which SVE2007 is often cited by others to deny AGW. It is difficult to know for sure, but occasionally scientists build a theory in an area of expertise, attempt to extend it far beyond as an explanation for many effects, and hold to it no matter what.

I consider "X, therefore not AGW" different from disinformation of the form "anything but AGW."

Disinformation (Z)

I originally placed some books in other categories, but came to believe they were (sometimes clever) disinformation (i.e., Z).

SIN1999, SIN2007 are Fred Singer's books. I picked up the first one very early, as Singer seemed to have a reasonable background, and at that point, "satellites != ground" was a legitimate concern, as seen in USNRC2000. My first thought was that Singer was just defending satellites, given his experience with them. In some cases, scientists defend "their" data no less fiercely than mother cats their kittens. I learned more later.

It is well worth comparing the two books, especially in light of the additional evidence that's accumulated in between. Of course, by the time I read SIN2007, I was much more familiar with Singer's actions, so was not surprised.

SIN1999 says GW isn't happening at all, but if it were, it would be good, and there should be no CO2 restrictions. SIN2007 says GW happens naturally every 1500 years, and there should be no CO2 restrictions.

At some point, I may do an additional Part that tracks through a comparison of these two, especially with regard to satellites.

ESS2002 is a book by Essex and McKitrick. At some point, I may do an additional Part that analyzes the McIntyre/McKitrick/Mann kerfuffle, using this book as context, and consider ideas why continuing this forever is attractive to some people, despite Wegman saying it was time to move on, years ago. For now, it is probably enough to review MOO2005 and understand the US Data Quality Act.

See SIM1996, LOM2001, LOM2007 for Bjorn Lomborg, but as discussed elsewhere, one really should read Julian Simon and the surrounding history for context. I originally gave Lomborg's TSE a "read it, carefully" rating in Amazon, before I'd had to time to really dig around, and before I'd gone back and reread Simon. I've written some pieces on this elsewhere recently, but maybe will pull them together later. Lomborg is probably has the cleverest anti-AGW tactics around.

Partial Book List (Should have Energy & Economy, but not yet)

[ARP1998] Halton Arp, Seeing Red - Redshifts, Cosmology, and Academic Science, 1998 (A). Arp is a fine observational astronomer, but doesn't believe in Big Bang or usual interpretation of red-shift.

[BER2000] John J. Berger, Beating the Heat, 2000. Arctic penguins - ugh; heart may be in right place, but counterproductive, IMHO. Alarmist! (B -> Z)

[BES2001] Joel Best, Damned Lies and Statistics - Untangling numbers from the media, politicians, and activists, 2001 (A).

[BRA2007] Allan M. Brandt, The Cigarette Century, 2007 (F). Oddly relevant, as the cigarette wars laid the foundation of anti-science tactics in the US, copied into acid rain, CFC, global warming and many other fights to suppress inconvenient science.

[CAP1987] Nicholas Capaldi, The Art of Deception, 1987 (C).

[DES2006] Andrew E. Dessler, Edward A. Parson, The Science and Politics of Global Climate Change, 2006 (F).

[EHR2001] Robert Ehrlich, Nine Crazy Ideas in Science- A few might even be true, 2001 (A). Physicist offers advice in evaluating crazy-sounding ideas; pp 5-10 is nice summary of evaluation criteria.

[EHR2003] Robert Ehrlich, 8 Preposterous Propositions, 2003 (A).

[ESS2002] Christopher Essex, Ross McKitrick, Taken by Storm - The troubled science, policy and politics of global warming, 2002 (F -> Z).

[FAG1999] Brian Fagan, Floods, Famines, and Emperors, 1999 (C). Fagan is an anthropologist, and writes readable books that sometimes show the climate effects on various civilizations.

[FAG2000] Brian Fagan, The Little Ice Age, 2000 (C).

[FAG2004] Brian Fagan, The Long Summer, 2004 (C).

[FRA1986] Kendrick Frazier, Ed, Science Confronts the Paranormal, 1986 (A).

[GAR1981] Martin Gardner, Science Good, Bad and Bogus, 1981 (A).

[GAR2000] Martin Gardner, Did Adam and Eve Have Navels - debunking pseudoscience, 2000 (A).

[GOO2006] Jeff Goodell, Big Coal, 2006 (F). Many blame Big Oil for funding all disinformation; look closer at Big Coal.

[GOR2006] Al Gore, An Inconvenient Truth, 2006 (B). I didn't learn any new science when I saw this, but since it essentially agreed with what real scientists said, with a few minor caveats, I appreciated the presentation.

[GRA1997] Thomas E. Graedel, Paul J. Crutzen, Atmosphere, Climate, and Change, 1997 (C). Readable, well-illustrated text by two heavyweights.

[HAR2000] L. D. Danny Harvey, Global Warming - The Hard Science, 2000 (D).

[HOU1997] John Houghton, Global Warming - The Complete Briefing, 2nd Ed 1997 (D).

[HUF1954] Darrell Huff, How to Lie with Statistics, 1954 (A). Classic, indispensable, cheap.

[IPCC2001] IPCC Climate Change 2001, (TAR- 3 volumes + Emissions Scenarios) (E).

[IPCC2007] IPCC Climate Change 2007 (AR4 - 3 volumes) (E).

[JON1995] Gerald Everett Jones, How to Lie with Charts, 1995 (A).

[KRA2003] Konrad B. Krauskopf, Dennis K. Bird, Introduction to Geochemisty, 3rd Ed 2003 (D).

[KUH1996] Thomas S. Kuhn, The Structure of Scientific Revolutions, 1996 (F).

[KUR2001] Paul Kurtz, ed, Skeptical Odysseys, 2001 (A).

[LOM2001] Bjorn Lomborg, The Skeptical Environmentalist, 2001 (F -> Z). Read SIM1996 first.

[LOM2007] Bjorn Lomborg, Cool It! (US edition), 2007 (F -> Z).

[LOM2007b] Bjorn Lomborg, Cool It! (Longer UK Edition), 2007 (F -> Z).

[MAC2003] Mackay, Battarbee, Birks, Oldfield, ed, Global Change in the Holocene, 2003 (D). Collection of many articles.

[MAN2008] Michael E. Mann, Lee R. Kump, Dire Predictions - Understanding Global Warming, 2008 (B/C). Very recent, popular-level guide explaining the IPCC findings.

[MIC2008] David Michaels, Doubt is Their Product, 2008 (F).

[MON1991] Mark Monmonier, How to Lie with Maps, 1991 (A).

[MOO2007] Chris Mooney, Storm World, 2007 (F).

[MOO2005] Chris Mooney, The Republican War on Science, 2005 (F). Science got really noticed by politics in WW II, and from then through the George H. W. Bush administration, science was mostly nonpartisan, but unfortunately, some very counterproductive politicization of science has happened since.

[ORE1999] Naomi Oreskes, The Rejection of Continental Drift, 1999 (F). Good history of real, long-running scientific controversy.

[PAU1988] John Allen Paulos, Innumeracy: Mathematical illiteracy and its consequences, 1998 (A).

[PIM2001] Stuart L. Pimm, The world According to Pimm, 2001 (C).

[PLA2002] Phlip Plait, Bad Astronomy, 2002 (A).

[RAN1986] James Randi, Flim-Flam, 1986 (A).

[REV2006] Andrew C. Revkin, The North Pole Was here, 2006 (B). Good general book with great photos.

[ROM2007] Joseph Romm, Hell and High Water, 2007 (F, B). Some climate science, more on energy and policy.

[RUD2005] William F. Ruddiman, Plows, Plagues & Petroleum, 2005 (C). Fine summary of world climate history, hypotheses/theories and science in progress, non-science. Clear and calm writing. From experience, it is fairly accessible to non-technical people new to AGW - I've given away copies to friends, and they have given away more.

[SCH1994] Jim Schnabel, Round in Circles, 1994 (A). People can believe in weird things like alien crop circles, even after "Doug and Dave" explained.

[SCH1989] Stephen Schneider, Global Warming, 1989 (C). Prof. Schneider was very early to articulate concerns.

[SCH1996] Stephen Schneider, Laboratory Earth, 1996 (C).

[SCH2002] Stephen Schneider & Terry Root, ed Wildlife Responses to Climate Change - North American Case Studies, 2002 (D). Some people worry incessantly about whether surface stations are perfect or not. Birds, insects, plants, animals, trees don't read thermometers, but they were already moving poleward, or uphill, if they could.

[SEL2004] Richard C Selley, The Winelands of Britain: Past, Present & Prospective, 2004 (F). Geologist/oenophile traces historical growth and shrinkage of UK wineries over two millennia. Current wineries are North of Medieval Warm Period and heading North quickly. Slightly out of date, a few vineyards are already in Leeds, Selley's projection for 2050. Visit the Loch Ness winery around 2100AD.

[SIM1996] Julian L. Simon, The Ultimate Resource 2, 1996 (F -> Z). Read this before reading Lomborg, and check Simon's affiliations.

[SIN1999] S. Fred Singer, Hot Talk Cold Science - Global warming's unfinished debate, Revised 2nd Ed, 1999 (C -> Z).

[SIN2007] S. Fred Singer, Dennis T. Avery, Unstoppable Global Warming every 1,500 years, 2007 (C -> Z). It is a good exercise to read SIN1999 and SIN2007, see what changes, and what doesn't change, especially in the light of major revisions to satellite and balloon results that happened between.

[STE1999] William K. Stevens, The Change in the Weather, 1999 (F).

[SVE2007] Henrik Svensmark, Nigel Calder, The Chilling Stars, A New Theory of Climate Change, 2007 (C, Z). The theory (cosmic rays) isn't new, the data clearly contradicts it, but is strongly held by Svensmark. Widely quoted by some to claim that CO2 has no effect, it's all cosmic rays.

[TUF1983] Edward Tufte, The Visual Display of Quantitative Information, 1983 (A). Most about doing it right, but Chapter 2 is about doing it wrong, and recognizing such. A truly wonderful and beautiful book, as are Tufte's later three, all of which are worth having for anyone who wants inspiration for good presentation of data.

[USGC2000] US Global Change Research National Assessment Synthesis Team, 2000 (E).

[USNRC1999] US Panel on Climate Observing Systems, Adequacy of Climate Observing Systems, 1999 (E).

[USNRC2000] US NRC, Reconciling Observations of Global Temperature Change, 2000 (E). This is a very good example of real scientists assessing uncertainties when different measurements disagreed. The disagreements mostly got resolved a few years later as major errors were discovered in satellite computations.

[USNRC2000b] US NRC, Issues in the Integration of Research and Operational Satellite Systems for Climate Research, 2000 (E).

[USNRC2001] US NRC, Climate Change Science, 2001 (E).

[USNRC2001b] US NRC, Improving the Effectiveness of U.S. Climate Modeling, 2001 (E).

[WAG2006] Wendy Wagner, Rena Steinzor, ed Rescuing Science from Politics, 2006. (F).

[WAR2007] Peter D. Ward, Under A Green Sky, 2007 (C). Extinctions, or how bad could it get sometime, by serious scientist. Alarming, but not alarmist.

[WEA2003] Spencer Weart, The Discovery of Global Warming, 2003 (C). Invaluable history - read this, or see AIP website.

g) Have you ever participated in peer-review as an author, reviewer, or editor?

A: yes, maybe a dozen times. Some even got accepted.

A&R: Internal reviews at Bell Labs.

R: Program Committee Hot Chips & USENIX & a few other conferences (every few years)

R: A few NSF proposal reviews

E: Guest Editor, IEEE Micro (twice); Program Chair/Co-Chair several times for Hot Chips and USENIX.

This is a modest level of such activity - academics and some industry researchers do much more.

h) Can you say anything about your background in physics and statistics?

Physics: until last term of undergraduate school, was planning to be physicist; skipped last course for computer science class, so just got BS Math, else would have had dual BS.

Subscribed to Scientific American since 1967, and followed physics somewhat since.
Worked summer jobs doing programming and data analysis with geophysicists at US Bureau of Mines.
Have helped design many computers used by physicists; many discussions with physicists at National Research Labs, climate modeling labs, universities, oil companies, etc.

Statistics: some study in high school. Several courses each in undergraduate and grad school. Frequent exposure to practical statistical methods at Bell Labs (home of John Tukey, Joseph Kruskal, etc).

Helped introduce more statistical techniques into industry-standard computer performance analysis, including the relevant section of the industry-standard Hennessy & Patterson computer architecture book. In last few years, gave invited lectures on statistical techniques for performance analysis at Stanford, Princeton, Cambridge, QMUL, U Texas Austin, UC Davis, Intel, SPEC, etc.

Obviously I am neither a physicist nor a statistician, although I have substantial experience in detecting misuses of statistics and graphs, and have studied disinformation tactics.

All this is a sample of things to study, and not necessarily an optimal set, more a subset of the books I have here and a description of my particular experience.

It's enough to understand the basics, and to know that others know much more.