Citizen science

DSCN5987-no The normally sensible Bart went slightly gooey over Citizen Science recently, although to be fair he linked to his earlier article wherein he was distinctly more dubious about the benefits, and indeed about the good intentions about some of the wannabe scientists. And in the comments he is more equivocal.

Then by coincidence I ran across Astronomers thankful for return of Jupiter's belt where Phil Plait again lauds the amateur:

Astronomy is an awesome science: it's one of the few where dedicated "hobbyists" can contribute, and do so in a critical and timely way. It's a big sky, with a lot to observe. And if I may say so, I'm thankful there are so many keeping an eye on it.

But notice that whilst the amateurs are contributing - by making observations - they aren't actually doing the science. Just making observations is stamp collecting. Working out what is actually going on is the science. Contributing meteorological observations isn't doing science, even if it is helpful (though it doesn't seem to be any more; as I recall the Met Office mostly gave up on encouraging amateur obs as the quality control was poor in some cases). In fact if I was taking a hard line I'd say that even ClearClimateCode isn't science - its coding.

Incidentally I'm not claiming that amateurs or "citroyens" can't do science - of course they can. In principle. But doing worthwhile science is hard (there are enough people formally called scientists who aren't really doing anything worthwhile, after all) and doing it as a citizen is harder. Most of those who think they are "citizen scientists" are just fooling themselves and contributing noise to the blogosphere. Much of this encouragement of the CS looks rather like the kind of patronising "oh haven't the little darlings done well" sort of stuff that people get to coo over their infants pre-school work.

Incidentally I'm not a scientist any more. Though I may occasionally do the odd very minor bit of science. But nowadays I find stuff like this exciting (go on, have a look) and think "oooooh, I can't wait until monday to try it out".

Update: SE has a great post on Why GCMs don't need IV&V.

More like this

Following my analysis, millions of persons all over the world receive the loans at various creditors. Therefore, there is a good chance to get a sba loan in all countries.


By gay!!!!!!!!!!!… (not verified) on 22 Jan 2012 #permalink

"among the many wonders of Vim is the fact that it supports Perl and Python embedded in Vim scripts."

They've done more than just coding, though. Now that they've finished replicating GISTemp in Python, they've been asking and answering questions about the temp record, as well.

Gavin Schmidt, some time back on Real Climate, said that a useful "citizen science" project would be to put in the work to grab data that's not yet been incorporated into GHCN, and massage it so it can be used together with the GHCN data to increase coverage, etc.

So recently, the CCC guys have scraped monthly Environment Canada post-1990 data off of the EC website, and added to the GHCN data (there's a huge drop in station numbers there starting in 1990), to investigate what effect this has on the ccc-gistemp product.

Now, they've written this up very informally - here's the money quote:

The additional Environment Canada [data] is welcome, and does affect the result just enough to be visible, but the trends and any conclusion one could derive are not affected at all.

but what they're doing with their new toy is close enough to science for me. Not earth shaking, just some fundamental stuff which tends to confirm the robustness of the GISTemp algorithms, but low-level science nonetheless.

Oops, sorry, didn't mean to paste that Vim comment in there from the blog post you referenced.

Having done so, though, I do hope your excitement is over the Python, not Perl, support ...

I have a slight disagreement, if not much.
Gathering good data is an important part of science, even if it is not science per se.

Note that data-gathering varies:
1) At one extreme, if somebody notices something and announces it, many others can look and verify it.

2) At the other extreme, the data is simply unverifiable, and its quality depends totally on the accuracy and correctness of the source.

At least some of the astro data is of the former type, and given the nature of telescopes, they have to be aimed in the right direction, and the more eyes looking, the better.

[Fair enough; there is always an edge where obs shades off into science. It isn't desperately important for these purposes where it is. And of course observations informed by the science are more valuable than mere uninformed observations, mostly for your part-1-ish reasons -W]

By John Mashey (not verified) on 27 Nov 2010 #permalink

Citizen Science strikes again.…

With an interesting bunch of amateur and not so amateur scientists.

Even before publication, Slaying the Sky Dragon was destined to be the benchmark for future generations of climate researchers. This is the world's first and only full volume refutation of the greenhouse gas theory of man-made global warming.

Nine leading international experts methodically expose how willful fakery and outright incompetence were hidden within the politicized realm of government climatology. Applying a thoughtful and sympathetic writing style, the authors help even the untrained mind to navigate the maze of atmospheric thermodynamics. Step-by-step the reader is shown why the so-called greenhouse effect cannot possibly exist in nature.

By deft statistical analysis the cornerstones of climate equations - incorrectly calculated by an incredible factor of three - are exposed then shattered.

This volume is a scientific tour de force and the game-changer for international environmental policymakers as well as being a joy to read for hard-pressed taxpayers everywhere.

Looks like the rabett has had his refutation of G&T debunked, and this book will be a standard text for climate scientists.

[Strong claims there. I bet S Fred is pissed off with them - shirley *he* has fully refuted GHG theory, not once but many times? However, these are not humble citizen scientists - Ball at least is the professional's professional -W]

4 Snide -- "Looks like the rabett has had his refutation of G&T debunked, and this book will be a standard text for climate scientists."

Hey Snide, look up one of your new fave authors, Professor Oliver Manuel, on google.

By deft statistical analysis the cornerstones of climate equations - incorrectly calculated by an incredible factor of three - are exposed then shattered.

This volume is a scientific tour de force and the game-changer for international environmental policymakers as well as being a joy to read for hard-pressed taxpayers everywhere.

"Roll up! Roll up! Get yer snake oil 'ere! We're 'ere to take away all yer pains and worries, Guv'nor!"

Looks like the rabett has had his refutation of G&T debunked, and this book will be a standard text for climate scientists.

Cool, this means that I can hold off buying a plug-in hybrid and just wait for Snide to start marketing cars powered by perpetual-motion engines!

Hey Snide, look up one of your new fave authors, Professor Oliver Manuel, on google.

Thank God the sun is made of a dense element like iron, rather than light elements like helium or hydrogen. We know the latter can't be true, if it were, the sun would just float away like a zeppelin!

I had to look up "gooey" in the on-line dictionnary, and still not sure how it applies, but I get your gist...

[Not really a word. In case it didn't become entirely clear, "gooey" is how people become when they partially melt - think of the reaction of a young swain in the presence of his love. So not entirely fair of me - more mood music -W]

I guess I wanted to counterbalance my previous rant about climategate (… ) by really searching for the positive/constructive, even if merely by the absence of the negative (i.e. energy better spent on analysing data than on ranting back and forth).

I don't disagree with your take on it (as is evident indeed from my longer post on the subject which you also linked to), though I have a less absolutist view of what is and isn't science.

[Attempting to clearly define the boundaries of science is impossible. I suspect we're largely in agreement on this, and indeed I'd noticed your attempt to play up the positive aspects, which is a role I rarely take :-) -W]

Let's not forget about the finest example of "citizen science", Watts' surfacestations!

"The data now is at 88% of the USHCN network surveyed, well over 1000 stations. I have two separate teams doing data analysis independently. The first team has completed its task and the conclusions are in, the second team is within about a week of finishing. All that remains then is to finish the narrative. There are a number of well known names in the scientific community coauthoring this paper with me.

A paper is forthcoming very soon for submission to a journal. When the paper is published, all current data and
analysis methods use will be made public online.â

That was written 7 months ago, and they still haven't 'finished the narrative'. Maybe they should ask dr. Curry for help?

[Well, to be honest, I would be prepared to count SS as citizen science - but only if they are prepared to do it as science. My suspicion is that they have the obs, they've done the analysis, and they don't like the results - which is anti-science. Time will tell -W]

Cool, this means that I can hold off buying a plug-in hybrid and just wait for Snide to start marketing cars powered by perpetual-motion engines!

I have always rejected the label of "scientist" for CCC, but recently we have started doing actual science: asking and answering actual science questions, and contributing to real science papers intended for publication. The Foundation is intending to do more of this (as well as a lot of other activities which are not science).

Having said that, there certainly are a lot of amateur scientists out there. Just not many in climate science.

By Nick Barnes (not verified) on 28 Nov 2010 #permalink

I can think of one example of citizen science that really got to the point of making scentific contributions and that's ACT-UP activists contributing to clinical testing and bench research on drugs for AIDS. The ACT-UP folks actually contributed in a significant way to changing the design of the testing protocols, influencing directions in basic research, and winning the praise of Anthony Fauci and Robert Gallo for making significant contributions to the way research was conducted. This was written up at length by Steven Epstein in his book, Impure Science (University of California, 1996)

Key excerpts:

"If the reforms won by activists are not to become mere stratagems for craven pharmaceutical companies swiftly to develop and market a whole series of additional nucleoside analogues (d4T, FLT, 3TC, etc.), activists must become more involved in the basic research process itself, forcing academic and industrial researchers to turn their attention to novel treatment approaches to HIV-induced immune suppression.â¦"
To get a better feel for the conduct of basic research, TAG members actually began spending time in Anthony Fauci's laboratory at NIAID. "I don't think they make any pretenses that they're immunologists or microbiologists or virologists," said Fauci. "But they want to understand as much of the down-in-the-trenches science as they can." Activists didn't need to comprehend every detail or nuance of the research, Fauci explained, in order "to evaluate the broad strokes of the studies that come out." Activists themselves were forthright about the limitations as well as the possibilities inherent in this new approach. "I can't talk cell lines with the big boys, that's for sure, but who cares?" commented Derek Link of TAG. "That's not my role."
Activists have cultivated these new relationships to bring the patient's perspective to the foreground in basic researchâto force bench scientists to be fully cognizant of the day-to-day realities of sickness and suffering beyond the laboratory walls. ... "I probably wouldn't have four drugs in clinical trials without the activists having had some [effect on] me," commented Robert Gallo in 1994. Another way that activists influenced the knowledge-making processes of basic research was by performing a bridging, or "pollination," function, bringing together researchers from different specialty areas who were unfamiliar with one another's work. Gallo, a participant in the Immune Restoration Think Tanks, agreed vigorously that activists have served as a "catalyst" that "forc[ed] people to communicate better" and to see beyond the limits of their individual specialty areas.

In comparison with the role of citizen science as it's discussed around climate change, there's a big difference because ACT-UP was not trying to second-guess the science community on matters of fact (etiology of AIDS, molecular pharmacology, etc.) but on practice: Is this the most effective experimental design for a drug test? Is this the best research strategy?

Where there were disagreements about statistical treatment of data (clinical researchers counted patients who had been put in the placebo group, but accidentally received AZT as placebo patients based on intent-to-treat rather than AZT patients based on what they actually got. ACT-UP citizen scientists were vociferous in pointing out that this muddied the statistics and demanded that subjects be classified according to what treatment they actually received), productive discussions ensued and scientific practice improved as a result.

It's noteworthy that I read a lot about democratization of science, yet this is the only significant case I know of mass citizen participation in something scientific other than observation, so the difficulty of finding the exception does tend to support what you say.

And regarding observation, don't think only of astronomy but also of biological observations, particularly the annual bird census, where amateur/citizen scientists make important and trustworthy observational contributions, but I agree with you that it's important to distinguish observational contributions with analytical ones (although I would not demean observation by calling it "stamp collecting").

["Stamp collecting" was a semi-deliberately-offensive term; I used it for reasons that will only make sense to me (and a very few others) so let me explain: when I joined BAS (~1990) the meteorology there was in the process of moving out of the stamp-collecting phase into the science phase (in a very broad-brush way). People there (my boss; and indeed me) had been deliberately appointed for this purpose.

Now for the birds, and the flowering shrubs: phenology, I think it is (not to be confused with phrenology :-). This is of some value, but I think often oversold, perhaps because it is easy to ge the public to help -W]

By Jonathan Gilligan (not verified) on 28 Nov 2010 #permalink

The "citizen scientists" of climate science are a varied group, many of who have strong data analysis skills and are wary of what they conceive as "arguments from authority"; they (incorrectly imho) conflate the existence of a scientific consensus based on the consilience of evidence with argument from authority. Yet, because of their skillset and critical (admittedly sometimes a tad uni-directional) nature, they could be powerful allies in the effort to increase the scientific literacy of the public. And in some cases they contributed useful analysis (you've said as much about McIntyre if I remember correctly), even though their framing rubs us the wrong way. I'm trying to find a way to see something positive/constructuve in their efforts and their skills.

In that sense I agree with Judith Curry, that the group she calls "citizen scientists" are a force not to be underestimated, with relevant skills. Whether their influence has been positive or negative on balance depends on the eye of the beholder, and on that point I start to part from Curry's view.

[Yes, I think I'd credit McI's original (by which I mean initial) contributions as science too. But of late (last 5 years?) he doesn't seem to have contributed much. I think it would be nice if some of the citizenry with analytic skills could lose their sceptic scales; the problem they then have, though, is what do they have to say? "Yes I agree" isn't very entertaining or original -W]

I think it was the female Leakey that said "Theories come and go, but a good data set is forever."

It makes more sense to me, I think, to speak of more or less observational sciences, rather then hiving off what amateurs do as not science.

That given, amateurs tend to be able to make larger contributions to the more observational sciences. Planetary astronomy is one example; Paleontology/Archeology are others.

Typically, amateurs are valued within a science for their observational skill (and local knowedge) over their theories, because they sometimes have crazy theories.

As a fool I wonder what the wise really should do? To not suffer fools, a common virtue for the deceased, or to suffer then gladly and risk an virtueless obituary?

As an aside, the UKMO are apparently looking to set up a new repository for public obs - though QC is one of their main worries at the moment. Still a long way to go though, apparently.

[Just a token I think; a gesture. With satellite obs, current public obs are less use than ever -W]

WMC & Bart:
could you say more about McI's actual science contribution?
Long ago, I thought maybe the PCA-centering discussion was worth a short note, but the statistics in McIntyre&McKitrick (2005) GRL piece is (I think) being demolished by Deep Climate. DC looked hard at the code, found that the Wegman Report used the same stuff, and that confirmed Dave Ritson's issues and worse, i.e.,

a) Focusing on the small fraction of processing that was the suboptimal decentered PCA.

b) Bad statistical-modeling, to generate more extreme hockey-stick graphs that reasonable parameters would.

c) Then, a 1% cherry-pick to use example(s) from the top 1% with the most extreme positive hockey-sticks. Wegman&co went even further in WR Fig 4.4, using a 12-curve graph that was generated by McI's script, but not used in the GRL article.

So, it's not clear to me that was science, but you guys know more than I about this, but maybe you guys mean something else?

[It was all so long ago. I think the answer is that there is, in theory, something of value to the PCA centering. But it was played up for far more than it was worth, and now looks rather as though Wegman / McI may have been somewhat economical with the truth in the manner of their presentation of results. I looked at this in 2003/4 I think ( and didn't find anything very interesting in the actual results. I don't think it is fair to say that DC has demolished this - he has pointed out the cherry-picking stuff you mention, which means that the practical impact is less than previously suggested.

All of which is rather deeply amusing; all the McI fanboyz never even bothered to read his code. But we knew that anyway.

To add: overall, I think McI's contribution to science has been negative: he did a small amount of good, but has contributed a vast amount of noise, and encouraged an unthinking distrust of science, which is bad -W]

By John Mashey (not verified) on 28 Nov 2010 #permalink

re: 12 WMC "Now for the birds, and the flowering shrubs: phenology..."

This reminds me of a delicious tidbit I encountered when carefully looking at the Wegman Report with microscope.

SSWR Appendix W.2.1 discusses the 3 errors in WR. p11 when copying a (correctly-attributed) table from Bradley. Of those, 2 were copied by McShane&Wyner(2010), but they missed copying the WR's misspelling of phenology as "phonology."

"Although paleoclimate researchers might find uses for 1000-year-old recordings of sounds from plants and animals, such seem unlikely to be found, whereas some cherry and peach blossom date records reach back before 1000AD."

By John Mashey (not verified) on 28 Nov 2010 #permalink

Anyway, in the UK isn't it "Subject Science"?

Don't forget Jean-Pierre Perraudin, the citizen-scientist founder (ok, an exaggeration) of climatology:

Obs are great for phenology, but not only for that. I also think citizen science might have a bigger role to play in applied science, like conservation biology. The native plant amateurs that I know here in California are pretty incredible, and respected by biologists.

[Things change as science progresses. Making accurate observations might well be a great contribution in the early days, and the same observations taken two centuries later quite trivial. The mutual respect from the plant amateurs to professionals might be relevant - the plant amateurs are going to learn something from interacting with professionals that the climate "skeptic citizens" never will, since they are so deeply contemptuous of them -W]