At Science today, contributing journalist Yudhijit Bhattacharjee reports on the decision by the National Science Board to drop discussion of survey questions about evolution from their 2010 Science Indicators report. As a reviewer of several previous versions of the report and as an expert who provided input and feedback on the design of the 2006 survey instrument, I have several thoughts on what I think Bhattacharjee in the article unfairly portrays as a "controversy."
The NSB is correct to be concerned about how these questions are interpreted by the public and by the scientific community, but by not including a discussion of these indicators in the report, the NSB missed an important opportunity to educate the science community about the nature of public opinion and the many social factors that shape public perceptions and understanding.
The NSB's decision was based on the accurate premise that the evolution question taps for the most part religious beliefs rather than factual knowledge. I've pointed to the validity issues surrounding this question in talks over the past few years including most recently in a lecture I gave last year at the National Academies (see discussion starting at 24 minute mark of video).
Consider what a split-ballot comparison in a 2004 University of Michigan survey revealed about the nature of responses to these long standing questions about evolution. In this survey experiment, one half of the sample was asked the following traditionally worded question:
True or false, human beings as we know them today, developed from earlier species of animals.
When asked this way, 42% answered true, a result that has been incredibly consistent across surveys since 1985.
The other half of the sample, however, was asked a slightly different version of the question:
True or false, according to the theory of evolution, human beings as we know them today, developed from earlier species of animals.
When asked this way, 74% answered true.
The implication is that context matters: Americans are not ignorant of what science says about human origins, in fact, as the second version of the question reveals, 3/4 of the public are familiar with the scientifically correct answer.
But when presented with the traditional version of the question, Americans are asked to choose between what they know to be the scientifically correct answer and their own religious beliefs. Therefore, as a direct measure of scientific knowledge, unlike the other items included in the scale measuring science literacy, the evolution item scores low in terms of validity.
As I wrote with my colleague Dietram Scheufele last year in a review article, the NSF Science Indicators reports have been used rhetorically by science advocates and pundits to reinforce a false deficit model about the public, one that decries widespread ignorance and that promotes a constant "science under siege" mentality. This outlook is distracting and harmful to public engagement.
It's also a myth. While Americans might score low on quiz like questions about technical areas of science, other questions reveal that Americans believe strongly in the promise of science and that scientists and their institutions hold almost unrivaled levels of public trust. Moreover, despite what many scientists and advocates might describe as an "enlightened" European public, the U.S. public scores as well if not better than many European publics when it comes to overall science literacy. The one indicator that falls below scores in other countries is the question about evolution. The NSB was therefore wise to be cautious about the reporting of this question.
Still, however, despite how the survey results have been misused and misinterpreted in the past, the NSB should have included a careful and detailed discussion of the evolution results in the chapter. I served as a reviewer on the 2006 and 2008 versions of the chapter but was not asked to review the 2010 version. My advice as a reviewer would have been to highlight in a call out box this problem with the survey question and in the process to emphasize that when it comes to public understanding of evolutionary science, the public is not ignorant, but instead divided in choosing an answer between what they know to be scientifically true and what they believe in terms of religious faith.
Including this type of context and explanation in the report would have gone a long way to improving the literacy of the science community, journalists, and policymakers about the complexities of science-society relations. This I hope will be a central goal of future chapters.
UPDATE: At the blog Thoughts from Kansas, Josh Rosenau, a staffer at the National Center for Science Education, and who is quoted in the Science article, links to a draft version of the chapter (PDF) that was uploaded at the Science web site.
Looking at the draft version, it does a very good job of explaining what I outlined above, adding to the Michigan survey by drawing on findings from Gallup. Why this section was deleted--given its value in shedding light on the complexity of survey response and public views on the issue--is not clear. Again, if I were a reviewer, I would have argued strongly for its inclusion, making it a highlight of the report. See specific text below:
Americans' responses to questions about evolution and the big bang appear to reflect factors beyond familiarity with basic elements of science. An experiment conducted in the 2004 Michigan Survey of Consumer Attitudes showed that respondents were more likely to answer these two questions correctly when the questions were prefaced by âaccording to the theory of evolution or âaccording to astronomers. These differences probably indicate that many Americans hold religious beliefs that cause them to be skeptical of established scientific ideas, even when they have some basic familiarity with those ideas (for additional details see NSB 2008).
Recent surveys conducted by the Gallup Organization provide similar evidence. A 2009 survey showed that more than half (55%) of Americans could correctly name evolution or another closely associated term, such as natural selection, when asked which scientific theory they associate with Charles Darwin. However, in a follow-up question, only 39% of Americans say they believe in the theory of evolution, 25% say they do not believe in this theory, and 36% do not have an opinion on this subject either way (Newport 2009).
- Log in to post comments
Oh, and I want to correct myself where I said that if 58% of the public are creationists, then 58% of the public are scientifically illiterate. This is not an accurate statement, it should have said "if 58% are creationists, then at least 58% are scientifically illiterate" because it's not as if not being a creationist automatically makes you scientifically literate.
This is not a false deficit model!
Knowledge of science is not equivalent to "knowing what the scientists say." Understanding and reconciling it into ones point of view is part of that.
Put another way, if someones religious beliefs conflicted with 2+2=4, if their religious belief indicates that 2+2=5, they're still wrong on a math quiz if they say 2+2=5. They still don't know math. Even if they can answer the question "Mathematician's belief that 2+2=?" with "4."
I'm a doctor, if one of my colleagues says they don't believe in the germ theory of disease, but they can pass medical boards by parroting the answers they know "the scientists" believe are correct, they are still a quack. They still don't know medicine.
As I wrote with my colleague Dietram Scheufele last year in a review article, the NSF Science Indicators reports have been used rhetorically by science advocates and pundits to reinforce a false deficit model about the public, one that decries widespread ignorance and that promotes a constant "science under siege" mentality. This outlook is distracting and harmful to public engagement.
B.S. If 58% of the public are creationists, then 58% of the public is scientifically illiterate, whether they know what the science says or not. Simple as that. Which side are you on?
GM,
I think your comment demonstrates my point.
--Matt
You aren't going to solve any problem by denying its existence
Matt, I think you're making a distinction where there isn't actually one. The statement "Human beings, as we know them today, developed from earlier species of animals" is a statement of scientific fact, and if someone disagrees with that, for whatever reason, they are disagreeing with factual knowledge. Yes, in most cases in the US the disagreement is due to religious beliefs, but it is because those religious beliefs are contrary to the factual knowledge, and not somehow independent of it. If you hold those beliefs, you don't accept that the statement is factual.
You're right that those with such religious beliefs are generally aware of what the scientific consensus is, but that is not the same as saying that they accept such consensus as "factual knowledge". And yes, I take that to mean that science is under siege from the religious fundamentalists in the US.
The chapter is about public understanding of science. If you answer false to the question, "human beings as we know them today developed from earlier species of animals," you lack understanding of science. It's not any more complicated than that.
The issue demonstrated by the two versions of the question is not one that allows a metric to be constructed.
What it does point out, is an important limitation in questioning people about anything at all. Questions are likely to be construed differently by a variety of responders due to differences in their perception of the semantic underpinnings of language.
As soon as you start asking questions using English, the variety of correct answers widens in the subject though the perception of the questioner does NOT.
If you fall back on definitions, as most scientists do, you fail to understand the nature of language and the nature of human cognition.
"when it comes to public understanding of evolutionary science, the public is not ignorant, but instead divided in choosing an answer between what they know to be scientifically true and what they believe in terms of religious faith."
Nope, they're not ignorant allright; just stupid as a dead monkey's arse..
Thank you for your insightful and subtle article. Fundamentalism (in religion, as well as in science, as demonstrated by many of the comments here) is just one reason people don't do what the data suggest. If we want to communicate, and generate solutions, we need more thoughtfulness, not more name-calling.
Matt,
I generally agree with what you're saying here, but I don't see how those two draft paragraphs fail to promote a "science under siege" mentality, as you say. Your point that a wrong answer to questions like these can be due to either ignorance (scientific illiteracy) or disagreement is a good one, but it is really better for scientists' views of religious fundamentalism that religious beliefs are actually in conflict with scientific fact rather than that religious people don't know what the science says in the first place? To my mind, the former sounds much more like "science under siege".
Surely disagreement is in general a bigger deal than ignorance - it's at least much harder to correct.
Gosh, all this chatter makes it sound like we scientist don't use things like hunches, intuition, extrapolations, etc. My insight is that most of the published matter on this subject (and others) hardly ever gives full reference back to the source of the claims of fact. It seems to be adequate only to say that 'this' is a fact.
Imagine I state that I understand: "Scientists tell us that the center of the earth is very hot," but then tell you, "But I don't believe that's true." I'm asked in the survey "The center of the earth is very hit", and I answer "False". [This is an actual survey question.]
Should my answer be excluded from the data because I'm aware of what the scientific consensus is?
Or, should me answer be excluded because I have a deeply-held, spiritual belief, that the Atlanteans live inside a hollow Earth, and manage the machinery of the weather from there? (Surely, this question would then conflate knowledge and belief!)
It seems to me that the point of the survey questions is to understand what portion of the population understands science and technology. That the center of the earth is hot is a fact. That humans evolved from other animals is a fact. (I express no bigotry, bias, or "fundamentalism" in those statements - I base them on the knowledge of ample evidence in support of both, and a complete lack of any credible evidence to contradict either. I am, of course, open to being further enlightened - by evidence, not opinion or argument.)
Or - perhaps I'm misinterpreting the delicately-phrased argument here. Perhaps what's really being said is, "These answers are invalid because respondents are culturally required to lie when asked about them, and provide responses they know to be false." Certainly, a more honest phrasing, but also, certainly, an important topic for discussion.
Matt: I appreciate your update and would note only that a similar discussion has been included in SEI reports since at least 2004, possibly earlier.
I question the premise that knowledge of the scientific consensus about evolution is the same thing as understanding evolution. To know something might reasonably seen as being able to parrot it back to a questioner. Understanding requires something deeper; being able to do more than simply recall something that can be stated in a sentence or two.
I agree with Matt that there is an important distinction between willful scientific "ignorance" (knowing the scientific consensus but rejecting it in favor of superstition) vs. simple ignorance. I do somewhat disagree about this "false deficit" characterization though (the reasons why we lag behind the rest of the developed world on this question do not make it any less shameful), and strongly disagree with this idea that making the proper distinction leads one to realize that science is not under siege after all -- on the contrary, it seems to me that this is more ammunition for the argument that science is under siege from religion. If it were mere ignorance, it might be explicable in terms of science failing to reach out sufficiently to the public. But these people aren't ignorant of the science, they just don't care...!
In particular, what this indicates is that giving students a more comprehensive education in evolution, while still a worthwhile goal in its own right, is unlikely to alter the indicator. Rather, what this suggests is that the problem needs to be attacked at its root: religion, or at the very least the special deference reserved for religion in the United States.
Frame that. :p
What everyone seems to be missing is the answer to another question. One-third of adults do not know that the Earth orbits the Sun in a period of one year. Even the Catholic Church has given in on this one. When such a basic, noncontroversial question is answered incorrectly, we are in BIG trouble. Science really is under siege.
So the NSF is adopting the same model as some American public schools. If the kids aren't getting enough questions right, eliminate the questions they keep getting wrong! Do this a few more times, and we'll all be geniuses (on paper).