Last Friday, in my post on Nature's comprehensive coverage of science journalism, I mentioned the recent Nature Biotechnology conference paper on science communications co-authored by scibling Matt Nisbet. I also said I'd come back to one of the points in it that bothers me.
As I said yesterday, most of the material in this paper (the issues of media fragmentation, framing problems, incidental exposure, etc.) has been expressed elsewhere. I agree with the majority of it, and it's nice to see it all in one place. But I have to take exception to a small piece of the paper - an example that I've heard Matt discuss twice before, and see again here: the framing of the National Academies' report, "Science, Evolution, and Creationism."
The Nature Biotechnology paper says:
Instead of relying on personal experience or anecdotal observation, it is necessary to carry out careful audience research to determine which frames work across intended audiences. Communication is both an art and a science. For example, the US National Academies (Washington DC) used focus groups and polling to inform the structure of a written report about the teaching of evolution and to frame publicity efforts. Their research indicated that an effective storyline for translating the relevance of evolutionary science for students was one emphasizing the connection to advances in modern medicine. Contrary to their expectations, the research concluded that an alternative frame emphasizing recent court decisions did not provide nearly as effective a message. (italics added)
OK: "contrary to" exactly whose expectations?!
The polling data must have been "contrary to the expectations" of some people involved with composing the report, because a 2008 CBE-Life Science Education paper by Jay Labov and Barbara Pope of the NAS, which is cited in the Nature Biotechnology paper, says,
Based on what was learned from understanding our audiences, the organization and presentation of sections of the final product are different than what the authoring committee and project staff originally envisioned. For example, the committee began its work shortly after the decision of Judge John Jones III was announced in Kitzmiller et. al. v. Dover Board of Education. The committee originally thought that this decision should be prominently touted throughout the booklet as one of the main reasons why various forms of creationism (including intelligent design) should not be taught in the science classroom - it's illegal.
This really puzzles me. I've never met a biologist who thought the best way to build an argument for the validity of evolutionary theory to the lay public - much less K-12 students - was around legal cases. If that was really the original plan, the NAS sure didn't ask me about it!
I do think the legal cases, like Dover, are important and interesting. I'd want to include them in materials for teachers and school boards, so they have some familiarity with the legal history of challenges to teaching evolution. Plus, the Scopes trial makes an interesting story. But descriptions of legal cases are hardly compelling reading for most people, who generally want to know how science is relevant to their daily lives.
If I may resort to the maligned "personal experience or anecdotal observation" (actually both): a few years ago, when I had to create a basic science literacy course for college non-majors, I structured it around the tangible benefits of evolutionary theory to medicine and agriculture - just like the NAS report did! I did that back in 2005, well before the NAS report was published. It seemed obvious to me as a teacher that I had to reach my students on their own terms, using issues that were important in their lives and their families' lives. Health is a universal issue of interest; plus, my college had lots of pre-nursing students. So I explained how we get antibiotic resistance and why vaccines need to be constantly updated. That's how I taught evolution. It just made sense.
My curriculum worked pretty well, even though I based it on personal experience with my community, and didn't use a focus group. Am I opposed to focus groups? No. I realize that when you teach, your students are your focus group; there was a lot of content in my class that could have worked better, and I took my students' candid feedback to heart when revising (and hopefully improving) the course. When it comes to preparing a static report, especially an expensive, high-profile report by a national body, running it by a few focus groups first is indeed important, so you don't waste valuable resources, or worse, generate a negative response. But I really dislike the implication that personal experience and anecdote aren't useful, because I think they're vital.
Let me be specific. I think it is absolutely vital in a democracy as diverse as ours that people with relevant personal experience, such as teachers with experience in schools that serve various socioeconomic groups and ethnicities, or nurses with experience treating patients in inner-city and rural clinics, be included in developing science outreach programs. My personal experience made it clear to me what type of "evolution frame" would work for my students.
That's why I find it unbelievable that this came as a surprise to the NAS team developing the report - the NAS has some pretty smart people on board, and the report's authoring board included some savvy communicators, like Neil DeGrasse Tyson.
So what really happened? I was just speculating, because I wasn't there. So I called someone who served on the NAS authoring board and asked. And what I heard makes a lot of sense. Specifically, at the time the revision of the NAS report was being designed, the Dover case was all over the news, and it was apparent that any NAS report would have to address public questions about it - especially since one important use of the document would be to present the NAS point of view in legal cases dealing with teaching evolution. The NAS report has a lot of potential uses - K-12 education, although it's the use emphasized by both the Nature Biotechnology and CBE-Life Sciences Education papers, was just one of them. So it makes sense that given the potential legal use of the NAS report, that legal framing would have been on the table.
But lots of things were on the table - the NAS report was never going to emphasize legal arguments to the exclusion of other arguments. As my source remembers it, the day-long authoring committee workshop included many different authors with diverse perspectives, including two high school biology teachers, ethicists, paleontologists, biologists, science writers and NAS staff. At least some of these individuals had previously advocated teaching evolution through practical, applied examples, and felt it was important to include that framing in the report. The committee's goal was to find balance - there was "no discussion about what was going to make up what % of the report." And when the polling came out, my source doesn't remember it being surprising. That doesn't mean no one on the authoring committee was surprised, but it certainly wasn't a universal sentiment.
So it appears the idea that the NAS authoring committee wanted a legal frame for the report, and was surprised by polling data indicating that would not be successful, is a bit of an oversimplification, if not misleading. Why does this bother me? I mean, in the end we got a great NAS report out of the process, right? Well, the reason it bothers me is that it makes it sound as if biologists don't know how to communicate with their students and the general public, and even worse, that personal experience is useless in science communication. I don't think that's true. My personal experience teaching low-income and nontraditional students in a small college in a conservative community is a vital influence on the way I frame science.
Last Wednesday, when I had to slip out of the State of Innovation Conference early, I had the good fortune to encounter the venerable sociobiologist E.O. Wilson in the elevator. (yay!) I spent almost five minutes talking to him; if there ever was a time in my life for an elevator pitch, that was probably it! But I didn't pitch anything. It was more important to simply thank him for his leadership in one particular area: communicating the importance of environmental conservation to religious America (as he does in his book, The Creation: An Appeal to Save Life on Earth
I grew up in a conservative religious town. The people I grew up with were neither bad nor stupid; most of them loved the natural world. They spent a huge proportion of their time outdoors - camping, fishing, hunting and hiking. But many of them couldn't relate to ecological arguments, because they perceived science - rightly or wrongly - as being materialistic and therefore anti-faith. They didn't see how their personal values could possibly be acknowledged, if a strictly evidence-based discourse was shaping policy. I know this probably sounds alien or bizarre to many scientists, but many people in the town where I grew up honestly felt oppressed by what they saw as powerful environmental lobbyists with sinister government scientists on their side. (You ecologists can quit rolling on the floor now; I'm serious.) See what I mean about the importance of understanding different perspectives?
E.O. Wilson tries in The Creation to make environmental arguments accessible to people who are working from a different worldview than most scientists. He doesn't patronize them. He assumes that most Americans want to do the right thing. And that's huge to me. If I had a copy of The Creation when I was a tree-hugging high school idealist with a Greenpeace sticker on her locker, I would've made every adult I knew read it. It says the sorts of things I wanted to say, but just didn't know how to say back then.
Anyway, I didn't get all that across to E.O. Wilson in the elevator. But when I mentioned where I grew up, he brightened and said, "I know just what you mean - I grew up in Alabama." When E.O. Wilson wrote The Creation, he was writing for people he knew - a community he was part of, and one he cared about. Those people are citizens of the United States. They one of the groups that science communicators and science policymakers should be most concerned about reaching. We can't write them off. And that is why it is so important that we bring different experiences and perspectives to the table: because as convinced as we may be of the importance of science, there are perfectly reasonable, genuine, caring Americans who do not understand science, do not relate to it, and do not know why they should bother to invest their time in it. If that statement is "contrary" to your expectations, perhaps you should try engaging your cabdriver or airplane seatmate in a genial debate about evolution (I've done both). It might be startling.
If we are going to succeed in improving public science literacy, we can't do it from a bubble or an ivory tower. In the real world, the perspective bestowed by personal experience - ideally, a diversity of many perspectives and experiences - can sometimes be more valuable than all the focus groups or polling data in the world. At least that's my take on it.
- Log in to post comments
For what it's worth, a lot of people did, and still do, think that describing legal rulings is the best place to start. Perhaps on the theory that people who don't accept scientific authority will respect the secular law or something. And the polling NAS did didn't just warn them away from bad messaging strategy, but toward rhetoric that was more effective, emphasizing practical applications of evolution and defusing religious issues that can be a barrier to any effective communication.
I disagree with one point you made, that we should not be so concerned with the question of whether science contradicts faith; it does, and let's just accept that. Religion rots much of your critical thinking capacities, and that's one of its primary functions. That you and others manage to mostly overcome this inculcation is a testament to your innate intelligence that managed to drag you from the pit. However, one must compare this to the literally millions who not only remain in that swamp but wallow in its fetid waters.
Maybe those people want to do the "right" thing, as you suggest, but their wants and their actions speak in opposite directions and I tend to take the tack of believing one's actions speak more loudly that non-effected "wants."
Having grown up, and spent far too much of my life, with the deeply religious, I have grown weary of trying to explain basic scientific knowledge to them. Trying to explain the unbelievably simple mechanism of evolution, not to mention the notions of abiogenesis, elicits nothing but biblical quotations and rejections. Finally I just gave up; I will explain science in many, many ways to many kinds of people, and if strongly religious people don't want to be condescended to, maybe they should try acting less like five-year-olds.
I refuse to extinguish the million-watt searchlight of knowledge and truth that is science in order to pander to those who still hold candles.
Isn't a focus group basically a tool for generating anecdotes at a predetermined time?
I think you have just nailed the point that was missed by both sides in the framing wars.
Arguments are best framed in terms of common experience between communicator and audience. Focussing on the application of science in medicine, ecology and so on bypasses both the "exclusionist" and the "accomodationist" arguments, firmly grounding the discussion in terms accessible to both sides without pandering to any particular viewpoint (other than that science works!).
Josh: What bothered me about the two papers is that they seemed to switch the basis for evaluation, from what the authoring committee was asked to do (create something to serve many different uses, including in courts) to focusing on efficacy in K-12 education. The report is not solely for use in K-12 education, but the two papers leave the impression that is its main purpose. And yes, if you think that describing legal rulings is the best way to start in creating resources for use in K-12 lectures and classrooms, then I do strongly disagree with you.
Blake: kind of, yes. Focus groups don't have a large enough n to satisfy me that they are significantly more powerful than anecdotes, or as I prefer to think of it, aggregate personal experience (since I taught evolution to around a hundred students in multiple classes). But on the other hand, at least focus groups can control for some things, which is better than none. Personally, I think the best way to assess curricula is to test them in the classroom repeatedly. But since the NAS report is not a curriculum, I can see why they went with a focus group. The point is not to criticize that choice, but to call into question whether there really is such a big difference in value between personal experience and focus group testing. Which is basically, I think, what you're saying.
dominich: Thanks. Yes, I am all about bypassing the drama in this particular area and focusing on practical outcomes.
Damien: "Maybe those people want to do the "right" thing, as you suggest, but their wants and their actions speak in opposite directions and I tend to take the tack of believing one's actions speak more loudly that non-effected "wants.""
I was describing specific people I know, that I grew up with. You don't know those people, so you can't assess "their actions."
It appears from your comment that your experience with religious people has been exclusively negative. But religious people do not have a monopoly on being idiots and assholes, and you can't generalize about all of them with accuracy. They're a mixed group of people, many of whom I've personally found to be basically uninformed about evolutionary theory, not vitriolically opposed. You may have different personal experience, but yours doesn't trump mine.
I refuse as usual to get into the "is faith incompatible with science" debate; that's what PZ's comment section is for. My point is about messaging: unless you have an accurate understanding of your audience, you can't craft good messages to reach them. You don't have to agree with your audience to craft a good message - but you have to be able to empathize. And wow, there are a lot of scientists who very vocally have NO empathy for the religious public. I'd prefer to do my own messaging vs. leaving them in charge, thank you.
Looks like you have added another title to my list of "must reads." I belong to a religious community that generally leans toward conservatism and some people I know clump environmentalism with other issues perceived to belong to a liberal agenda. Therefore environmentalism is sometimes dismissed out of hand. It is maddening and maybe meaningful dialogue is not a realistic expectation, but it is certainly worth preparing for. Thanks for your intelligent & consistently reasonable voice. :)
It's possible that people whose main job involves developing policies and regulations tend to find legal arguments particularly persuasive and expect other people will as well. I have no idea if that is true of the people who wrote either the paper or the report.
John, that was why I called someone who is a biologist at a research university and served on the authoring committee, and why I found out more about the authoring committee that wrote the report (as described in the post). The people on the authoring committee were diverse: it probably included some policy people, but sounds like it was mainly scientists. Perhaps the "surprise" was on the part of NAS staff responsible for managing the report's composition, I don't know.
Jessica,
Thanks for this valuable post. Personal experience, professional training, intuition, and talent are extremely important in science communication, as you wisely observe and very effectively argue. I also enjoyed the anecdote about EO Wilson, a scientist who drew on his personal background and intuition to craft a compelling narrative about environmental science for a non-traditional audience, in the process bridging perceptual divides on subject.
I would add, however, that when science organizations or universities have available resources to invest in a major public engagement initiative--whether on climate change, evolution, nanotechnology etc--it makes sense to go beyond personal experience and intuition and to take a scientific approach to designing and structuring that initiative.
An ideal research protocol would begin with a review of the relevant past social science literature in the area, followed by focus groups and/or in depth interviews. This qualitative, small "n" research would then suggest additional research questions and hypotheses that can be examined and tested in the analysis of nationally representative survey data or surveys specific to targeted audience segments.
This research would provide answers on how to structure information in a way that is accessible, interesting, and accurate to key audiences; that reaches audiences through their commonly used media and communication channels; and/or the types of partners and other opinion-leaders who might be trusted collaborators in communicating about the topic etc.
There's another article coming out later this year that reviews many of these issues and others in much more detail.
--Matt
RE: Wilson's Creation. I think it was an admirable effort, but I wonder how much impact it actually had among the audience he was trying to reach. Christians I know who did read it felt that Wilson came off as a bit condescending and were more likely to get on board with environmentalism because of their pastors more than the venerable scientist from Alabama.
This isn't to say Wilson's book had no effect. Maybe it influenced some priests or pastors who then became more concerned with global ecology (a sort of trickle down effect). Despite Wilson's effort, though, the people who I have seen complimenting The Creation are people who already agreed with Wilson in the first place. Is there any indication that it really reached his intended audience?
Brian, those are really good questions. I don't know if Wilson's book succeeded in persuading anyone of anything - it would be nice to think it did, but there isn't any outcomes data. I know what you mean about the condescending tone - I winced a few times. On the other hand, it's not like Wilson should have pretended to be anything other than what he is - an evolutionary biologist - or pretended to agree with his audience.
Anyway, I'm okay with the book's uncertain outcomes, because I think it was successful simply by opening the dialogue. It's not like there are many books by scientists written in an explicitly conciliatory way towards Christians. In fact, I can't think of any. . . do you know of some, Brian?
My own anecdotal experience with Wilson's book was that I could not convince anyone to read it. I like to think The Creation would have changed their minds, had they read it, but they did not read it.
My understanding of the main point of "messaging" is to get out a specific message. So what is the message you're trying to get across? My message has always been "this is why evolution is true, and this is what science has brought us." The opposition I've received on both of these fronts has convinced me that there is no way to properly convey this message that will not either muddy the core point or anger the audience.
No, you're right, your experience with religious people is not trumped by mine. It should be, however, trumped by what is happening in reality. And the reality is that the majority of religious people in this country do not believe in evolution. We rank second to Turkey in the number of people who disbelieve it. Is it because they're misinformed? Maybe. But I would argue that those doing the misinforming are to blame, and those people are almost certain to be religiously motivated. The people who vote in this country vote for people who disbelieve evolution; should we tailor our message to appeal to these people?
No, I don't think so. The mushy religious middle who can be swayed by works and information will be swayed; it is the dangerous elements in our society that openly and vigorously reject basic scientific knowledge that must be dealt with in mockery.
That's who I've hung around, that's who is the enemy.
Jessica; I assume you mean by agnostics/atheists/secular humanists/&c. addressing Christians and not the slew of books by the likes of Ken Miller, Francis Collins, &c.? In terms of evolution, at least, there is a huge amount of literature by scientists for Christians who are more fundamentalist than the authors.
In terms of books by non-Christian scientists for Christians, however, it is hard to think of examples. S.J. Gould's Rocks of Ages most immediately jumps to mind, and some of Carl Sagan's work straddles the boundary (criticizing religion while talking about science in a religious context, as in The Varieties of Scientific Experience).
Writing a conciliatory book is difficult if you do not share the beliefs of your target audience. Maybe scientists fear they run the risk of offering solutions to pacify an audience that they themselves cannot accept. Trying to find a middle ground is damned hard to do.
Brian: yes, I was talking about books about science, as opposed to books about why-I-can-be-a-scientist-and-a-Christian-and-I-can-still-sleep-at-night-and-so-can-you.
The latter genre I don't feel is about science so much as theology/philosophy/biography. E.O. Wilson isn't trying to change the ideology/worldview of readers of the Creation so much as he's trying to explain why environmentalism isn't fundamentally in conflict, and may in fact be very consonant with, the worldview they already have. That's something very unusual that I rarely see.
Damien: As soon as someone starts framing things as "scientists vs. Christians" and calling other human beings "the enemy" and "dangerous elements in society" as you just did, I completely tune out of the conversation. It makes me really sad when scientists start using the same rhetoric used by fundamentalists and bigots.
Jessica; Good point. I think ecology is a topic that can be more broadly shared than some others. How deeply held certain beliefs are/how central they are to someone's particular religion would seem to determine how much common ground could be reached. Maybe that's why we don't see more of these kinds of books.
It's not "scientists vs. Christians," Jessica, it's "scientists vs. those who purposely retard scientific progress." And to be blunt, I find it ludicrous that you would disagree. The "enemy" is not a person per se, it is the rampaging ignorance in those people who will have evolution, global warming and vaccination explained to them over and over and over again, reject the empirical evidence sitting in front of them, and then go running around saying that they've seen the evidence and it's just not all there. The fact of the matter is that in this country, by far, it is most likely that those people are Christians.
I'm more shocked that you're able to look at our country where one major political party put up 7 candidates that publicly, on television rejected evolution, where clergy have more say over scientific progress than scientists, where a major spokesman for Catholicism can wave away the abuse of 30,000 children, where religious right violence has claimed 6 lives thus far in 2009; how you can look at all of that and still not think that the correct course of action is to take some of the piss out of irrational beliefs. Now, if we lived in Pakistan, I'd say that meant Islam, but we live in the USA, so it's Christianity.
It's not "science vs. Christians," it's "science vs. irrationality."
This is another fantastic post, Jessica. I'm jealous of your chat with E.O. Wilson! His stock rises and falls in my hero market, but he always ranks near the top. His recent efforts to reach "across the aisle" to people of faith are laudable, even if I agree that his lectures and writing on the subject are most well-received by folks like myself, lovers of science and theology.
I find myself belonging to what the media would have us believe is an American minority, a group that finds no inherent conflict between religion and science. As a painter and a writer, the right brain reverie that informs the studio time is similarly awakened by religious mysticism, meditation, or even a runner's high. I know that you've read some of my writing on wonder, the fundamental impetus of both science and reverent faith, but I assume that Damien has little experience with folks like me, those who elect to "believe in" the twin holies of science and worship (in whatever form the latter may take).
Admittedly, faith in the unseen integrity of the universe is at once rational and irrational, but I'm OK with that. The "enemy" of the scientific method and the spread of Enlightenment values is not magic or faith, Damien, but rather dogma, whatever the stripe. The only kind of ignorance that is "rampaging" is that which is willed and reactionary. Such reaction is born of allegiance to one particular set of ideas or doctrines. Incuriosity, then, is the enemy, Damien, not mere faith.
I get the feeling, Damien, that you'd hand me a black hat as soon as you heard me profess my penchant for theological rumination. Or perhaps, like Dawkins, you'd argue that my faith is not really the brand of faith that you're attacking. I identify myself as a panentheist, a label that Dawkins sometimes refers to as the Einsteinian type, and takes pains to exclude from the believers that he wants his army of atheists to line up against. The trouble is that Dawkins, when he says that people of faith are stupid and superstitious, is offending me, and he forces those of us in the middle to the other side of the line. Fortunately, I adore Dawkins, so even his mistaking biblical literalism for religion doesn't put me off his witty, incisive explorations of other subjects. Still, he could take some lessons on communication from Jessica!