Political opinion polls are very tricky. Answers to questions depend on the order they're asked in, and on precisely how they are phrased. If you ask people whether they're in favor of killing unborn children, you'll get a much different response than if you ask if there's any situation where women should be allowed to terminate a pregnancy.
What's even more difficult is to assess public opinion on complex pending legislation. Most polls find that most Americans like the idea of requiring everyone to buy health insurance. But it's only a slim margin -- 56 to 41 percent. Kevin Drum cited a recent study that asked a follow up: would you change your mind if low-income families got government assistance to pay for insurance? Now 34 percent of naysayers changed their mind to support the requirement. But Kevin wondered if it's a fair question. Wouldn't some people change their minds no matter what?
So Kevin suggested that someone do a study to see if some people flip-flop no matter what question is asked. If some people will always flip-flop, then that suggests poll questions about flip-flops aren't very helpful in determining what the "true" public opinion is. We decided to take him up on the suggestion. Last week we created a set of six opinion questions about issues we felt our readers were likely to disagree on. 491 people responded. For each question, we came up with two different follow-ups. So, for example, everyone was asked "Should the United States adopt a government-run health care system based on Medicare?" But each respondent saw only one of the follow-ups:
- Suppose the plan involved a required pay-cut for doctors of 10 percent. Would your opinion change?
- Suppose the plan involved a required pay-increase for doctors of 10 percent. Would your opinion change?
Each of these groups was divided in half again. One group just answered the questions, while the other group had to keep track of the total number of flashes of a flashing square (like this) in the corner of their screen while they answered. The hope was that this task might simulate some of the distractions a typical respondent might face while answering the polling questions over the phone.
So, did the flashing square affect whether respondents flip-flopped their answers? Here are the results:
This graph charts the average number of flip-flops each respondent made over the course of all six questions. While people who saw the flash flipped slightly more than the people who did not, the two groups were not significantly different.
But to return to Kevin's question, did we find that a certain number of respondents flipped their answers no matter what the follow-up question was? While it is true that someone changed their answer for each question, in some cases, very few people did. Consider the responses to the question "Should the United States withdraw all troops from Afghanistan?" Respondents were roughly evenly divided. But depending on the follow-up question, there was a big difference in the portion of flip-floppers:
While 35 percent of respondents said they'd change their answer if the US kept one base in Afghanistan to address only the terrorist threat, only 4 percent said they'd change their answer to the original question if the US also closed the prison at Guantanamo Bay. And this wasn't the only question that resulted in a small number of flips. There were a total of four (out of twelve) follow-ups that caused 4 percent or fewer to flip. Just 1 percent of respondents flipped their response about lotteries when asked "Suppose J.K. Rowling had never written the Harry Potter books. Would your opinion change?"
So to answer Kevin's question, it seems that the baseline level for flip-flopping is quite low -- perhaps as low as 1 percent, and certainly below 4 percent.
Interestingly, for nearly every follow-up we asked, the flip-flops were quite evenly distributed. No matter how you answered the original question, you were equally likely to change your mind. Is this a general phenomenon, or just a coincidence? Since it didn't happen every time, I think it's just a coincidence.
Here's an example where it didn't happen. The original question was whether recreational drugs should be legalized. When followed up with "Suppose the law also prohibited health insurance companies from providing substance-abuse treatment coverage. Would your opinion change?" the response to the original question had an important effect on flip-flopping:
If you initially wanted to legalize drugs, you were nearly twice as likely to change your opinion in this case as someone who had wanted to keep them illegal.
But how biased was our sample? There's no doubt that our audience at CogDaily is more educated than the average American. What about politics? We asked readers to rate their political philosophy on a simple scale. Here are the results:
As you can see, CogDaily readers are quite a liberal bunch. Does stated political philosophy affect your likelihood of flip-flopping? You bet:
While the differences between individual groups on this graph aren't significant, the overall trend is: the more conservative you said you are, the less likely you were to change your answer to the questions on this survey. Does this mean liberals as a group are wishy-washy? Possibly. Or it's also possible that we did a good job coming up with questions that liberals were likely to be ambivalent about: after all, that was our goal when when we created the study.
(Just a reminder: All Casual Fridays studies are non-scientific. This doesn't mean we can't use scientific principles to assess what's going on, but we can't make general claims based on the results)
- Log in to post comments
Your ideology explanations don't take into account the literature on cognitive differences between political ideologies. Specifically, conservatives prefer firm answers (they have a high need for closure), so if they came to their decision on the original question they would be less likely to flip-flop. Could liberals be "wishy-washy?" Sure, but its just as likely that conservatives are "rigid."
I can't speak for all conservatives, but I'm "rigid" as you say because I have a unchanging foundation upon which I base my decisions.
Mark B.,
I'd be curious to discover the data upon which you base your generalization.
I can only offer anecdotal evidence, but quite a few of my friends who are conservatives are "P" on the Myers-Briggs, and quite a few of my friends who are liberal are "J" on the "Structure" continuum.
Perhaps it is only "literature" to which you refer, as you contend.
Jost et al., (2003) Psychological Bulletin.
Shook & Fazio (2009) Journal of Experimental Social Psychology
Amadio et al (200?) Nature Neuroscience
Or a quick Google search with "need for closure" and "conservatism" as the search terms.
A quick Google search of Myers-Briggs suggests that it is an out-dated measure of personality with questionable validity and reliability.
None of these studies are perfect, but taken together they support my comment. Like most psychological evidence there will always be exceptions--perhaps these include your friends, but perhaps not. Nonetheless the scientific literature suggests that conservatives are higher on the need for closure etc. The data from this blog post support that assertion as does the data in the citations I included in this comment.
I merely suggested that Dave could have contextualized his ideology results (which I think are interesting) within this literature.
I'm sorry, to be clear I tend to shy away from value laden descriptions of psychological constructs. I only said that conservatives were "rigid" as a rhetorical counterpoint to the idea that liberals are "wish-washy". The same data can be described in the same way.
[angry comment arguing results are meaningless without error bars]
Conservatives are rigid while liberals are fluid? Except for conservatives who change their mind all the time, and liberals who don't, obviously.
Conservatives are definite and liberals are fuzzy? So conservative notions like "market forces" and "family values" are definite and non-slippery, whereas liberal notions like "gay marriage is good" and "climate change is real" are vague and unclear? No.
There's another small problem with such an analyses like the one Mark B gives at #1. The far-right are prominent and visible - think of Pat Robertson, Orly Taitz and the "conservative movement". The moderate right are also visible - think Colin Powell. The moderate left are actually in power in America, though you could be forgiven for thinking they need to do deals with the far-right.
So where is the far left? Ralph Nader and Michael Moore aren't socialists, whatever the Wingnut Daily calls them. Maddow and Olbermann are traditional left-democrats. The far left does exist, but it's almost invisible. There's Chomsky, I suppose - he's an anarcho-syndicalist.
The point is, the American popular image of "left" and "right" is skewed to make the moderate left look like they're the far left.
Maybe I missed it, but I'm a little surprised at nobody noting the obvious - that liberals/progressives, IMO thoughtful almost by definition, will reassess positions based on salient new information. Wishy-washy? Hardly. Howzabout anchored in the real world and its increasing demands for informed, flexible thinking, as opposed to reliance on simplistic, predetermined schemata which result from pre-conscious needs for emotional regulation. It's pretty simple, really - one approach attempts to see the world as it is (as much as that is possible); the other filters sensory info through the lens of personal, primary emotional needs. One approach is modern; the other archaic. Good thing epigenetic human evolution is happening much faster than previously thought.
Good and true points, Kapitano.
One thing I did not see any discussion of here were the questions at the end of the survey regarding general knowledge. Does that mean there were no interesting correlations?
Casey,
There was a small, significant positive correlation (r=.10) between quiz score and liberal political philosophy, but no relation between quiz score and flip-flopping.
So liberals did slightly better on the quiz, but otherwise it doesn't tell us much.
(The "quiz" was actually just two questions: Who wrote "Two Treatises of Government" and who is the President of France)
Liberal mush vs. Conservative dogma is off topic.
People in general are disillusioned with Government, albeit for good reasons. Perhaps the Liberal wants change to solve that problem; perhaps the Conservative wants more controls to solve the problem, but any case, what to do is a social conundrum.
Ether group is floundering in the quest for better Government. The idea that anything will make a difference may or may not be given tacit support. In my opinion, most people have a subconscious notion, and many a conscious notion, that everything we have tried has come to nothing.
Still, we tilt the windmills with a primordial yearning for betterment. Like going to war, that is an idea we could do without; like free health care, as if; like it is not too late to stop the runaway global warming; like religion can be separate from state; like we could find politicians who are not corruptible; like business can do anything different than to make the most Buck for the bangâ¦
Perhaps the initial problem is in the question. Surveys? There is no end to problems with trying to look at a multi-dimensional problem in a one dimensional world. Our minds consider many different aspects of an important question. These aspects or dimensions of an issue are inter-related. So it would seem that as you try to look at some important sociological struggle, any one of two different one-dimensional evaluations can appear to be conflicted. That may not be the case.
Life and death issues engender deep emotional responses. Emotional and rational responses are not always congruous. And yet, in my opinion, it is the whole person (mind, body, and spirit) working together that is really always at play. Many seeming dissonances between the three are resolved differently in each different case. In addition, each individual finds their own balance between the three.
I suggest that most anybody will have many complex ideas about many different aspects of Government. How many dimensions will that be? We must have many, maybe more than we would like to admit, facets to our convoluted thinking about government.
Do I want no government? Yes. Do I want government to guarantee my right to life liberty and justice? Yes. Am I conflicted? Perhaps, from one point of view, I am; but the truth be told, all of us desire seemingly dissonant goals. [Did I use the word âall?â] That would make this a general phenomenon.
The science of the mind is still in the very early stages of development.
à Hill