Another clueless article on the meaningless BBC phone-in poll

Posted by Tim Lambert on January 23, 2004
(12)
More »

Yet another columnist has demonstrated profound ignorance of opinion polling. Scott Norvell writes about the meaningless BBC phone-in poll (discussed earlier here and here):

Britain’s chattering classes sure can get their knickers in a knot with the will of the people offends their liberal sensibilities.

A phone-in poll does not represent the will of the people in any way, shape or form. Norvell compounds his error by leaving out important details about the shooting by Martin, like the fact that the burglar was shot in the back while fleeing and left to die. Norvell also seems to have based his description of Pound’s reaction on third-hand reports, mischaracterizing it as “apoplectic”. You can actually listen to Pound’s comments and find out that his tone was not “apoplectic”. Too bad Norvell didn’t bother.

What is worse is that Norvell isn’t just some random blogger or columnist, but London Bureau Chief for Fox News. Why can’t these people understand opinion polls?

Update: Eugene Volokh understands that such polls are meaningless. Here he writes about another meaningless poll on gay marriage. Unlike the BBC, the people who ran that poll (the American Family Association) reneged on their promise to put forward the results to lawmakers.

Glenn Reynolds, however, either does not understand or does not care. He posts for the second time on it, misrepresenting the result as “The will of the people”. I sent him a copy of this post and he added this update:

Tim Lambert emails (as I expected him to) that the poll is unscientific. Maybe so—but that’s an argument against the BBC using it—not an argument for discounting it after it produced a result the BBC didn’t like.

  1. I sent him this post and the link to it, but he does not link to this post, preventing his readers from seeing my arguments.
  2. The fact that the result is meaningless is, in fact, an argument for discounting it, regardless of what the result was.
  3. Stephen Pound did not like the result, but he is neither the BBC nor an employee of the BBC. And he did not discount the result but accepted that he would have to put forward to other lawmakers a proposal that he personally considered most unwise.
(12)
More »

Comments

  1. #1 Kevin Baker
    January 23, 2004

    And are more interested in ratings/readership (and advertising dollars) than in anything approaching the truth?

    Just a thought.

  2. #2 Kevin Baker
    January 23, 2004

    When you think about it, a telephone poll does reflect the “will of the people.” It reflects the will of the people who are willing to take the time to voice their opinions.

    You know, like voters do.

    Case in point: Poll after poll after poll tells pollsters that “the will of the people” is for more gun control, but the will of the people who vote is “not whatever it is you’re proposing.” This is another of those cases. People were told that whatever they voted for would be proposed as law. The people who cared voiced their opinions, and the one that won the most votes was the one that stuck in the politician’s craw.

    Kind of humorous, eh?

  3. #3 Tim Lambert
    January 23, 2004

    Kevin, it’s not like voters on gun control issues. At elections, voters vote for candidates, not issues. Voters who oppose gun control generally feel that it is a more important issue than those that support gun control, so the issue is more likely to affect their vote. That’s why, even if there is majority support for some gun control measure, it could still cost a cadidate votes, on net, to support it.

  4. #4 Michael Peckham
    January 23, 2004

    Tim, it’s obvious that Reynolds places his trust in this poll because he has a vested interest in the issue. He would just as glibly dismiss a poll as unscientific if it showed strong support for a ban on handguns above a certain caliber. As I argued elsewhere, what would Reynolds say if another issue in the poll (e.g., public smoking ban, which garnered strong support) came out on top?

  5. #5 Michael Peckham
    January 23, 2004

    Another problem with comparing phone-in polls with voters is that phone-in and Internet polls are easily skewed by repeat voters. Flooding the ballot box is generally harder to do in a free, democratic election.

    Nevertheless, I think Kevin has a valid point since many countries will hold referenda on contentious issues. This allows advocates from both sides to mobilize support irrespective of other factors (such as party representation) and may lessen the effect of one side feeling less strongly about the issue. Still, I suspect that a referendum is not directly comparable to a self-selected, easily rigged phone-in poll.

  6. #6 Kevin P.
    January 23, 2004

    Michael Peckham wrote:
    Still, I suspect that a referendum is not directly comparable to a self-selected, easily rigged phone-in poll.

    But who exactly is rigging the poll? Do you have some evidence of systematic rigging of the BBC poll?

    I kinda agree with Kevin. Yes, the BBC poll is self-selected, like all phone-in or Internet polls. If that is a big problem, the BBC should not conduct such polls. By definition, the winning issue, whatever it was, would be self-selected. Now that it has conducted the poll, it has to honor the results with good grace, not with the condescension it has shown so far.

    Moreover there is no reason to believe that the vote in favor of self-defense is unrepresentative of British public opinion.

    http://observer.guardian.co.uk/crime/_story/0,13260,942118,00.html

    “Do you think it is acceptable or unacceptable for householders to use potentially deadly force to protect their property against intruders?

    Acceptable 68% Unacceptable 32%

  7. #7 Kevin P.
    January 23, 2004

    A case study in opinion polls versus actual elections (which are completely self-selected)
    is Initiative 676 in the state of Washington in 1997.

    Initiative 676 was a referendum placed on the ballot by gun controllers to
    create a handgun licensing regime. You can read the text here:
    http://www.secstate.wa.gov/elections/initiatives/text/i676.htm

    This article contains a very comprehensive analysis of the politics, issues
    and strategies that surrounded the campaign:
    http://www.secstate.wa.gov/elections/initiatives/text/i676.htm

    For the purpose of opinion polling and its non-representativeness, a couple
    of selected quotes:

    The New York Times declared the referendum a "crucial test for NRA’s
    lobbying prowess," pointing out that, "Polls show that if the vote
    were taken today, the measure would very likely pass by a wide margin."

    Polls taken as late as September showed that 676 was supported by nearly
    two-thirds of the state’s voters. But that was before the public campaign began.

    On November 4th, Initiative 676, the hope of gun control advocates nationwide,
    went down in a fiery blaze: 71 percent of the voters, 899,176 to 371,914, said
    no. In the end, reported the Seattle Post-Intelligencer, it "sank like
    a rock," losing every county in the state.

    According to polls, the turnaround brought about by the NRA’s campaign
    was stunning, especially among women. Reportedly, a month before the election,
    Seattle females supported the proposition 85-15 percent. On election day, they
    voted against it 60-40 percent.

     

  8. #8 Michael Peckham
    January 23, 2004

    Kevin wrote: But who exactly is rigging the poll? Do you have some evidence of systematic rigging of the BBC poll?

    Of course I don’t have evidence, but the very fact that such things can be so easily manipulated calls their credibility into account. Furthermore, I said nothing about “systematic rigging.” Phone-in polls by their nature are subject to all kinds of biases and potential abuses. Trust them at your peril.

  9. #9 Michael Peckham
    January 23, 2004

    Kevin P., that’s an interesting case. Thanks for pointing it out.

    I still don’t agree that a phone-in survey is an accurate measure of public support for an initiative just because it and a referendum are both self-selected. Do they draw from the same population? Does the audience of a radio or television show skew in a different direction from the general populus?

    What your case study demonstrates very well is that public opinion can be changed through political action. It’s a valuable thing to keep in mind.

  10. #10 Kevin Baker
    January 23, 2004

    “What your case study demonstrates very well is that public opinion can be changed through political action.”

    Not exactly. Yes it can, but in a lot of cases polls don’t really examine “public opinion” as they claim to do. A lot has to do with how the poll is worded. A poll that asks whether an individual supports “more effective gun control laws” would garner a very high percentage of YES responses. But a law says “We want to do THIS.” It’s a specific, concrete idea, to which the overwhelming response of the voters is usually “Not THAT!” Did public opinion actually change? No, they still support “more effective gun control,” they just rejected the specific law put forth.

  11. #11 Michael Peckham
    January 23, 2004

    Kevin B., you’re right. My statement was too broadly worded.

    I actually participated in a research project years ago that attempted to gauge how people respond to opinion polls. One of the interesting findings was that a large majority of people (in this particular study) had not previously formulated any solid opinion on a substantial proportion of the questions until they were asked. In other words, they were basically deciding on the spot when presented with a choice, even though they hadn’t previously given the issue much thought. Naturally, this is a big problem when your respondant isn’t interested in a particular issue — you may be recording an “opinion” from somebody who really doesn’t have one.

    The people who take the effort to go to the polls presumably have thought of these things beforehand. At least, I hope so.

  12. #12 Kevin P.
    January 23, 2004

    For some reason, my link to the article describing the politics of I676 is linking to the web site of the Washington Secretary of State.

    Here is the article I intended:
    Gun play: how the NRA defeated Initiative 676 in Washington State.
    http://www.findarticles.com/cf_dls/m2519/n1_v19/20585263/p1/article.jhtml