Mark Hoofnagle has a MD and PhD in physiology from the University of Virginia, and is now a general surgery resident. His interest in denialism concerns the use of denialist tactics to confuse public understanding of scientific knowledge.
I keep on hearing that the political polls are inaccurate because pollsters do not call wireless phones.
I commission polls at UC Berkeley and we call wireless phones. Seems like a no brainer to me. So, I've never quite understood why professional polling firms wouldn't call cell phones. (I'm an expert in telemarketing laws; survey firms can call cell phones legally so long as it is not a front for marketing.)
Today, I poked around at some prominent pollster's sites to see whether they call cell phone users:
Gallup: Yes, when it is a "national telephone Gallup Poll" survey.
Zogby: No. Here are their reasons.
Princeton: It looks like yes, but it's not clear that they always do. Looks like Pew did a study with Princeton on this issue finding:
The Pew Research Center for the People & the Press has conducted three major election surveys with both cell phone and landline samples since the conclusion of the primaries. In each of the surveys, there were only small, and not statistically significant, differences between presidential horserace estimates based on the combined interviews and estimates based on the landline surveys only. Yet a virtually identical pattern is seen across all three surveys: In each case, including cell phone interviews resulted in slightly more support for Obama and slightly less for McCain, a consistent difference of two-to-three points in the margin.
Peter Hart: No statement on the matter.
Politics
Technology
Comments
Calling people on their cellphones probably will invoke great hostility and misleading answers (I certainly would be hostile.)
I am very saddened to see that the law is softened to actually permit this(sadly much like the obscene exception to the 'do not call' list... political parties can call as much as they want).
Where do pollsters come off feeling they have a right to bother people any more than salesmen? There is no critical need for constant polling just more mental masturbation for the media.
Posted by: jayh | October 25, 2008 4:24 PM
Even those who do call mobile numbers are up against the bias induced by it being, in effect, a collect toll call. I might or might not be willing to cooperate with a call to my land line, but I'm really not amused at the idea of paying $0.30/minute to take part in a poll.
Actually, since most of these now seem to be of the automated voice asking you to hold for someone, I never even find out what they're about.
Posted by: D. C. Sessions | October 25, 2008 5:59 PM
@jayh & D. C. Sessions, remunerating the call recipient is possible, I believe. I'll look into it.
Posted by: Chris H. | October 25, 2008 11:36 PM
While we're on the subject of polls/surveys, and you're associated with some... I'm going to bitch further to poll/survey designers in general.
I (as well as my wife) no longer accept survey calls either at home or cell
1) Get to the point. 15 questions tops. I know you want all these stats, but I've been trapped on survey calls for 20 minutes or more.
2) Don't ask essentially the same question over and over. I know sometimes you are cross checking, but one or two instances are enough.
3) And don't assume that the responder is interested in your subtleties. I got really annoyed at a survey of healthcare issues that repeatedly asked the same set of questions about each nuance of my healthcare exprience. I have no interest in breaking it down into such fine categories.
4) If you use multiple choice answers (typically so) be sure that the choices cover every reasonable position. It's annoying, and certainly skews your results when I can't provide an accurate answer.
5) Allow for some freeform responses. Yes it's more work to categorize these at the end, but you will find information that you did not anticipate when creating the survey.
6) Please select your callers carefully. I fully sympathize with the difficulties of English for foreign born people (my French is awful), but decoding an accent that I'm not used to hearing really turns it into an ordeal.
7) Auto dialers: If I pick up the phone and get 'please hold' I hang up immediately. If you do use one, the message should immediately and plainly state the purpose of the call.
8) You are not entitled to a 'statistically valid sample'. While the survey may be very interesting to you, it's at best a petty annoyance to the people you call. People who are actually busy will never be fully represented.
9) A couple of surveys have actually paid me (cash or small gift) for my cooperation. They got much more of my attention.
Posted by: jayh | October 26, 2008 9:24 AM
Thanks for the informative post on this. I was having a discussion with a co-worker regarding this matter recently and I mentioned to her that I thought the polls are probably a little skewed because of cell phones and the lack of calls to them. Personally, I've never been queried on my cell.
Posted by: Rick Schauer | October 27, 2008 1:36 AM
This concern dates back to the Dewey/Truman campaign, which was one of the first to be predicted by extensive telephone polling. As the story goes, Dewey was favored heavily by telephone respondents, but phones were a relative luxury in 1948, so the respondents skewed to the wealthy side. The po' folks then put Truman over the top.
Early on cell phone owners would also have skewed wealthy, although now I expect they skew towards the young. So leaving them out would probably lead to understating Obama's support.
For this election, I'm more concerned about the effect sometimes called the "Wilder Effect", where minority candidates seem to collect lower vote totals than their polls indicate.
Not to mention the "Diebold Effect" in Ohio... not that I'm cynical or anything.
Posted by: padraig | October 29, 2008 4:09 PM