Is this the year the polls go wrong?

COMMENTARY | October 302, 2010

Sooner or later there’ll be a blunder, writes the noted journalist/polling expert Phil Meyer. If it happens in 2010, the problem could be lack of interviews with cell-phone only households. We’ll know soon enough.

By Philip Meyer
pmeyer@email.unc.edu

Sooner or later, the major national pre-election polls will make a blunder that forces them to reconsider their methods. Will this be the year?

Could be. If so, reporters assigned to write the explainer might find the following background helpful.

Pollsters will often deny trying to predict elections, but, of course, they do. An election is one of the few occasions when polls are easily compared to reality. It separates the good polls from the bad or the lucky from the unlucky.

The best historical examples of both are the Literary Digest presidential election polls of 1932 and 1936. In 1932, the Digest was right on the money in calling the election for Franklin D. Roosevelt. It was lucky, because it did this with a biased mail sample, names drawn from auto registration and telephone lists.

Cars and phones were markers of relative affluence in the Great Depression. This bias didn’t hurt the poll because economic status did not strongly correlate with voter choice.

But what was good luck for the Digest turned to bad luck in 1936 after Roosevelt had succeeded in organizing his New Deal coalition: white southerners, northern minorities and factory workers, intellectuals, big-city machines, and small farmers. The parties realigned that year along economic lines and suddenly affluence did correlate with voting intention.

The magazine, made confident by its 1932 performance, repeated its methodology with an even larger sample and emphatically predicted that Alfred M. Landon of Kansas would be the next president. A former journalism professor named George Gallup thought otherwise and made a different call based on personal interviews and a quota sample that provided the right proportions of key demographic groups. And, just to show off, he drew a small sample from the same sources that the Literary Digest used, and before the magazine could count its huge supply of ballots, correctly predicted that it would name Landon the winner.

The sleeper issue in polling this year is the absence of cell-only households from most polls’ samples. Cell-only users now account for 20 percent of the voting-age population. Getting lists of such households is hard work, and most don’t go to the trouble.

Maybe it doesn’t matter. Maybe being a member of a cell-only household has little to do with how a person will vote. But we do know that such households are disproportionately young and that young voters, when they show up, are more likely to vote Democrat.

Most polls try to correct for this undercount of younger voters by statistical weighting. In other words, the young people they find in landline households get counted more than once to make up for their cell-only peers.

That works, given one condition: that the landline young people vote just like the cell-only young. If they don’t, expect trouble.

One poll, run by Princeton Survey Research Associates on behalf of Newsweek, has consistently shown stronger support for the Democrats than other polls. It has the Democrats ahead in generic Congressional choice and reports higher approval for Obama. And the Newsweek poll includes cell-phones. A recent poll had two-thirds of its respondents coming from landlines and a third from cell phones.

Getting a representative sample was fairly easy when polling was done with face-to-face interviewers in your home. It was even easier when most households had single landlines. Random-digit dialing could ensure that nearly every household had an equal chance of being called.

But technology had disrupted all that. Voice mail, caller ID, call-waiting, number portability, and multiple lines get between the interviewer and the potential voter.

I am old enough to remember when getting a long distance call was a big deal. They were expensive, and when one came, whoever answered the phone would announce, “It’s long distance.” And a hush would fall over the house.

But now they are just part of the background noise of the information age. And that noise is slowly undermining the ability of polls to get the right answers.

When the next polling disaster comes, it will be time to invent a new methodology. Maybe it will use the Internet. Or even snail mail. Or some weird new sampling system.

We’ll soon know if this is the year.

Resources:
A new report by an AAPOR (American Association for Public Opinion Research) cell phone task force: http://aapor.org/Cell_Phone_Task_Force.htm

http://pewresearch.org/pubs/1761/cell-phones-and-election-polls-2010-midterm-elections

http://people-press.org/methodology/sampling/#2

Philip Meyer, a 1966-'67 Nieman Fellow, recently retired as the holder of the Knight chair of Journalism at the University of North Carolina. His most recent book is "Paper Route: Finding My Way to Precision Journalism."

E-mail: pmeyer@email.unc.edu

Comments

Jump to bottom of list

Posted by Rachel
10/302/2010, 07:58 PM
I hope this is the year.

Posted by The Classical Liberal
11/305/2010, 02:15 PM
People have been saying that since 2004 and yet the accuracy of the polls doesn't seem to have diminished.

Two bigger problems seems to be 1. estimating the percentage of people showing up at the polls and 2. predicting the proportion of Democrat and Republican (and all the component sub-breakdowns).

It looks as if most polls are way over-estimating the Democratic vote. There have been many polls whose breakdowns have shown an increase of Dem participation in 2010 then there was in 2006 and 2008. I find that impossible to believe.

Jump to top of list