CELL-PHONE-ONLY VOTERS IN THE 2008 EXIT POLL AND IMPLICATIONS FOR FUTURE NONCOVERAGE BIAS

Transcription

1 Public Opinion Quarterly, Vol. 73, No , pp CELL-PHONE-ONLY VOTERS IN THE 2008 EXIT POLL AND IMPLICATIONS FOR FUTURE NONCOVERAGE BIAS MICHAEL MOKRZYCKI SCOTT KEETER COURTNEY KENNEDY Abstract Amid growing concern about potential noncoverage bias in random digit dial survey samples that exclude cell phones, a national face-to-face exit poll provided an opportunity to reach November 2008 Election Day voters regardless of telephone status and to evaluate how the cell-only subgroup has changed since the 2004 election. The National Election Pool s survey found a sharp increase in cell-only incidence, comparable to trends for the general public in government surveys, with cell-only status approaching the norm for voters under age 30. But voters age 30 and older actually abandoned landlines at a faster rate, and the difference in presidential vote preference between the cell-only and landline-accessible voters in this age group was even greater than for younger voters. This suggests that typical poststratification weighting adjustments for age may be less likely to mitigate noncoverage bias in future landline-only RDD surveys. In the past five years the survey research profession has devoted extensive study to the question of whether landline random digit dial (RDD) samples may suffer noncoverage bias by omitting people who only have cellular phones (see e.g., American Association for Public Opinion Research Cell Phone Task Force 2008). 1 The potential problem attracted considerable attention from news reporters and commentators in the 2004 presidential election and this MICHAEL MOKRZYCKI is an independent consultant, 168 Middle Street, West Newbury, MA , USA. SCOTT KEETER is with Pew Research Center, 1615 L Street NW, Suite 700, Washington, DC 20036, USA. COURTNEY KENNEDY is with the University of Michigan, Program in Survey Methodology, 426 Thompson Street, Room 4050, Ann Arbor, MI 48104, USA. Address correspondence to Michael Mokrzycki; mike@mikemokr.com. 1. At least twenty-five papers presented at the 2009 annual AAPOR meeting focused entirely or largely on aspects of cell phones in survey research, according to the conference program. And a special 2007 issue of Public Opinion Quarterly 71(5) was devoted to the subject. doi: /poq/nfp081 C The Author Published by Oxford University Press on behalf of the American Association for Public Opinion Research. All rights reserved. For permissions, please journals.permissions@oxfordjournals.org

2 846 Mokrzycki, Keeter, and Kennedy continued in However, until 2008, empirical evidence largely reassured researchers the impact of cell phones on survey quality was minimal for most topics. While the number of cell-phone-only Americans had steadily increased, it remained a small fraction of the general population. Certain subgroups were much more likely to be cell-only (most notably younger people) but differences on demographic and attitudinal measures were small between them and their cohorts who still had landlines. A 2006 study found that typical poststratification weighting techniques in landline RDD samples produced survey estimates that hardly differed nominally, much less to a statistically significant degree, from blended landline and cell phone samples (Keeter et al. 2007). And analysis of the 2004 Election Day exit poll found that cell-only and landline-reachable voters differed little within age cohorts (Keeter 2006). Yet, substitution of wireless for landline telephone service has increased unrelentingly since those studies were conducted. By the second half of 2008, fully one in five U.S. households was cell-only according to the in-person National Health Interview Survey (NHIS) (Blumberg and Luke 2009). Concern about potential noncoverage bias prompted a number of national pollsters to include cell phone samples in their pre-election surveys during the 2008 campaign. And these more recent dual-frame telephone studies began to suggest these concerns may have been warranted. For example, in September 2008 the Pew Research Center discerned a pattern in its national dual-frame telephone polls on preference in the presidential race: including a sample of cell phone interviews produced a 2- to 3-point shift in the margin toward Democrat Barack Obama and away from Republican John McCain in all three surveys (Keeter, Dimock and Christian 2008a). Other survey firms found directionally similar results (Benford et al. 2009, Langer et al. 2009) although the differences were just shy of statistical significance. A later Pew analysis of combined data from its six pre-election polls from September through the weekend before the election found a statistically significant but small 2.4-percentage point decrease in Obama s lead when cell phone interviews were excluded (Keeter, Dimock and Christian 2008b). Fortunately, we find that final pre-election polling in 2008 was generally accurate in the end, with surveys that excluded cell phones and those that included them performing equally well compared to actual vote for president. Despite the sizeable coverage gap, weighting adjustments appeared to have corrected for the growing bias. But new evidence presented in this study indicates this 2. A search of the keyword combination of cell phones, pollsters and polls in the Nexis database for the four weeks before the 2008 election yielded 160 stories in major newspapers, news magazines or broadcast media; the same search of Factiva for the comparable time period in 2004 found 150 stories (Keeter 2006). In both cases very tight search criteria likely produced conservative estimates, and the search parameters excluded blogs. In 2008 some media references may have been reports of results of surveys that began including a cell phone frame, but many reports questioned whether the omission of cell phones from RDD samples might bias survey estimates.

3 Cell-Phone-Only Voters in the 2008 Exit Poll 847 situation is not likely to last indefinitely. The seeming discrepancy between the small noncoverage bias identified in recent dual-frame studies and their generally good performance in comparison with the single-frame final pre-election polls is a puzzle with many possible solutions, but not a cause for future complacency. And aside from how accurately they predict vote preference, surveys suffer a growing validity challenge on their face when they exclude one-fifth or more of the population of interest. The national exit poll conducted mostly in person outside voting places on November 4, 2008 offered an opportunity to reach a sample of all Election Day voters regardless of their telephone status. As in 2004, the news media consortium that commissions the exit polls included a question on the 2008 national exit poll that allows analysis of attitudinal and demographic differences between voters who only have cell phones and those who have landline service. The sharp increase in the incidence of cell-only Election Day voters from 7 percent in 2004 to 20 percent in 2008 enhances the statistical power of the latest subgroup analysis (with nearly 1,500 cell-only respondents). These exit poll data allow us to examine how the relationship between telephone service and voting behavior has changed since 2004, as well as whether demographic weighting to adjust for noncoverage of the cell-only voters remains effective. We proceed by first assessing the impact of cell-phone-only voters in the preelection polls, comparing the performance of dual-frame telephone polls with single-frame polls sampling only landline numbers. We then present findings from respondent-level analysis of our primary data source, the Election Day national exit poll. We find evidence that the risk of noncoverage error in landline surveys has increased since the 2004 election because the voting behavior of landline and cell-only respondents within age groups is less similar than it was four years ago. Somewhat surprisingly, the difference in presidential vote preference between cell-only and landline-accessible voters was greater among older voters than among younger voters in We close by drawing implications for future telephone surveys. The Predictive Accuracy of Dual- versus Single-frame 2008 Pre-election Polls To begin to understand the consequences of cell phone noncoverage in 2008, we reviewed fifteen national telephone polls with RDD sampling conducted during the final week of the presidential campaign. More than half of these polls were conducted for news media outlets; the rest were conducted by private or academic survey organizations for marketing or analytic purposes. Results for these polls were released to the public and summarized on political polling websites including Pollster.com and RealClearPolitics.com. Six used a dual-frame design and nine sampled landlines only. All the polls projected the correct winner, but given Obama s 7-point victory margin, this is perhaps

4 848 Mokrzycki, Keeter, and Kennedy Table 1. Accuracy of Pre-Election Polls by Sample Design Obama McCain A s.e.(a) Election result Landline telephone RDD FOX News/Opinion Dynamics CNN/Opinion Research Corp American Research Group IBD/TIPP Rasmussen George Wash. U. (Lake/Tarrance) Diageo/Hotline Research Marist College Mean A for landline RDD Mean absolute value (A) for landline RDD Landline and cell telephone RDD Ipsos/McClatchy Pew Research Center NBC News/Wall Street Journal ABC News/Washington Post CBS News/New York Times Gallup Mean A for dual-frame RDD Mean absolute value (A) for dual-frame RDD NOTE. Means are based on A values prior to rounding. SOURCE. Pre-election poll estimates compiled by Pollster.com and RealClearPolitics.com. to be expected. To gauge the relative performance of the polls, we computed the A value for each (Martin, Traugott and Kennedy 2005). The measure A summarizes the degree to which the poll estimate of the relative vote share of the top two candidates deviated from the election result. A values closer to zero reflect greater accuracy than those farther from zero in either direction. Values with a positive sign reflect overestimation of support for Obama (and underestimation of support for McCain) relative to the outcome. Negative values reflect overestimation of support for McCain (and underestimation of support for Obama). The results are presented in table 1. Several of the polls allocated the percentage of undecided voters to the candidates in their final estimates. A is not altered if the undecided voters are dropped, allocated proportionally, or kept as undecided. Of course, there are other differences among the polls that could affect their accuracy, such as how the data are weighted or how they screen for likely voters. We cannot account for these differences. As seen in table 1, landline polls and dual-frame polls differed relatively little in accuracy by this measure. The mean of the absolute values of A was for

5 Cell-Phone-Only Voters in the 2008 Exit Poll 849 the landline RDD polls versus for the dual-frame polls. At the same time, the observed pattern taking account of the sign of A is consistent with the finding in pre-election polls that dual-frame surveys have a higher vote share for Obama. Of the nine landline-only polls, four had negative values for A (signifying an underestimate of Obama s margin). By contrast only one of the dual-frame surveys had a negative value for A. Nonetheless, judged by predictive accuracy the dual-frame surveys did not outperform single-frame surveys. Design of the 2008 National Exit Poll Although dual-frame pre-election surveys provide a way to gain insights into the thinking and demographics of cell-only voters, the national exit poll is a unique source of data because it doesn t rely on telephones to reach respondents and it only interviews those who actually have voted. The exit poll also asks a wider range of questions about political attitudes and behavior than in-person government surveys that provide benchmarks of the size of the cell-only population. In the U.S. general elections of 2004 and 2008, the National Election Pool (NEP) ABC, The Associated Press, CBS, CNN, FOX and NBC commissioned surveys of voters in all fifty states and the District of Columbia from Edison Media Research and Mitofsky International, as well as a national sample. Interviews predominantly were conducted in person on Election Day in a probability sample stratified by geography and past vote. (See Appendix for methodological details.) In recent years, a growing number of voters in many states have cast ballots early or absentee before Election Day. By November 2008 nearly a third of all votes (32.7 percent) were cast before Election Day, up from less than a quarter (22.5 percent) in 2004 (Associated Press 2009). To cover early voters, the inprecinct exit polls which interview voters as they exit the voting booth have been supplemented in numerous states and nationally by telephone surveys conducted the week before the election. Note that all analysis herein is based solely on voters who cast ballots on Election Day in the national sample of 300 precincts, as the 2008 NEP telephone surveys in eighteen states plus a national phone sample only covered households with landline phones. An analysis of possible effects of excluding cell-only voters from the NEP early voter telephone samples is beyond the scope of this paper; that noncoverage has no bearing on the population being studied here, Election Day voters. Election Day respondents complete the exit poll by filling out a paper questionnaire. There were four versions of the national exit poll instrument, two of which included the question shown in figure There were three minor differences in the presentation of the question in 2004: The words in your home were underlined; the parenthetical Check only one instruction was not capitalized; and in the second response code there was a comma after the word regular.

6 850 Mokrzycki, Keeter, and Kennedy Figure 1. Telephone-Status Question as It Appeared on the 2008 National Exit Poll Questionnaire. Source. Election Day national exit poll conducted by National Election Pool, November 4, Although this survey offers a unique opportunity to examine the cell-only phenomenon, it is far from perfect. For one thing, the wording of the telephone status question does not permit identification of cell-mostly voters individuals who have landline phones but mostly rely on their cell phones and thus may be difficult or impossible to reach in landline samples. 4 Also, data are not available on many substantive and demographic variables in the exit poll, such as marital status or religious affiliation, which were not included on the same forms as the telephone status question. And with limited questionnaire space, the exit poll did not include measures of some demographic characteristics that government studies consistently find to be strong predictors of cell-only status, such as living in a household with only unrelated adults or renting rather than owning one s home. Results INCIDENCE OF CELL-PHONE-ONLY VOTERS OVERALL We first look at the basic incidence of cell-phone-only voters in the exit poll sample in 2008 compared to The proportion of Election Day voters who live in cell-only households nearly tripled over four years, to 19.9 percent in 2008 (table 2). This is similar to the finding of the general population National Health Interview Survey, which found 20.2 percent of households had no landline but at least one wireless telephone in the second half of Another 4.1 percent of Election Day voters reported in the exit poll that their household had no telephone service at all, indicating that pre-election polls 4. Cell-mostly or cell-mainly voters are of increasing interest to survey researchers. The evidence presented thus far is that the potential for bias from omitting cell-mostly respondents from samples is more modest than from noncoverage of those who are cell-only. See, for example, Keeter et al. 2009, one of at least six papers presented at the 2009 AAPOR annual conference that addressed this topic. NEP considered modifying the wording of the telephone status question on the exit poll to capture cell-mostly data but opted instead to keep the 2004 wording in 2008 for trend analysis with the largest possible sample size.

7 Cell-Phone-Only Voters in the 2008 Exit Poll 851 Table 2. Household Telephone Status in Election Day National Exit Polls Weighted Unweighted Weighted Unweighted (%) N (%) N Both regular landline , ,962 and cell phone service Only regular landline ,087 phone service Only cell phone service , No telephone service at home Total , ,619 SOURCE. Election Day national exit polls conducted by National Election Pool, November 4, 2008, and November 2, using only landline samples failed to cover about 24 percent of the Election Day electorate. 5 VOTE FOR PRESIDENT BY TELEPHONE STATUS AMONG ELECTION DAY VOTERS We now turn to analyzing how the growing population of cell-only voters differs from those reachable by landline. Table 3 shows how Election Day voters cast their ballots for president in 2004 and 2008 by telephone status. The 2008 results reinforce the findings from four years earlier: cell-only voters remained a source of outsized support for the Democratic candidate, while the Republican continued to fare best among Election Day voters with both regular landline service and cell phones in their household. This is not surprising, given that vote for Republicans tends to grow with higher income, and those with higher income can better afford to have both landline and cell phones. THE GROWING CELL-ONLY VOTER POPULATION While much of the focus on cell-only voters remains on the youngest age groups 6, where the incidence rates are very high, the majority of cell-only 5. These estimates are subject to debate. Recent research on list-assisted RDD designs suggest that actual noncoverage rates for both landline RDD and dual-frame RDD are even higher (Fahimi, Kulp, Brick 2008). Meanwhile, the estimated 4.1 percent incidence of households with no telephone service is higher than the NHIS estimate of 1.9 percent in the second half of Reasons for this difference may include sampling error, the fact that the sample of Election Day voters covers a smaller portion of the U.S. public than does the NHIS sample of all U.S. households, and differences in NEP and NHIS question wording. 6. The 2008 NHIS study finds a smaller proportion of all adults age (33.1 percent) are in wireless-only households compared to voters in the Election Day exit poll (42.2 percent)

8 852 Mokrzycki, Keeter, and Kennedy Table 3. Presidential Vote by Household Telephone Status in Election Day National Exit Polls Obama McCain Kerry Bush (%) (%) (%) (%) ACTUAL POPULAR VOTE (Election Day + early) Estimate among all Election Day voters Both regular landline and cell phone service Only regular landline phone service Only cell phone service No telephone service at home SOURCE. Election Day national exit polls conducted by National Election Pool, November 4, 2008, November 2, 2004; Federal Elections Commission. voters are thirty and older. This was true even in 2004, when 51.7 percent of cell-only voters in the NEP exit poll were at least thirty years old. Of course, part of this is simply the age composition of the voting public: the vast majority of voters are at least thirty years old more than 80 percent in both 2004 and The 2008 NHIS study indicates that 21.6 percent of the age group was cell-only in the last half of 2008 (up from 19.1 percent in the first half of the year), very close to NEP s estimate for this age group among voters (20.2 percent cell-only). Table 4 shows these differences in telephone service from 2004 to 2008 by age groups in the Election Day national exit poll. Note that in this and following tables, the results for Landline voters collapse two categories in the exit poll question: Both regular land-line and cell phone service and Only regular land-line phone service. The percentage of voters ages who were cell-only more than tripled between 2004 and 2008, from 7.3 to 23.2 percent. Interestingly, as a percentage increase over the 2004 baseline this growth rate was even greater than for voters Blumberg and Luke While these datasets lack all the necessary variables to draw firm conclusions, one possible explanation for this finding would be that the youngest cell-phone-only adults are more likely to vote than their cohorts with landlines. This may be related to a difference between young adults away at college and thus more likely to be wireless-only compared to young cohorts who may still be living at home with parents and thus more likely to have landlines in the household; in the exit poll, cell-only 18- to 24-year-olds were more likely than their landlinereachable cohorts to be college graduates or have some college education. Another factor in the difference between the NHIS and NEP Election Day estimates may be differential distribution of cell-only 18- to 24-year-olds between the early voting and Election Day electorates. Question wording also may contribute to the difference.

9 Cell-Phone-Only Voters in the 2008 Exit Poll 853 Table 4. Telephone Service by Age in Election Day National Exit Polls Total 2008 Landline Cell only No phone Landline Cell only No phone Cell-only Pct increase on base 128% 104% 218% 253% 165% 226% 182% SOURCE. Election Day national exit polls conducted by National Election Pool, November 4, 2008 November 2, p <.05. under age 30. This can be explained in part by aging over four years of the cohort that in 2004 was nominally most likely to be cell-only those who then were twenty to twenty-nine years old. In 2008 the cell-only growth rate in the survey among the age group was even greater than for those aged 30 39, even though the absolute change was smaller. Statistically significant increases also occurred among those and 65 and older. With wireless substitution growing rapidly across all age groups, differences in the growth rates by age can have consequences for poststratification adjustments by age in landline-only RDD surveys. TELEPHONE STATUS BY OTHER DEMOGRAPHIC AND ATTITUDINAL VARIABLES Cell-phone-only status also is related to other key demographic correlates of electoral behavior. These relationships are important because of the implications for demographic weighting used to adjust for noncoverage. Keeter (2006) reported how the incidence of cell-only status varied by race, education, income and other factors in the 2004 national exit poll. We compared those results to the 2008 figures to determine how the rate of transitioning to cell-only status changed along these dimensions. The results are presented in figure 2. The most sizable increases in cell-phone-only status occurred for the lowest income and least educated voters, and for Latinos. In 2004 some 13 percent of Election Day voters with an annual income under $15,000 were cell-only; by 2008 that

10 854 Mokrzycki, Keeter, and Kennedy Figure 2. Proportion of Election Day Voters Who Are Cell-Only, by Demographics. Source. Election Day national exit poll conducted by National Election Pool, November 4, 2008 and November 2, The 2008 standard errors range from 2.0 for the estimate for whites to 8.2 for the estimate for non-high school graduates. figure nearly tripled to 37 percent. Similarly, 9 percent of Latino or Hispanic voters in 2004 were cell-only, but this rose to 28 percent in Among most Election Day voters, the likelihood of being cell-only declined as income rose, but in both 2004 and 2008 the trend was not monotonic. Curiously, the trend reversed slightly for those with the highest incomes. In 2008 this curvilinear effect of income is highly significant in logistic regression models predicting cell-only status, with and without controls for other demographics. The mechanism behind this pattern is unclear because the ability to afford landline service is clearly not a factor for those at the highest income levels. One potential consequence of this phenomenon may be to diminish the reliability of landline-only surveys that target high-income population subgroups.

11 Cell-Phone-Only Voters in the 2008 Exit Poll 855 Table 5. Presidential Vote by Telephone Service in Election Day National Exit Polls 2008 Landline Cell only Diff Landline Cell only Difference Obama Kerry McCain Bush Other Other Obama Kerry McCain Bush Other Other Obama Kerry McCain Bush Other Other Obama McCain Other Obama McCain Other SOURCE. Election Day national exit polls conducted by National Election Pool, November 4, 2008 and November 2, p <.05 in difference of proportions test. PRESIDENTIAL VOTING BY AGE AND PHONE STATUS Perhaps the most striking finding of the 2008 exit poll question on telephone status was how differences in voting patterns between cell-phone-only and landline-reachable respondents played out among age groups. Differences in presidential vote between the landline and cell-only groups in 2008 were largest among voters ages (table 5). Obama was the choice of 62.6 percent of cell-only voters ages but only 51.3 percent among those reachable by landline, a difference of 11.3 percentage points that was significant at the.05 level. The difference was smaller and not significant among all other age groups in the survey: 7.9 points among those 40 49, 7.4 points among those 18 24, and less among those and 50+. In 2004 older cell-only voters were such a small part of the voting population that their voting patterns, however different,

12 856 Mokrzycki, Keeter, and Kennedy posed little risk of biasing election surveys based on landline samples. This had clearly changed by Among all Election Day voters thirty and older, Obama was the choice of 55.5 percent of those who were cell-only and 47.8 percent of the landlineaccessible, a significant difference of 7.7 percentage points. Among Election Day voters 18 29, the difference was 4.9 points, though not statistically significant. Four years earlier, the difference in vote for Democratic presidential nominee John Kerry among the 30+ age group was 2.8 points; among those it was 2.2 points. Although the sample sizes in the poll cannot support a definitive conclusion, the disjuncture between landline and cell-only voters appears to have grown more among older voters than among younger ones. Because age is typically a poststratification weighting variable, this result suggests adjustments by age may be insufficient to overcome noncoverage in landline-only RDD surveys. Generally, voter attitudes other than presidential vote choice differed somewhat less by telephone status. Table 6 shows cell phone status by party identification and ideology, within age groups. The differences across telephone status within age groups indicate how well cell-phone-only voters are represented by their landline-accessible peers. Several differences are apparent. Among older Election Day voters (age and 40+), those living in landline households are significantly more likely to consider themselves conservative than their cell-only peers. Among younger voters, by contrast, telephone status appears to be more closely related to party identification, though these differences were not statistically significant. Although not shown, we analyzed a number of other measures asked on the exit poll questionnaire. Significant differences by telephone status appeared in variables related to partisanship and ideology; for example, cell-only respondents were 10 points more likely than landline-reachable voters to strongly disapprove of George W. Bush s performance as president. On the other hand, there were no significant differences by telephone status on whether the candidates race or age were factors in voting for president. And measures of the economy produced mixed results: there was no significant difference in likelihood of calling the economy the nation s most important issue nor in worry about the economy s direction over the next year, but cell-only Election Day voters were a little more likely than the landline-reachable to say the economy currently was poor yet slightly less likely to call it not so good. These results suggest it may not be easy to predict how cell phone noncoverage bias may affect estimates on individual attitudinal items within a survey. There also were interesting differences by phone status in voter mobilization, as shown in table 7. Cell-only Election Day voters were far more likely than those with landlines to have voted for the first time in 2008, though naturally this correlates strongly with age (the youngest voters are most likely to be cell-only and also most likely to be new voters). But this same group also was far less likely to have been contacted by either campaign. Even Obama s

13 Cell-Phone-Only Voters in the 2008 Exit Poll 857 Table 6. Ideology and Party Identification by Telephone Status and Age in Election Day National Exit Poll Landline phone Cell phone only Difference Age Party identification Democrat Republican Independent Other Ideology Liberal Moderate Conservative Age Party identification Democrat Republican Independent Other Ideology Liberal Moderate Conservative Age 40+ Party identification Democrat Republican Independent Other Ideology Liberal Moderate Conservative SOURCE. Election Day exit poll conducted by National Election Pool, November 4, p <.05 in difference of proportions test. campaign renowned for its technological innovation, including its plan to use text messaging to notify 3 million supporters of Obama s vice-presidential pick was more likely to contact landline-reachable than cell-only voters, according to the Election Day exit poll. (So it s not only pollsters who have trouble reaching those who are wireless-only.) These differences remain statistically significant even within age groups. Given extensive research showing that mobilization efforts are effective at stimulating voter turnout, these findings have important normative implications.

14 858 Mokrzycki, Keeter, and Kennedy Table 7. Voter Mobilization by Telephone Status in Election Day National Exit Poll Landline phone Cell phone only Difference (%) (%) (%) Is this the first year you have ever voted? Yes Did anyone call you or talk to you in person on behalf of either major presidential campaign about coming out to vote? Contacted by both Contacted by Obama only Contacted by McCain only Contacted by neither SOURCE. Election Day exit poll conducted by National Election Pool, November 4, p <.05 in difference of proportions test. Young people, not surprisingly, are much less likely than older people to have developed habits of political engagement and are in greater need of mobilization to vote in elections. Yet, because of their telephone status, they are far less likely to be reached by campaigns, even if the campaigns intend to reach them. A similar logic may apply to lower income and minority individuals. This pattern almost certainly will exacerbate political inequality. POSSIBLE REASONS FOR POLITICAL DIFFERENCES BETWEEN CELL-ONLY AND LANDLINE-ACCESSIBLE VOTERS The observed political differences between the cell-only and landlineaccessible voters likely stem from the outlined life-cycle and socio-economic differences between these two groups, which correlate with political views. Pre-election polls typically collect some indicators of these mechanisms. Age, marital status, and presence of children in the household are commonly used indicators of life-cycle stage. Income, education, and homeownership are commonly used indicators of socio-economic status. Many but not all of these variables were measured alongside telephone service in the 2008 national exit poll. We used logistic regression to better understand how these factors relate in a multivariate setting. The outcome we modeled is vote for Obama among those voting for either of the major party candidates. The model was estimated separately for those aged 18 29, 30 39, and 40 and older, because we expect

15 Cell-Phone-Only Voters in the 2008 Exit Poll 859 Table 8. Logistic Regression Models Predicting Vote for Obama, by Age Groupˆ Age Age Age 40+ Parameter B s.e. B s.e. B s.e. Telephone service No telephone Cell-only Male African-American Hispanic Education Income Incomeˆ Intercept Model sample size (1,514) (1,428) (3,727) % Predicted correctly 73.5% 64.0% 58.3% NOTE. The results do not change substantively in any of the three models when weighting is used. SOURCE. Election Day exit poll conducted by National Election Pool, November 4, p <.05. ˆReference groups are has landline, female, not African-American, and not Hispanic. Models are based on Election Day voters voting for either Barack Obama or John McCain. Model estimates are unweighted so as not to distort the effect of demographic predictors. the predictors to interrelate differently depending on the voter s age. We also want to maintain our focus on how noncoverage error might arise in landline telephone polls if cell-only voters differ from landline-accessible voters within their age group, since age is a common variable used in weighting. Presidential vote was regressed on telephone status (cell-only, no phone, with landline as the reference group), gender, race, Hispanic ethnicity, education, income, and a curvilinear effect for income. In a preliminary model, we tested for interaction effects between telephone service and the demographic variables, but none was significant. The final model estimates are reported in table 8. Cell-only status is a highly significant predictor of voting for Obama among those years old, even when controlling for common weighting variables. It also remains significant for those 18 29, though to a slightly lesser degree. Telephone service is not, however, predictive of an Obama vote among those aged 40 and older. The regression analysis predicting the vote finds that including the available demographic variables age, sex, education, income and race reduces the impact of telephone status somewhat (compared with a model in which the demographic factors are not included) but leaves much of the variability unexplained. The gap in predicted probability of a vote for Obama between

16 860 Mokrzycki, Keeter, and Kennedy cell-only and landline-accessible voters, without controlling for demographic characteristics, is 10.7 percentage points. Including the demographics in the model reduces this difference to 6.7 points. These results suggest that noncoverage bias can persist in landline survey estimates even when standard demographic weighting is employed. Unfortunately, other important variables related to phone status and the vote were not asked on the same forms of the questionnaire that included phone status, including marital status and presence of children in the home. Religion, which is associated with social integration and is correlated with phone status, also was not asked on the same forms. Homeownership with renting known to be a strong predictor of cell-only status was not asked in the exit poll at all. A similar multivariate analysis using a September 2008 Pew Research Center poll found that the independent impact of cell-only status on vote choice diminished to insignificance when marital status and homeownership were included along with standard weighting demographics, suggesting that more elaborate weighting protocols might help to reduce noncoverage bias (Keeter et al. 2009). Conclusions Considering the trend in telephone usage and the evidence for the potential growth of coverage error, researchers should ensure that the cell-only population is covered in future pre-election surveys, including the National Election Pool s own surveys of early and absentee voters. It may seem odd to make such a recommendation in the absence of strong evidence of bias in vote preference estimates in pre-election polls to date. But the risk of error posed by noncoverage is no longer trivial. And there are other reasons beyond forecast accuracy to improve the coverage of polls. The percentage of Election Day voters who were cell-only nearly tripled between 2004 and 2008, mirroring trends in government surveys of the general population. There is no evidence that this rapid rate of growth is likely to diminish in the near future. Because of the sizeable differences between landline-accessible and cell-only voters, the potential for noncoverage bias in future landline-only surveys may be substantial. The good news is that it appears that noncoverage bias did not present a large problem for national pre-election polls in Surveys that included cell phones were no more accurate in forecasting the presidential race than were those that relied only on landline samples. But warning signs appeared. Several dual-frame pre-election polls in 2008 found that estimates based on combined landline and cell phone samples tended to increase the estimates of preference for Barack Obama by 2 3 percentage points, compared with the landline sample alone. In a race tighter than the 2008 presidential election, even a bias this small could cause some surveys to estimate at least nominally, if

17 Cell-Phone-Only Voters in the 2008 Exit Poll 861 not necessarily with high statistical confidence that the wrong candidate is ahead. Additionally, to the surprise of many analysts, the potential noncoverage bias is not concentrated solely among the younger voters. Among Election Day voters in the national exit poll, the majority of the cell-only population was at least 30 years old, and larger differences in voter preference by telephone status were observed for those than for those a pattern also seen in Pew Research Center pre-election surveys during the fall campaign. The vote preference differences by phone status in the exit poll remained significant in a multivariate analysis that controlled for most of the factors normally used in weighting pre-election polls. Although this bias was modest in 2008, the potential for bias may continue to grow as cell-only status becomes more common among older age groups. Another issue is the declining ability of landline-based polls to describe young voters with an acceptable level of reliability. As the cell-only population has grown, landline samples are obtaining fewer younger adults, leaving many polls with insufficient samples for such analyses. For example, only 9 percent of respondents in Pew Research s landline samples were ages in 2008, compared with a population parameter of 22 percent (Keeter 2009). Young voters were the subject of intense interest and study in the 2008 cycle, given their comparatively high levels of engagement in the campaign and the fact that a higher percentage of them supported Obama than in any other age group. Pre-election polls needed to be able to describe these young voters and explain their opinions and motivation, but many were limited in their ability to do this because of the coverage issue. The National Election Pool s landlineonly pre-election polls were similarly constrained from describing how the rapidly growing number of young and older people who voted early or absentee compared with those interviewed in the Election Day national exit poll. Including cell phone samples is an effective remedy for these problems, although admittedly more costly than landline-only samples. Although not addressed in the current analysis, future survey research must also contend with the cell-mostly population. There is, as yet, no evidence of serious coverage bias related to voters who have a landline but rely mostly on their cell phones. But a better understanding of this population is needed, and the size and characteristics of this group should be monitored closely. If necessary, steps should be taken to ensure such individuals are covered and measured appropriately in future pre-election polling and surveys of actual voters. The inclusion of cell phone samples can address this problem as well. Correctly forecasting the margin in an election is important for polling, but it is not the only or even the most important contribution polls make to the electoral process. Polls help citizens, the news media and policy makers understand how voters are making their judgments and what considerations animate them. Polls are critical for informing an understanding of why elections turn out as they do, and thus have implications for how candidates, once elected,

18 862 Mokrzycki, Keeter, and Kennedy may govern. For these reasons it is critical that polling fully represent the voting public. Ultimately, polls that omit one-fifth or more of voters voters who are demonstrably different from those included in standard polling are not likely to have the credibility they need, even if they correctly predict the final election tally. Appendix: Methodology and Question Wording The sample for the Election Day national exit poll used in this analysis was drawn from the population of all people who voted in person on November 4, 2008, in the U.S. general elections. A probability sample of precincts was drawn in each of the fifty states and District of Columbia from among all voting locations in the state; a subsample of the state precincts was drawn to create the national sample. Within each precinct, voters were selected at a systematic interval throughout Election Day so all voters in that precinct had an equal chance of being interviewed. Each state sample was stratified by geography and partisan voting history. Precincts were selected with a probability proportionate to the number of voters in each precinct, except that in some states, precincts with large minority populations were sampled at a higher rate, and in New Hampshire several very large precincts were sampled with certainty. In those cases, sample weighting adjusted representation of precincts to their correct share of the total vote. Other post hoc weighting of the national exit poll included: weighting to take into account the probability of selecting a precinct; adjustments based on the number of respondents and interviewers tallies of the gender, estimated age (into three bands) and race (black or nonblack) of voters selected in the random sampling interval but who were missed or refused to cooperate; forcing of survey estimates of each candidate s vote share to actual vote received; weighting to targets for age, race and gender; and adjustments based on size of vote by region. (For data about the entire electorate, the national Election Day exit poll was combined with National Election Pool telephone polling conducted the week before the election to sample early and absentee voters; this analysis focused only on those who voted in person on Election Day.) See National Election Pool (2005) for additional methodological detail. The sampling and interviewing procedures did not substantively change in Documentation for the 2008 study will be available at the Interuniversity Consortium for Political and Social Research (ICPSR) and Roper Center for Public Opinion Research. The completion rate for the 2008 Election Day national exit poll was 43.1 percent. This rate is the number of voters who completed the survey out of the total number selected to take the survey, as tallied by the interviewers. AAPOR Standard Definitions do not define how to calculate response rates for in-person exit polling of individuals, only for in-person surveys of households.

19 Cell-Phone-Only Voters in the 2008 Exit Poll 863 Following is the exact wording of the main questions used in this analysis. The complete 2008 national exit poll questionnaires are available in the supplementary data online at What type of telephone service is there in your home that you could use or be reached on? (CHECK ONLY ONE) 1. Both regular land-line and cell phone service 2. Only regular land-line phone service 3. Only cell phone service 4. No telephone service at home In today s election for president, did you just vote for: 1. Barack Obama (Dem) 2. John McCain (Rep) 9. Did not vote 0. Other: Who? To which age group do you belong? or over Are you: 1. White 4. Asian 2. Black 5. American Indian 3. Hispanic/Latino 6. Other Are you of Hispanic or Latino descent? 1. Yes 2. No (Note: Those who answer Yes to the Hispanic/Latino ethnicity question are counted as Hispanic/Latino in the race demographic even if they selected something else there.) Are you: 1. Male 2. Female 2007 total family income: 1. Under $15, $75,000 $99, $15,000 $29, $100,000 $149, $30,000 $49, $150,000 $199, $50,000 $74, $200,000 or more

20 864 Mokrzycki, Keeter, and Kennedy What was the last grade of school you completed? 1. Did not complete high school 2. High school graduate 3. Some college or associate degree 4. College graduate 5. Postgraduate study No matter how you voted today, do you usually think of yourself as a: 1. Democrat 2. Republican 3. Independent 4. Something else On most political matters, do you consider yourself: 1. Liberal 2. Moderate 3. Conservative Is this the first year you have ever voted? 1. Yes 2. No Did anyone call you or talk to you in person on behalf of either major presidential campaign about coming out to vote? 1. Yes, for Barack Obama 2. Yes, for John McCain 3. Yes, for both Obama and McCain 4. No, I was not contacted Acknowledgements The 2004 exit poll data are available from a dataset that is archived at the Roper Center for Public Opinion Research, University of Connecticut. The 2008 data are from an NEP dataset that at this date remains unpublished; we obtained the dataset as all three authors worked or consulted for NEP members during the 2008 elections. Supplementary Data Supplementary data are available online at

21 Cell-Phone-Only Voters in the 2008 Exit Poll 865 References American Association for Public Opinion Research Guidelines and Considerations for Survey Researchers When Planning and Conducting RDD and Other Telephone Surveys in the U.S. with Respondents Reached via Cell Phone Numbers. Available at uploads/final_aapor_cell_phone_tf_report_ pdf. Accessed October 7, Associated Press Election Research Turnout and Advance Voting. Unpublished report dated April 21, Cited by permission. Benford, Robert, Trevor Tompson, Barry Feinberg, Geoff Feinberg, Annie Weber, Nicole Speulda and Christopher Fleury Wireless and Wireline: Dual-frame Implications for Sample Design Decisions on Estimates, Weighting and Costs. Paper presented at the 64 th Annual Conference of the American Association for Public Opinion Research, Hollywood, FL, USA, May Blumberg, Stephen J., and Julian V. Luke Wireless Substitution: Early Release of Estimates from the National Health Interview Survey, July December National Center for Health Statistics. Available at Accessed October 7, Fahimi, Mansour, Dale Kulp and J. Michael Brick Bias in List-Assisted 100-Series RDD Sampling. Survey Practice September 28, Keeter, Scott The Impact of Cell Phone Noncoverage on Polling in the 2004 Presidential Election. Public Opinion Quarterly 70: New Tricks for Old and New Dogs: Challenges and Opportunities Facing Communications Research. March 3, Available at survey-problems-new-techniques-communications-research. Accessed October 7, Keeter, Scott, Michael Dimock, and Leah Christian. 2008a. Cell Phones and the 2008 Vote: An Update. Pew Research Center for the People & the Press. Sept. 23, Available at Accessed October 7, b. Calling Cell Phones In 08 Pre-Election Polls. Pew Research Center for the People & the Press. Dec. 18, Available at Accessed October 7, Keeter, Scott, Courtney Kennedy, April Clark, Trevor Tompson, and Mike Mokrzycki What s Missing from National Landline RDD Surveys? The Impact of the Growing Cell-Only Population, Public Opinion Quarterly 71: Keeter, Scott, Jocelyn Kiley, Leah Christian, and Michael Dimock Perils of Polling in Election 08. Paper presented at the 64 th Annual Conference of the American Association for Public Opinion Research, Hollywood, FL, USA, May Available at pubs/1266/polling-challenges-election-08-success-in-dealing-with. Accessed October 7, Langer, Gary, Peyton Craighill, Patrick Moynihan, Jon Cohen, Jennifer Agiesta, and Dave Lambert These Nutty Pollsters : Methodological Issues in ABC News/Washington Post 2008 Pre- Election Polling. Paper presented at the 64 th Annual Conference of the American Association for Public Opinion Research, Hollywood, FL, USA, May Martin, Elizabeth A., Michael W. Traugott, and Courtney Kennedy A Review and Proposal for a New Measure of Poll Accuracy. Public Opinion Quarterly 69: National Election Pool, Edison Media Research, and Mitosky International National Election Pool Poll # 2004-NATELEC:National Election Day Exit Poll. Roper Center version. Somerville, NJ: Edison Media Research/New York, NY: Mitofsky International [producers], Storrs, CT: Roper Center for Public Opinion Research, University of Connecticut [distributor]. Available at