The 2011 Federal Election: Lessons Learned for Public Opinion Researchers

Transcription

1 The 2011 Federal Election: Lessons Learned for Public Opinion Researchers MRIA Seminar, Ottawa Thursday September 22, 2011 Derek Leebosh, VP Public Affairs, CMRP Environics Research Group

2 Environics and Election Agenda Focus on some methodological issues we explored before, during and after the 2011 election campaign: Pre-election experiment with federal vote question wording Weekly tracking during the campaign as telephone and online omnibus Post-election telephone survey for the CBC Recent experiences with online and IVR methodologies 2

3 Federal vote polling question wording issues Little consistency in how polling companies actually pose the party preference question. Party names vs. party names and leaders Who to prompt? (Green party??) Prompting at all vs. an open-ended question In 2009, Environics conducted a split sample experiment on federal vote preference. With party name prompting the Green Party would get as much as 12 or 13 percent. With no prompting of any party names at all Green support would fall to 6 or 7 percent. No real pattern as to which parties benefit from the reduced unprompted Green vote. 3

4 Federal vote polling who is a voter? Little consistency in how Canadian polling companies deal with who is a likely voter. In American polling this is a cottage industry. In the past we had found that final polls of all eligible voters tended to be closer to actual election results than when we isolated ltd those claiming li i to be absolutely certain to vote. This may have changed for the 11 election. Certainty of vote choice as a predictor of likelihood of voting is risky in the current Canadian party system/ideological spectrum. Hypothesis that final polls underestimate Conservative support but looking at 2004 and 2006 it may be that they underestimate INCUMBENT support. 4

5 Environics Tracking polls Election 2011 Environics runs a weekly national telephone omnibus survey of 1,000 Canadians as well as regular online omnibus surveys. The federal vote question was the first question on the phone omnis conducted: March 30-April 5, April 5-10, April and April The federal vote question was also placed on the online omnibus survey of March 30-April 5 identical dates to the first phone poll The results of the April 5-10 wave were never publicly l released. No final survey in the final week of the campaign The question was asked of all eligible voters. Undecided voters were asked if they were leaning to anyone. We used the totally unprompted approach and no party names were read at all. 5

6 Environics election 2011 campaign tracking telephone omnibus March 30- April 5,2011 April 5-10, 2011** April 12-17, 2011 April 18-21, 2011 Results Conservative NDP Liberal Bloc Québécois Green ** Never publicly released 6

7 Telephone vs. online March 30-April 5 Telephone March 30-April 5 Online March 30-April 5 Online Ontario Online Quebec Cons NDP Liberal BQ 8 12 n/a 45 Green

8 The rogue poll of April 5 to 10 or was it? Telephone April 5-10 Canada Conservative 43 Quebec 20 Ontario 51 NDP Liberal Bloc Québécois 6 25 N/A 8

9 Environics/CBC Post Election Survey The CBC commissioned Environics to conduct a post-election survey on motivations of voters and post-election attitudes. 2,000 interviews i with Canadians 18 years of age and over were conducted by phone with live interviewers May 12 to 22,

10 Environics/CBC Post Election Survey: Recalled Federal Vote TOTAL n = 2,000 Voters only Actual vote May 2 The Conservative Party The New Democratic Party The Liberal Party The Bloc Quebecois Green Party Another Party Did not vote dk/na Voted in election

11 Environics/CBC Post Election Survey: Why people voted the way they did Conservative NDP Liberal Liked party government / promises / messages / philosophy Want change / need for a change Desire to stop Conservatives Liked local candidate Disliked other parties/leaders Liked party leader Wanted majority government Did not like Ignatieff Long-time party supporter

12 Post-election BC research Online vs. IVR Two proprietary surveys were conducted by Environics in British Columbia in August. An online survey of 1,000 BC residents and an IVR survey of over ,000 BC residents. Those taking part in each survey were asked how they voted in the May federal election and this was compared to the actual federal vote in BC. 12

13 Post-election BC research BC - Federal Election Actual Results BC recalled federal vote online BC recalled federal vote IVR (unweighted) Conservative NDP Liberal Green

14 Post-election BC research Online vs. IVR Online survey using a general population p panel yielded a sample with demographic characteristics that closely matched the characteristics of the BC population. Online 18 to 29 year olds slightly under-represented represented at 15% - but they tend to vote less. No shortage of wired seniors - 28% of online respondents were over 60. Sample had plenty of Chinese-Canadians but was a bit light on South Asians. Recalled federal vote in August was almost identical to the actual federal vote in British Columbia in May. 14

15 Post-election BC research Online vs. IVR The raw un-weighted data from the IVR survey yproduced more demographic skews. There was over-representation of older people 46% of respondents were 60+. Just 7% were under 30 even with a quota of 500 cell phone only respondents who skew younger. The recalled federal vote showed that NDP voters were over-represented (36%) and CPC voters under-represented (42%) even though the sample skewed old and older people tend to vote CPC. Had we only performed a demographic weighting the recalled federal vote would have been a tie between the CPC and NDP (compared to the actual 13 point CPC lead). Weighting by actual federal vote was applied. 15

16 Post-election BC research Online vs. IVR Analysis of IVR responses by sub-region of BC showed that the response rate was particularly low in middle and working class areas of the Lower Mainland with large Chinese and South Asian populations. IVR does offer the advantage of being able to get a very large sample size at low cost allowing for a lot of geographic analysis. The size of the sample allows for the application of quite a bit of weighting. Many theories about biases in IVR polling. In the US many feel it is skewed to the GOP. In Canada, recent evidence suggests it has an opposite bias. Conclusion: There is a place for traditional telephone, online and IVR methodologies in Canadian election polling. 16

17 33 Bloor St East, Suite 900 Toronto, Ontario M4W 3H1 Tel ca