The 2011 Federal Election: Lessons Learned for Public Opinion Researchers MRIA Seminar, Ottawa Thursday September 22, 2011 Derek Leebosh, VP Public Affairs, CMRP Environics Research Group
Environics and Election 2011 - Agenda Focus on some methodological issues we explored before, during and after the 2011 election campaign: Pre-election experiment with federal vote question wording Weekly tracking during the campaign as telephone and online omnibus Post-election telephone survey for the CBC Recent experiences with online and IVR methodologies 2
Federal vote polling question wording issues Little consistency in how polling companies actually pose the party preference question. Party names vs. party names and leaders Who to prompt? (Green party??) Prompting at all vs. an open-ended question In 2009, Environics conducted a split sample experiment on federal vote preference. With party name prompting the Green Party would get as much as 12 or 13 percent. With no prompting of any party names at all Green support would fall to 6 or 7 percent. No real pattern as to which parties benefit from the reduced unprompted Green vote. 3
Federal vote polling who is a voter? Little consistency in how Canadian polling companies deal with who is a likely voter. In American polling this is a cottage industry. In the past we had found that final polls of all eligible voters tended to be closer to actual election results than when we isolated ltd those claiming li i to be absolutely certain to vote. This may have changed for the 11 election. Certainty of vote choice as a predictor of likelihood of voting is risky in the current Canadian party system/ideological spectrum. Hypothesis that final polls underestimate Conservative support but looking at 2004 and 2006 it may be that they underestimate INCUMBENT support. 4
Environics Tracking polls Election 2011 Environics runs a weekly national telephone omnibus survey of 1,000 Canadians as well as regular online omnibus surveys. The federal vote question was the first question on the phone omnis conducted: March 30-April 5, April 5-10, April 12-17 and April 18-21. The federal vote question was also placed on the online omnibus survey of March 30-April 5 identical dates to the first phone poll The results of the April 5-10 wave were never publicly l released. No final survey in the final week of the campaign The question was asked of all eligible voters. Undecided voters were asked if they were leaning to anyone. We used the totally unprompted approach and no party names were read at all. 5
Environics election 2011 campaign tracking telephone omnibus March 30- April 5,2011 April 5-10, 2011** April 12-17, 2011 April 18-21, 2011 Results Conservative 38 43 39 39 40 NDP 20 20 22 25 31 Liberal 25 23 24 22 19 Bloc Québécois 8 6 9 7 6 Green 8 7 6 6 4 ** Never publicly released 6
Telephone vs. online March 30-April 5 Telephone March 30-April 5 Online March 30-April 5 Online Ontario Online Quebec Cons. 38 39 41 14 NDP 20 22 25 17 Liberal 25 18 23 14 BQ 8 12 n/a 45 Green 8 8 9 9 7
The rogue poll of April 5 to 10 or was it? Telephone April 5-10 Canada Conservative 43 Quebec 20 Ontario 51 NDP 20 27 16 Liberal 23 21 21 Bloc Québécois 6 25 N/A 8
Environics/CBC Post Election Survey The CBC commissioned Environics to conduct a post-election survey on motivations of voters and post-election attitudes. 2,000 interviews i with Canadians 18 years of age and over were conducted by phone with live interviewers May 12 to 22, 2011 9
Environics/CBC Post Election Survey: Recalled Federal Vote TOTAL n = 2,000 Voters only Actual vote May 2 The Conservative Party 23 38 40 The New Democratic Party 18 31 31 The Liberal Party 11 19 19 The Bloc Quebecois 4 7 6 Green Party 3 4 4 Another Party 1 1 2 Did not vote 16 - - dk/na 25 - - Voted in election 60 -- 61 10
Environics/CBC Post Election Survey: Why people voted the way they did Conservative NDP Liberal Liked party government / promises / messages / philosophy 54 34 27 Want change / need for a change 3 28 4 Desire to stop Conservatives - 13 33 Liked local candidate 10 7 20 Disliked other parties/leaders 13 11 5 Liked party leader 13 10 9 Wanted majority government 11 - - Did not like Ignatieff 6 3 1 Long-time party supporter 3 3 9 11
Post-election BC research Online vs. IVR Two proprietary surveys were conducted by Environics in British Columbia in August. An online survey of 1,000 BC residents and an IVR survey of over 6000 6,000 BC residents. Those taking part in each survey were asked how they voted in the May federal election and this was compared to the actual federal vote in BC. 12
Post-election BC research BC - Federal Election Actual Results BC recalled federal vote online BC recalled federal vote IVR (unweighted) Conservative 45.6 46 42 NDP 32.5 33 36 Liberal 13.4 11 14 Green 77 7.7 9 8 13
Post-election BC research Online vs. IVR Online survey using a general population p panel yielded a sample with demographic characteristics that closely matched the characteristics of the BC population. Online 18 to 29 year olds slightly under-represented represented at 15% - but they tend to vote less. No shortage of wired seniors - 28% of online respondents were over 60. Sample had plenty of Chinese-Canadians but was a bit light on South Asians. Recalled federal vote in August was almost identical to the actual federal vote in British Columbia in May. 14
Post-election BC research Online vs. IVR The raw un-weighted data from the IVR survey yproduced more demographic skews. There was over-representation of older people 46% of respondents were 60+. Just 7% were under 30 even with a quota of 500 cell phone only respondents who skew younger. The recalled federal vote showed that NDP voters were over-represented (36%) and CPC voters under-represented (42%) even though the sample skewed old and older people tend to vote CPC. Had we only performed a demographic weighting the recalled federal vote would have been a tie between the CPC and NDP (compared to the actual 13 point CPC lead). Weighting by actual federal vote was applied. 15
Post-election BC research Online vs. IVR Analysis of IVR responses by sub-region of BC showed that the response rate was particularly low in middle and working class areas of the Lower Mainland with large Chinese and South Asian populations. IVR does offer the advantage of being able to get a very large sample size at low cost allowing for a lot of geographic analysis. The size of the sample allows for the application of quite a bit of weighting. Many theories about biases in IVR polling. In the US many feel it is skewed to the GOP. In Canada, recent evidence suggests it has an opposite bias. Conclusion: There is a place for traditional telephone, online and IVR methodologies in Canadian election polling. 16
33 Bloor St East, Suite 900 Toronto, Ontario M4W 3H1 Tel. 416 920 9010 www.environicsresearch.ca ca