QUESTION WORDING AND PUBLIC SUPPORT FOR CONTRA AID,

Transcription

1 QUESTION WORDING AND PUBLIC SUPPORT FOR CONTRA AID, BRAD LOCKERBIE AND STEPHEN A. BORRELLI Abstract How does question wording affect the results of polls? How can we distinguish question wording effects from genuine change? In this analysis, we make use of a series of polls concerning President Reagan's policy toward Nicaragua. We identify five variations in wording in the survey items that have been used on this topic, and we find evidence that these are responsible for variations in support for the Contras. If the ideological nature of the Sandinistas, President Reagan, the amount of money to be spent, and/or the role of the Contras are mentioned, or if a balanced question format is employed, the poll results can be substantially affected. Once these factors are controlled for, we can argue that there was a small but statistically significant increase in support for Reagan's policy over the period One of the most vexing problems facing poll watchers is the comparison of poll results over time and across polls. An examination of two or more polls on the same general topic can show widely disparate results (Borrelli, Lockerbie, and Niemi, 1987). In the popular media, one can see that different survey organizations employing different questions come up with very different results. 1 Without taking the differences over time and across polls into account, any comparisons that are made rest upon somewhat shaky ground. In this research, we attempt to demonstrate the strong effects of question wording on aggregate public opinion regarding the funding of the Contras. Over the past few years, the debate over U.S. government support for the Nicaraguan Contras has been one of the most tendentious in American politics. People on both sides, using different polls, have claimed that public opinion tends to support their position. How can it BRAD LOCKERBIE is Assistant Professor of Political Science at the University of Georgia. STEPHEN A. BORRELLI is Instructor of Political Science at the University of Alabama. 1. See, for example, U.S. News and World Report, 11 July 1987, Pubbc Opinion Quarterly Volume 54: C 1990 by the American Ajjocialion for Public Opinion Research PubtoJwd by The University of Chicago Press / O X/9(V /$2.50

2 196 Brad Lockerbie and Stephen A. Borrelli be that people appear, at the same time, to favor funding the Contras and also to be opposed to funding the Contras? Well, as we know, subtle variations in question wording can seriously affect the results of a poll (e.g., Mueller, 1973; Bishop, Tuchfarber, and Oldendick, 1978; Bishop, 1987). In short, the potential exists for individuals to construct survey items to favor their own side. While we do not think this is done here, it is important to demonstrate how even subtle variations in question wording can provide (unintentionally) differing results. It is also important to attempt to estimate the size of these question wording effects, and to identify the specific variations in wording that appear to bring them about. From 1983 through 1986, the percentage favoring funding of the Contras has varied considerably, ranging from a low of 13% to a high of 42%. Appendix 1 presents a listing of the polls employed in this analysis, the percentages favoring the funding of the Contras, the percentages opposing funding of the Contras, the percentages not knowing whether the Contras should be funded, as well as the exact wording of the questions used by each poll. 2 Is this variance primarily a response to changing political evaluations or is it primarily a function of survey construction? The questions used to ascertain public opinion on the funding of the Contras have varied considerably across polling organizations and across time, but there are enough similarities that we can identify key characteristics that might influence the results of a poll. It should be emphasized that during the period under study (1983 to mid 1986) the public's knowledge of the Nicaraguan conflict, and the United States role in it, was still in a very formative stage. A Gallup survey administered on 8-11 March 1985, for example, indicated that 48% of those sampled had followed "recent developments in Nicaragua... not at all closely," and that only 38% of the sample could correctly identify which side the United States was supporting. We argue that in this context of low information, the public should be especially attentive and responsive to positive or negative cues within the texts of survey questions that might indicate to them the "correct" position to take on the issue of aid to the Contras. Below, we identify five major cues whose presence or absence in our sample of question items are expected to influence public support for the Contras in a predictable way. First, we suspect that a survey item explicitly associating President 2. The data employed in the analysis are made available by the Roper Center at the University of Connecticut. No polls after mid 1986 are included, for fear that the analysis would be contaminated by the Iran-Contra scandal. In more recent polls, people may be giving their opinions on the scandal instead of responding to Contra funding. All interpretations herein are those of the authors. For the complete and exact wording of the survey items used, along with the coding decisions made by the authors, see Appendix 1.

3 Question Wording and Contra Aid 197 Reagan with Contra aid would evoke a higher level of support for the program than would be expressed in the absence of his imprimatur. Several studies employing a split-ballot technique have reported that respondents react more favorably to survey items mentioning presidential endorsement of a given policy than to similar items that do not mention the president's support of the policy (e.g., Rosen, 1973; Thomas and Sigelman, 1985). For testing the hypothesis that the association of Contra funding with President Reagan increases support for Contra aid, we create a dummy variable (Reagan mention) in which all polls that mention Reagan are coded 1 and the others are coded 0. One concern in American politics is the likelihood of the United States going to war. The Reagan administration has taken great pains to argue that funding of the Contras makes it less likely that U.S. troops will be fighting and dying in Nicaragua (Speakes, 1984; Congressional Quarterly Almanac, 1985). If the respondents are explicitly told that somebody other than Americans will be doing the actual fighting, it is likely that support for the funding will be higher than it would otherwise be. Additionally, being told that there is internal opposition to the Sandinista regime may increase the likelihood that people will support the policy, since it is less likely to appear to be a U.S. effort with no support within Nicaragua. The questions that mention the Contras (Contra mention) are coded 1, and the questions with no mention of the rebels are coded 0. 3 A third variation on question wording that may influence support for funding the Contras is the way in which the Nicaraguan government is described. On one side, the Sandinistas are either not mentioned at all or are referred to as "the government in Nicaragua," "government troops," or "the Sandinista government in Nicaragua." In the remaining polls, the Sandinistas are identified as being "leftists," "Marxist," or "pro-soviet." A good deal of the Reagan administration's campaign on behalf of the Contras has been devoted to emphasizing the Marxist- Leninist character of the Sandinista regime and its close relations with the Soviet Union (Kenworthy, 1987). For testing the hypothesis that identifying the Sandinistas as leftist, Marxist, or pro-soviet increases support for the Contras, we have coded the survey items that label the Sandinista government as any of the above as 1 (Ideolog. label) and all others as 0. All of the cues examined to this point mention of Reagan, mention of the Contras, and ideological labeling of the Sandinistas are expected to exert a positive impact on public support for the Contras. Two additional cues may well reduce levels of support for funding the 3. The polls that mention the Contras, rebels, or forces in Nicaragua righting the Sandinistas are all coded I.

4 198 Brad Lockerbie and Stephen A. Borrelli Contras. First, specifying the amount of money to be spent by the U.S. government should act to depress levels of support for the Contras. Those who favor the U.S. government providing some money, but not as much as the actual amount being discussed, may state that they object to funding the Contras. Questions that mention the amount of money to be spent are coded 1 (Dollar amount), and all others are coded 0. Second, it has been shown that balanced question formats, giving each side of an issue roughly equal attention, can yield vastly different results from questions that merely ask the respondents to react negatively or positively to one side of an issue (Yeric and Todd, 1983). If the question format is balanced, opponents of Contra aid who might otherwise hide their views for fear of holding unpopular or unreasonable opinions would be encouraged to express their true opinion on the issue. Moreover, the balanced format (see Appendix 1 for actual wording) implies there is no necessary connection between opposition to the Sandinistas and support for funding of the Contras. The variable representing this (Balanced format) is coded 1 if the question employed is balanced and 0 if the question is not balanced. 4 Of course, we do not believe that the reported variations in support for the Nicaraguan Contras are entirely the result of methodological artifacts. A cursory examination of the percentage favoring the funding of the Contras shows that there has been a modest increase over time. A time-trend variable (Time trend) that increases by one unit every day has been created; the closing date of the earliest survey in our sample is day one, and all other surveys are coded according to their closing dates. The relationship between this time-trend variable and the proportion favoring the U.S. government funding of the Contras is modest but statistically significant. The regression slope of the percentage favoring Contra aid on the time-trend variable is.009 (significant at.01, two-tailed) and the R 2 is.28. To control for this apparent shift in public opinion, we have included the time-trend variable alongside the methodological variables. We perform multivariate ordinary least squares (OLS) regression analysis predicting a poll's reported percentage expressing support for aid to the Nicaraguan resistance from that poll's scores on the five methodological dummy variables described above and the time-trend 4. For the exact wording of the "balanced" item and of all the items utilized in our sample, see Appendix I. We should also note that we attempted to include an item representing whether or not the word "overthrow" was used in describing U.S. (and Contra) goals in Nicaragua, but we expected that it would have little additional explanatory power given the items already mentioned. As it turns out, the "overthrow" item does not attain the.20 two-tailed significance level when added to the model presented in Table 1.

5 Question Wording and Contra Aid 199 Table I. OLS Regression of Percentage Supporting Contra Aid on Question Wording Variables Variable Coefficient Std. Error T Ratio Beta Reagan mention Contra mention Ideolog. label Dollar amount Balanced format Time trend Constant % ** 4.27*** 1.48* -1.69* -2.22** 3.51*** 5.99*** NOTES: R 2 =.74; adjusted R 2 =.67; N = 28; D.W. = Significance tests are two-tailed. *p <.20. **p <.05. /? <.01. variable. To repeat, the dependent variable is the reported percentage in favor of the Reagan administration's policy with regard to funding the Contras, and the independent variables are the question wording cues: our expectation is that mention of Reagan, mention of the Contras, ideological labeling of the Sandinistas, and the time-trend variable will be positively associated with the percentage expressing support, while the balanced question format and the mention of a specific dollar amount will be negatively associated with support for the administration's position. 5 The OLS regression results are displayed in Table 1. We find that the methodological variables, together with the time-trend variable, account for nearly three-quarters of the variance in support for Contra aid. The coefficient for the time-trend indicates that the overall increase in support for Contra aid persists even after controlling for the question-wording effects. On average, support for Contra aid increased at a rate of.008 percentage points per day (or approximately a percentage point every four months). The performance of the question-wording items, however, leaves no doubt that methodology must be taken into account in any discussion of trends in support for Contra aid. All of the coefficients for the question-wording items are in the expected directions (positive for the 5. Concern that the time-trend variable would lead to greater opposition as well as greater support, due simply to the decreasing number of don't knows, led us to run the same regression model as that presented in Table 1 with percentage opposing Contra aid as the dependent variable. When this analysis is run, we find that the level of opposition does decrease over time.

6 200 Brad Lockerbie and Stephen A. Borrelli first three items, negative for the last two). Three of the five coefficients for the methodology items are statistically significant at the conventional (.05) level, or better; a fourth is extremely close to conventional significance, 6 and there is less than a one in five probability that the fifth does not have the anticipated effect. Judging by the beta weights in Table 1, three of the five question-wording cues mention of Reagan, mention of the Contras, and the balanced question format are at least as important in accounting for support as the overall time trend. 7 The interpretation of the coefficients is straightforward, and they help to reveal how sizable the effects of question wording are. All else being equal, mentioning Reagan's name is associated with an estimated 4.92-percentage-point increase in support for his policy. Mention of the Contras yields, ceteris paribus, an average of 8.98 percentage points in increased support, while use of a balanced format creates a slightly larger effect in the opposite direction. The pollsters' use of ideological labels increases support by an estimated 4.38 percentage points. Conversely, mentioning the amount of money to be spent on the policy tends to cause support for the president's policy to drop by an estimated 3.94 percentage points. 8 In conclusion, the results of this analysis confirm the traditional wisdom but also offer reason for optimism. The way in which a ques- 6. The critical value for significance at the.05 level for 21 degrees of freedom is 1.721; the t ratio for the "dollar amount" variable is Because the number of data points is not large and the number of different combinations of question wording cues is limited, there is justifiable concern about multicollinearity among the independent variables in our model, which might prevent us from making accurate assessments of their impact on Contra support. The equation in Table 1 shows no obvious signs of multicollinearity; for example, there is no discrepancy between the overall fit of the model and the significance of the individual variables, and there are no coefficients with perverse signs or unexpectedly small magnitudes (see Agresti and Agresti, 1979: ). We decided nevertheless to examine the bivariate Pearson's correlations among the independent variables, and these are presented in Appendix 2. The correlation matrix shows one bivariate relationship large enough to present a possible problem: the correlation of.83 between "ideological label" and "balanced format." This relationship results from the fact that the balanced questions in our sample always used an ideological label, although a labeling term appeared in two nonbalanced questions as well (see Appendix 1). Appendix 3 shows the effects of eliminating from our multivariate model each one of these two variables "balanced format" and "ideological label" in tum. Just as one would expect, because "balanced format" and "ideological label" push the dependent variable in opposite directions, removing the controlling effects of the excluded variable weakens the estimated effect of the remaining variable; "balanced format" holds up better, but we suspected from Table 1 that its impact is stronger anyway. Until more data are assembled that can differentiate these influences more cleanly, the reader can choose which is the greater evil: underspecification or multicollinearity. 8. The Durbin-Watson test statistic for this model is in the "region of indeterminacy." There is no evidence, therefore, that statistically significant autocorrelation is affecting our estimates.

7 Question Wording and Contra Aid 201 tion is posed can alter the results of a poll considerably. 9 Fortunately, however, question wording has potentially distinguishable effects. Rather than throwing up our hands in despair when presented with seemingly noncomparable poll results, we should endeavor to determine systematically what the actual effects of these differing methodologies are, for then it is possible to make more valid comparisons across polls. 9. It may be asked whether the poll results vary because of "house effects" above and beyond the different question wordings utilized by the different polling organizations. A test of this hypothesis was attempted by adding dummy variables representing the different polling organizations to the model in Table 1. The model could not withstand this many variables, so the house effects were then assessed one at a time; for example, a dummy variable for CBS/New York Times was included in the Table 1 model, then a dummy for ABCJWashington Post was substituted, etc. None of these dummy variables proved statistically significant when added to the Table I model, suggesting that whatever "house effects" might be present are due primarily to question wording. The dummy variable for "Gallup" (coded 1 for polls 7 and 17 in Appendix I) was a perfect linear function of the other predictors in Table I, so its impact could not be measured.

8 Appendix I. Texts and Codings of Questions on Contra Aid, Poll Text and Coding For Against DK/NS Jun 83 ABCIWP Jun 83 CBS/NYT 28 Jul-1 Aug83 ABC/WP 26 Oct 83 CBSINYT 27 Oct 83 CBSINYT 3-7 Nov 83 ABCIWP Do you think the United States should secretly try to overthrow the govern- 13% ment of Nicaragua, or not? [Reagan mention = 0, Contra mention = 0, Ideological label = 0, Dollar amount = 0, Balanced format = 0] The Reagan administration says the U.S. should help people in Nicaragua 23% who are trying to overthrow the pro-soviet government there. Other people say that even if the country doesn't like the government in Nicaragua, we should not help overthrow it. Do you think we should help the people trying to overthrow the government of Nicaragua, or not? [Reagan mention = 1, Contra mention = I, Ideological label = 1, Dollar amount = 0, Balanced format = 1] As you may know, the United States, through the CIA, is supporting the 20% rebels. Would you say you approve of the United States being involved in trying to overthrow the government of Nicaragua? [Reagan mention = 0, Contra mention = 1, Ideological label = 0, Dollar amount = 0, Balanced format = 0] Same wording as Jun 83 23% Same wording as Jun 83 21% Same wording as 28 Jul-1 Aug 83 30% p://poq.oxfordjournals.org/ at Penn State University (Paterno Lib) on May 9, 2016

9 18-21 Nov 83 Gallup Jan 84 ABC/WP Apr 84 CBS/NYT Oct 84 CBS/NYT Oct 84 CBS/NYT Feb 85 ABC/WP Mar 85 ABC/WP 29 May-2 Jun 85 CBS/NYT 29 May-2 Jun 85 CBS/NYT The U.S. is providing military assistance to the forces trying to overthrow the leftist government in Nicaragua. Do you approve or disapprove of this U.S. policy? [Reagan mention = 0, Contra mention = 1, Ideological label = 1, Dollar amount = 0, Balanced format = 0] Same wording as 28 Jul-1 Aug 83 Same wording as Jun 83 Same wording as Jun 83 Do you think the United States government should provide military assistance to the people trying to overthrow the government of Nicaragua, or not? [Reagan mention = 0, Contra mention = 1, Ideological label = 0, Dollar amount = 0, Balanced format = 0] Should the U.S. be involved in trying to overthrow the government in Nicaragua, or not? [Reagan mention = 0, Contra mention = 0, Ideological label = 0, Dollar amount = 0, Balanced format = 0] Same wording as Feb 85 Same wording as June 83 Should we send military supplies and weapons to the people trying to overthrow the government of Nicaragua? [Reagan mention = 0, Contra mention = 1, Ideological label = 0, Dollar amount = 0, Balanced format = 0] ttp://poq.oxfordjournals.org/ at Penn State University (Paterno Lib) on May 9, % % 27% 29% 30% % % 32% 24%

10 Appendix I. (Continued) Poll Text and Coding For Against DK/NS Jun 85 ABC/WP Aug 85 Gallup/Newsweek 6 Mar 86 ABC 6 Mar 86 ABC 16 Mar 86 ABC Mar 86 ABC/WP 25 Mar 86 ABC Same wording as Feb 85 Should the U.S. be giving assistance to the guerrilla forces now opposing the Marxist government in Nicaragua? [Reagan mention = 0, Contra mention = 1, Ideological label = 1, Dollar amount = 0, Balanced format = 0] President Reagan is asking Congress for new military aid for the Nicaraguan rebels known as the "Contras." Do you agree or disagree with Reagan that Congress should approve the money? [Reagan mention = 1, Contra mention = 1, Ideological label = 0, Dollar amount = 0, Balanced format = 0] As you may know the United States is supporting the rebels. Would you say you approve or disapprove of the United States being involved in trying to overthrow the government in Nicaragua? [Reagan mention = 0, Contra mention = 1, Ideological label = 0, Dollar amount = 0, Balanced format = 0] Same wording as second question for 6 Mar 86 Same wording as Feb 85 Same wording as second question for 6 Mar 86 p://poq.oxfordjournals.org/ at Penn State University (Paterno Lib) on May 9, % 71 29% 58 34% 59 32% 52 30% 54 28% 62 42%

11 o 6-10 Apr 86 Do you think the U.S. should give $100 million in military and other aid 25% CBS/NYT to the Nicaraguan rebels known as the contrast [Reagan mention = 0, Contra mention = 1, Ideological label = 0, Dollar amount = 1, Balanced format = 0] Apr 86 The Reagan administration has proposed giving $100 million in military, med- 33% NBC/WSJ ical, and economic aid to the rebels fighting the Sandinista government in Nicaragua. Do you favor or oppose this proposal? [Reagan mention = 1, Contra mention = 1, Ideological label = 0, Dollar amount = 1, Balanced format = 0] Apr 86 Same wording as 6-10 Apr 86 33% NBC/WS./ Apr 86 Do you favor or oppose U.S. efforts to help the rebels in Nicaragua fight 34% YCS against the government troops in that country? [Reagan mention = 0, Contra mention = I, Ideological label = 0, Dollar amount = 0, Balanced format = 0] Apr 86 Do you generally favor or oppose the U.S. granting $100 million in military 28% 65 7 ABC/WT and other aid to the Nicaraguan rebels known as the contrasl [Reagan mention = 0, Contra mention = 1, Ideological label = 0, Dollar amount = 1, Balanced format = 0] Jun 86 Do you generally favor or oppose the U.S. granting military and other aid 29% 62 8 ABC/WP to the Nicaraguan rebels known as the contrasl [Reagan mention = 0, Contra mention = I, Ideological label = 0, Dollar amount = 0, Balanced format = 0] p://poq.oxfordjournals.org/ at Penn State University (Paterno Lib) on May 9, 2016 NOTES: ABC «= ABC News; ABC/WT = ABC News/Washington Post; CBSINYT = CBS fiew&inew York Times: NBC/WSJ Wall Street Journal; YCS = Yankelovich Clancy Shulman. NBC News/

12 KJ Appendix 2. Bivariate Pearson's Correlation Matrix Percent support Time trend Contra mention Ideological label Dollar amount Balanced format Reagan mention Percent Support 1.0 Time Trend Contra Mention Ideological Label ttp://poq.oxfordjournals.org/ at Penn State University (Paterno Lib) on May 9, 2016 Dollar Amount Balanced Format Reagan Mention

13 Question Wording and Contra Aid 207 Appendix 3. Alternative Models Variable Coefficient Std. Error T Ratio Beta Without "balanced format" Contra mention ***.58 Time trend ***.59 Ideological label Dollar amount * -.20 Reagan mention Constant *** Without "ideological label" Contra mention 9.80 Time trend.008 Dollar amount Reagan mention 4.43 Balanced format Constant R 2 =.68 Adjusted R 2 =.61 N = 28 R 2 = *p <.20, two-tailed. References Adjusted R 2 =.65 **p <.10, two-tailed. 4.70*** 3.28*** -1.82** 1.98** -1.61* 5.%*** N = 28 ***p <.01, two-tailed Agresti, Alan, and Barbara Finlay Agresti (1979) Statistical Methods in the Social Sciences. San Francisco: Dcllen. Bishop, George F. (1987) "Experiments with the middle category in survey research." Public Opinion Quarterly 51: Bishop, George F., Alfred Tuchfarber, and Robert W. Oldendick (1978) "Change in the structure of American political attitudes: The nagging question of question wording." American Journal of Political Science 22: Borrelli, Stephen, Brad Lockerbie, and Richard G. Niemi (1987) "Why the Democratic-Republican partisanship gap varies from poll to poll." Public Opinion Quarterly 51: Congressional Quarterly Almanac (1985) "Congress votes to resume Nicaraguan rebel aid." Washington: Congressional Quarterly Press. Kenworthy, Eldon (1987) "Selling the policy." Pp in Thomas Walker (ed.), Reagan vs. the Sandinistas: The Undeclared War on Nicaragua. Boulder, CO: Westview Press. Mueller, John (1973) War, Presidents, and Public Opinion. New York: Wiley. Rosen, Cory M. (1973) "A test of presidential leadership of public opinion: The split ballot technique." Polity 6:

14 208 Brad Lockerbie and Stephen A. Borrelll Speakes, Larry (1984) "Statement to the press." Pp in Weekly Compilation of Presidential Documents, 16 April. Washington: U.S. Government Printing Office. Thomas, Dan, and Lee Sigelman (1985) "Presidential identification and policy leadership: Experimental evidence on the Reagan case." In George Edwards, Steven Shull, and Norman Thomas (eds.), The Presidency and Public Policy Making. Pittsburgh: University of Pittsburgh Press. Yeric, Jerry L., and John R. Todd (1983) Public Opinion: The Visible Politics. Itasca, IL: F. E. Peacock.