Time-Sharing Experiments for the Social Sciences. An Experimental Investigation of the Rally Around the Flag Effect.

Transcription

1 An Experimental Investigation of the Rally Around the Flag Effect Journal: Manuscript ID: TESS-0.R Manuscript Type: Original Article Specialty Area: Political Science

2 Page of 0 0 An Experimental Investigation of the Rally Around the Flag Effect Few political phenomena have received as sustained scholarly attention over the last forty years as the rally around the flag effect. Initial treatments argued that the rally was a reflexive response driven by patriotism to external events (Mueller 0). However, subsequent studies have found that not all foreign crises or even uses of force generate a rally; accordingly, a series of case study and large-n analyses have endeavored to understand the factors that produce rallies of varying sizes in some instances and not in others. Although prior scholarship has proposed many factors that may influence the size of a rally, this study focuses on two in particular: the reaction of other domestic political elites and the nature of the president s response to a crisis. Summarizing a wealth of research (e.g. Brody, Zaller, Baker and Oneal 00, Kam and Ramos 00), Groeling and Baum (00, ) argue that the reaction of other domestic political elites is the most widely accepted explanation for the rally-around-the-flag phenomenon. Existing scholarship has also paid some attention to the varying nature of presidential responses to foreign crises, but with mixed results. For example, James and Rioux () find that rallies increased as the level of the president s response increased; however, the rally dissipated if the president ultimately resorted to military force. Baker and Oneal (00), by contrast, find modest evidence that presidents enjoy a greater rally effect when they escalate a crisis to the use of force or war than when force is merely threatened. This proposal breaks new ground by using an experiment to assess the relative influence of these two factors in shaping public support for the president s handling of a hypothetical crisis scenario. Coupled with a Large-n studies have examined the influence of a myriad of domestic and international factors on the size of the rally including the state of the domestic economy, the proximity of elections, the salience of the dispute in the mass media, whether the Soviet Union was involved, the nature of the principal policy objective of the use of force, the reaction of the UN and other international organizations, and crisis severity among others (inter alia, see: Jentleson, Lian and Oneal, Oneal and Bryan, Oneal and Joyner, Chapman and Reiter 00, Chapman 00, 00, Fang 00, Grieco et al 0).

3 Page of 0 0 similar experiment already conducted on a nationally representative sample of Britons, it will also afford the first ever cross-national experimental investigation into the dynamics driving the rally effect. Using Experiments to Explore the Rally Effect Much prior research on the rally effect has focused on comparative case studies and large-n multivariate analyses of the forces driving the size of rallies in the wake of events in the ICB, MIDs and Blechman and Kaplan use of force data sets. Both methods have important advantages; however, one problem that virtually all analyses relying on observational data alone share is the almost intractable difficulty of controlling for all of the differences between idiosyncratic historical cases to isolate the causal influence of any one variable. For example, Brody () compares the sizeable rally President Ford received after his attempt to rescue the crew of the Mayaguez in with the absence of a rally for President Johnson after the seizure of the Pueblo in. He attributes this to elite discourse elites rallied behind Ford, but split on the wisdom of Johnson s diplomatic approach. While elite reactions were very different in the two ship seizure cases, the presidential responses to the crisis were also dramatically different; given the multiple differences between even these cases chosen for their close similarities, it is all but impossible to determine which factor or factors were most responsible for the divergent public reactions. Experimental methods offer significant advantages in this regard and can complement prior research. However, using experimental approaches in this context entails a problem of its own (in addition to standard concerns about external validity). The rally effect is traditionally measured as the change in presidential approval in the wake of an international crisis or use of Multivariate large-n analyses are better equipped to test the relative influence of various factors on the size of the rally effect; however, even here it is virtually impossible to control for the myriad of idiosyncrasies that distinguish the various cases in the commonly used data sets from one another.

4 Page of 0 0 force. As such, there are difficulties in using experiments to measure the rally effect directly. However, experiments can easily assess the relative importance of various factors in determining the degree to which the public supports the president s handling of a crisis. While this is distinct from the rally effect, the two are closely related. One particularly plausible mechanism that explains the significant variance in rally effects observed in previous research is that foreign crises do not reflexively lead to increases in presidential approval, but rather prime the public to weight foreign policy more heavily when evaluating the president. When the public supports the president s handling of the crisis, his approval rating is poised to increase accordingly. By contrast, when public support for his crisis management is low, no rally in approval ratings should emerge. Experimental Design All respondents are informed of the following hypothetical scenario: A band of African rebels has captured an American merchant ship in the Red Sea. They have taken 0 Americans hostage. Respondents are then randomly assigned to one of four treatment groups. Groups one and two are told that the president has decided to try to negotiate the hostages release and that members of Congress either support or oppose the decision. Groups three and four are told that the president has decided to use military force to try to rescue the hostages and that members of Congress either support or oppose the decision. All respondents are then asked whether they support or oppose the president s handling of the crisis. This will afford a test of the relative importance of the nature of the presidential response and the reaction of congressional elites. The experiment then proceeds to a second stage in which subjects are informed about the outcome of the president s policy. Respondents initially assigned to treatments one and two are It is possible to administer the approval question, then present respondents with a hypothetical scenario, and then re-ask them the approval question to measure the change. However, such an approach is potentially problematic in a number of ways.

5 Page of 0 0 now assigned to one of two new treatment groups informing them that after several weeks or after several days of negotiations the hostages were safely released. Respondents initially assigned to treatments three and four are randomly assigned to one of four new treatment groups. All three groups are told that the military action succeeded in rescuing all 0 hostages; however, they are given varying information concerning the resulting American casualties (0; ; ; + dozens wounded). All respondents are then asked again whether they support or oppose the president s handling of the crisis. Several hypotheses will be tested at this stage. First, comparisons across the latter four treatments afford an interesting test between theories emphasizing the importance of American casualties (e.g. Mueller 0, Gartner and Segura ) and those emphasizing the public s willingness to tolerate casualties as long as the mission succeeds (e.g. Eichenberg 00, Gelpi, Feaver and Reifler 00). Second, the analysis will examine whether initial congressional reactions continue to influence support for the president s handling of the crisis after its successful resolution. Third, the analysis will test whether the president received greater levels of support in the use of force conditions even those that entailed casualties than in the negotiation conditions, which would indicate a strong, baseline public preference for an aggressive military response in such a scenario. Because the proposed experiment comprises only units of length, the maximum sample size of,000 respondents is requested. This will insure sufficient sample sizes in each cell to explore how the effects of each treatment are conditional on respondents partisan affiliations. Preliminary Results and Significance of Proposed Work In August 0, a similar version of the proposed experiment was administered to a nationally-representative sample of,000 Britons (Table ). Both elite reaction and the nature of The additional treatment is to insure that the speed of the ultimate resolution, rather than the chosen tactic of negotiation, is not driving any observed differences between the use of force and negotiation scenarios. This would accord with research in cognitive psychology (e.g. Kahneman and Renshon 00).

6 Page of 0 0 the PM s response significantly influenced public support. However, in the British case the nature of the PM s response was the greatest determinant of public approval. In this specific hypothetical scenario, Britons plainly preferred an aggressive military response; at the initial stage, support for the PM was 0% higher in the use of force case even when it was opposed by many MPs than in the negotiation case when supported by MPs. This greater support continued over to the outcome stage, as regardless of whether MPs supported or opposed the PM s policy, a greater percentage of Britons supported the PM when he resorted to force than when he tried a diplomatic route, even though all hostages were freed in both scenarios and only the military option resulted in the death of servicepersons. By isolating the causal influence of three aspects of a rally event the nature of the executive response, the reaction of other domestic political elites, and the ultimate outcome of the crisis (including the number of casualties sustained) the proposed experiment will complement existing observational studies and afford the most definitive test to date of multiple theoretical claims drawn from the extensive literature on rally effects. The experimental results will also speak to important debates within the broader literature on public support for war (e.g. Berinsky 00, Gelpi, Feaver and Reifler 00). Moreover, coupling this experiment with the UK results will create unique opportunities for cross-national, comparative research on use of force opinion formation. Such designs are exceedingly rare in the existing literature. As a result, the proposed study promises to shed valuable insight both into whether the influence of each experimental factor is the same in two different political contexts, and into whether the variable influence of each factor falls along the same partisan/ideological lines in both countries.

7 Page of 0 0 Table : Support for the PM s Handling of the Ship Seizure Crisis Across Treatments Initial Response: MPs support MPs oppose Use of force % % Negotiate % % Note: All differences in percentages across each row and column are statistically significant, p <.0. Final Response: MPs support MPs oppose Use of force released, KIA % % Negotiate released % % Note: Use of force, MP support/opposition difference is significant, p <.0. Negotiate, MP support/opposition difference is significant, p <.. MP support, use of force/negotiate difference is significant, p <.0. MP opposition, use of force/negotiate difference is significant, p <..

8 Page of 0 0 References Baker, William and John Oneal. 00. Patriotism or Opinion Leadership?: The Nature and Origin of the Rally Round the Flag Effect. Journal of Conflict Resolution : -. Berinsky, Adam. 00. In Time of War: Understanding American Public Opinion from World War II to Iraq. Chicago: University of Chicago Press. Brody, Richard.. Assessing the President: The Media, Elite Opinion, and Public Support. Stanford: Stanford University Press. Chapman, Terrence. 00. International Security Institutions, Domestic Politics, and Institutional Legitimacy. Journal of Conflict Resolution : -. Chapman, Terrence. 00. Audience Beliefs and International Organization Legitimacy. International Organization : -. Chapman, Terrence and Dan Reiter. 00. The United Nations Security Council and the Rally Round the Flag Effect. Journal of Conflict Resolution : -0. Eichenberg, Richard. 00. Victory Has Many Friends: U.S. Public Opinion and the Use of Force, -00. International Security : 0-. Fang, Songying. 00. The Informational Role of International Institutions and Domestic Politics. American Journal of Political Science : -. Gartner, Scott and Gary Segura.. War, Casualties, and Public Opinion. Journal of Conflict Resolution : -0. Gelpi, Christopher, Peter D. Feaver, Jason Reifler. 00. Paying the Human Costs of War : American Public Opinion and Casualties in Military Conflicts. Princeton University Press. Grieco, Joseph, Christopher Gelpi, Jason Reifler and Peter Feaver. 0. Let s Get a Second Opinion: International Institutions and American Public Support for War. International Studies Quarterly : -. Groeling, Timothy and Matthew Baum. 00. Crossing the Water s Edge: Elite Rhetoric, Media Coverage, and the Rally-Round-the-Flag Phenomenon. Journal of Politics 0: -. James, Patrick and Jean-Sebastian Rioux.. International Crises and Linkage Politics: The Experiences of the United States, -. Political Research Quarterly : -. Jentleson, Bruce.. The Pretty-Prudent Public: Post-Post Vietnam American Opinion on the

9 Page of 0 0 use of Military Force. International Studies Quarterly : -. Kahneman, Daniel and Jonathan Renshon. 00. Why Hawks Win. Foreign Policy : -. Kam, Cindy and Jennifer Ramos. 00. Understanding the Surge and Decline in Presidential Approval Following /. Public Opinion Quarterly : -0. Lian, Bradley and John Oneal.. Presidents, the Use of Military Force, and Public Opinion. Journal of Conflict Resolution : -0. Mueller, John. 0. Presidential Popularity from Truman to Johnson. American Political Science Review : -. Oneal, John and Anna Lillian Bryan.. The Rally Round the Flag Effect in U.S. Foreign Policy Crises, -. Political Behavior : -. Oneal, John, Bradley Lian and James Joyner.. Are the American People Pretty Prudent? Public Responses to the Use of Force, -. International Studies Quarterly : -. Zaller, John.. The Nature and Origins of Mass Opinion. New York: Cambridge University Press.

10 Page of 0 0 Experimental Design The experiment begins by providing all respondents the following prompt: Please consider the following scenario that may occur in the future: A band of African rebels has captured an American merchant ship in the Red Sea. They have taken 0 Americans hostage. Respondents are then randomly assigned to one of four treatment groups: INITIAL TREATMENT GROUP : The President has decided to try to negotiate their release. Congressional leaders of both parties SUPPORT the President s decision not to use force. INITIAL TREATMENT GROUP : The President has decided to try to negotiate their release. Congressional leaders of both parties OPPOSE the President s decision not to use force. INITIAL TREATMENT GROUP : The President has decided to use military force to try to rescue the hostages. Congressional leaders of both parties SUPPORT the President s decision to use force. INITIAL TREATMENT GROUP : The President has decided to use military force to try to rescue the hostages. Congressional leaders of both parties OPPOSE the President s decision to use force. All respondents are then asked the following question: Q: Do you support or oppose the President s handling of the crisis? Strongly support Support Neither support nor oppose Oppose Strongly oppose Respondents are then provided an additional prompt concerning the outcome of the president s policy choice. Respondents in initial treatment groups and are randomly assigned to ONE of two outcome treatment groups:

11 Page of 0 0 OUTCOME TREATMENT GROUP : After weeks of negotiations, the rebels released the boat and all 0 American hostages were safely returned. OUTCOME TREATMENT GROUP : After several days of negotiations, the rebels released the boat and all 0 American hostages were safely returned. Respondents in initial treatment groups and are randomly assigned to ONE of the following four outcome treatment groups: OUTCOME TREATMENT GROUP : The military action succeeded in freeing all 0 American hostages. However, American soldiers were killed in the attack, and dozens were wounded. OUTCOME TREATMENT GROUP : The military action succeeded in freeing all 0 American hostages. However, American soldiers were killed in the attack. OUTCOME TREATMENT GROUP : The military action succeeded in freeing all 0 American hostages. However, American soldiers were killed in the attack. OUTCOME TREATMENT GROUP : The military action succeeded in freeing all 0 American hostages. No American soldiers were killed in the attack. All respondents are then asked the following question: Q: Do you support or oppose the President s handling of the crisis? Strongly support Support Neither support nor oppose Oppose Strongly oppose

12 Page of 0 0 This concludes the experiment itself. Time-permitting, we also propose asking all respondents the following manipulation checks: Q: Based on what you read, did congressional leaders support or oppose the president s decision? Support Oppose Don t Know Q: Based on what you read, how many American servicemen killed in an effort to release the hostages? 0 Don t Know

13 Page of 0 0 Table : Results of 0 Pilot Experiment Responses to initial query: Congress support Congress oppose Use of force 0% % Negotiate % % Responses to final query after learning of mission outcome: Use of force released, KIA Weeks of negotiations released Congress support Table : Results of 0 Follow-Up Experiment Responses to initial query: Responses to final query after learning of mission outcome: Use of force released, KIA Days of negotiations released Congress oppose % % % % Congress support Congress oppose Use of force % % Negotiate % % Congress support Congress oppose % % % %