A COMPARISON BETWEEN TWO DATASETS

A COMPARISON BETWEEN TWO DATASETS Bachelor Thesis by S.F. Simmelink s1143611 sophiesimmelink@live.nl Internationale Betrekkingen en Organisaties Universiteit Leiden 9 June 2016 Prof. dr. G.A. Irwin Word count: 7.975

1. INTRODUCTION Even though it makes intuitively more sense to surrender when the enemy consists of multiple countries instead of one single country, well-known empirical evidence states that this does not have to be true in the case of economic sanctions. According to Hufbauer, Schott, Elliott and Oegg (hereafter HSE), sanctions that are imposed by a single country are more effective than sanctions imposed by a coalition of countries. This conclusion seems quite counterintuitive because multiple countries are for instance able to impose higher costs on the target country than one single country. Bapat and Morgan (2009) tried to explain why multilateral sanctions are less effective by reproducing the research by HSE with a different and much larger dataset called Threat and Imposition of Economic Sanctions (hereafter TIES). While Bapat and Morgan (2009) were looking for the same results as HSE considering the effectiveness of multilateral sanctions, they came to the more logical but different conclusion that multilateral sanctions are more effective than unilateral sanctions. In response to these contradictory findings by HSE and Bapat and Morgan, the following research question will be examined in this thesis: What explains the difference in findings about the effectiveness of multilateral sanctions when comparing the datasets of HSE and TIES? By answering this question, I will attempt to clear up the confusion about the conflicting findings of HSE and TIES. In order to answer this question, I have developed two hypotheses that will be discussed in the theoretical framework. These hypotheses function as possible explanations and will be answered by comparing the empirical evidence of HSE (2007) and Bapat and Morgan (2009). The reason for states to cooperate in multilateral sanctions are diverse. One of the reasons has for instance to do with the relative gain when compared to unilateral sanctions. According to Lektzian and Souva (2007, 850), states are willing to cooperate and impose multilateral sanctions because targets of unilateral sanctions are more likely to find alternative suppliers which would minimize the amount of economic damage of the sanctions. Multilateral sanctions could limit the number of alternative suppliers and thereby impose greater costs on the target which will eventually contribute to the possible success of economic sanctions. There are also other certain factors that tend to increase the level of cooperation such as international institutions, high costs to the major sender and assistance to the target (Martin 1

1992, 90). To distribute for instance the high costs to the sender, states will seek for support and promises of complementary actions from other potential sanctioners to make sure the economic sanctions have nonetheless an impact on the target state (Martin 1993, 408). For the same reason states would like to minimize the assistance to the target. Also, multilateral sanctions will occur when there are several potential sanctioners with different interests which could create a problem, but this problem can be solved by bounding the potential sanctioners by issue-linkages. According to Martin (1993, 431), states will cooperate because of tactical issue-linkages, which makes the credibility of commitments the main explanation for cooperation. The commitments (or threats to commit) of the main sender state have to be credible to the additional senders otherwise these additional senders will have the incentive to free-ride. Finding an answer to the research question is scientifically relevant, because it is still unclear what explains the discrepancy in findings about the effectiveness of multilateral sanctions. The ground-breaking conclusion of HSE about the effectiveness of unilateral sanctions led scholars to develop several theories to explain the ineffectiveness of multilateral sanctions. These theories focused for instance on a selection effect, public goods or spatial theory. However, according to the research by Bapat and Morgan (2009), multilateral sanctions are more effective. Hence, it seems that there is a contradiction in the scientific literature about economic sanctions which has not yet been solved. This thesis is overall relevant because policy makers base their recommendations partially on scientific findings. Right now, there are contradictory scientific findings which could imply that policies have been recommended based on findings which were flawed. 2

2. LITERATURE REVIEW The literature on economic sanctions discusses mostly the effectiveness of economic sanctions and the determinants for success. The effectiveness of economic sanctions is often seen in the context of other foreign policy tools. Economic sanctions are seen by proponents as an effective and more humane foreign policy tool than the use of military force (Pape 1997, 90). According to HSE (2007, 5), economic sanctions are part of international diplomacy and these sanctions can be useful as a tool to coerce a target country in a particular direction. Whether the target country will concede, or in other words, whether the economic sanctions will succeed, is dependent on several factors. For instance, the health and stability of the target country, the warmth of prior relations between the sender and target, international assistance to the target or cooperation with the sender and the duration of the sanctions can influence economic sanctions (Hufbauer & Schott 1985, 730). The most important factor for this thesis is the influence of the level of cooperation on the effectiveness of economic sanctions. 2.1 WHY UNILATERAL SANCTIONS ARE MORE EFFECTIVE There are three general theories in the literature that explain why multilateral sanctions are less effective than unilateral sanctions. These theories are based on expectations and speculation, but not on empirical research. The first theory is about the selection effect, the second theory discusses the possibility to free ride and is mainly about public goods and the third theory examines the spatial theory about coalitions looking for a common demand in the policy space. Selection effect Overall, there is according to McGillivray and Stam (2004, 158) a selection effect in the literature on economic sanctions because if the sanctions are going to work, they will not need to be applied. If the target state concedes just because of the threat of the sanctions, it is not an imposed economic sanctions case but the sanctions did actually work. This statement by McGillivray and Stam was made before the release of the TIES dataset which includes cases where only threats had been made. But, the selection effect is also applicable when studying unilateral and multilateral sanctions. According to Miers and Morgan (2002), multilateral sanctions are less effective than unilateral sanctions because there is a selection effect. Economic sanctions will only be multilaterally imposed when the issue at stake is 3

fundamentally important to both the sender and the target. If that is the case, the costs can be very high to the target, but it will not change its behaviour if the sanctions concern a high salience issue. According to Bapat and Morgan (2009, 1077), the selection effect states that it is costly for countries to build a coalition because countries are only willing to build a coalition if the issue at stake is important, and because the target is not likely to concede when the issue is also important to the target, the costs will be high for the coalition. Bapat and Morgan (2009, 1078) state that if sanctions are mostly multilaterally imposed when the target is least likely to acquiesce, it is not surprising that multilateral sanctions appear to fail more than unilateral sanctions do. Interestingly, Bapat and Morgan (2009) use this argument in a different way than Miers and Morgan (2002). Miers and Morgan (2002) use this argument to eventually conclude that the number of issues involved are crucial for the success of economic sanctions, while Bapat and Morgan (2009) use this argument to argue that the salience of the issue can be determining for the success of economic sanctions. Drezner (2003) approaches the selection effect in a different manner and suggests that the selection effect implies that we observe mainly failures in economic sanctions because countries have an incentive to reach an agreement in the threat phase, before the sanctions could be actually implemented. Because it is difficult to observe threats, it is possible according to Drezner (2003, 644) that a selection bias has affected empirical research of the success of economic sanctions. The argument of Drezner (2003) does not state whether this is more common with unilateral or multilateral sanctions. Public goods In economic terms, Kaempfer and Lowenberg (1999, 40) claim that multilateral sanctions are thought to be more effective because the more nations participate in the sanctioning coalition, the more damage can be imposed on the target economy. Politically, on the other hand, multilateral sanctions can backfire the situation in the target country (Kaempfer and Lowenberg 1999, 51). Precisely because multilateral sanctions can have devastating economic effects, their political effectiveness can be diminished. The main cause has to do with the opposition in the target country. According to Kaempfer and Lowenberg (1999, 51) sanctions, unilateral or multilateral, could only succeed when there is a well organized opposition group whose political effectiveness could be enhanced as a consequence of the sanctions. For 4

example, when a coalition of states wants to change the regime type in a certain state by using sanctions, the opposition in the target state has to be strong enough to overthrow the ruling government. Nevertheless, unilateral sanctions are thought to be more effective than multilateral sanctions when the opposition is not strong enough, because the ruling government will establish draconian controls over the economy when the economic costs are high as in the case of multilateral sanctions. Next to that, even when the opposition is well organized, unilateral sanctions are more effective because, according to Kaempfer and Lowenberg (1999, 53), when sanctions are imposed multilaterally there is an incentive for countries to free-ride in their responsibilities for enforcing the sanctions. Furthermore, it can be difficult for an alliance of countries to determine the compensation to countries that have to bear a great amount of the costs of the sanctions, for instance neighbouring countries of the target (Kaempfer & Lowenberg 1999, 53). This might affect the credibility of the sanctions. Drezner (2000) claims that there is an incentive for countries to free-ride because, even if all countries of the coalition are better off with the imposition of multilateral sanctions, individual countries could be better off if they unilaterally defect while the other countries cooperate. As the coalition becomes larger, the incentive to defect rises (Drezner 2000, 83). Drezner (2000) states that there are two different forms of defection; either private agents engage in illicit trading or secondary senders announce an official change in policy and overtly trade with the target country due to domestic pressure. Miers and Morgan (2002, 119) add to the first form that since sanctions require governments to prohibit trading by firms, it is easy for those governments to turn a blind eye while firms continue doing business with firms in the target country. Bapat and Morgan (2009, 1079) notice the same problem concerning free-riding and state that there is no such incentive in unilateral sanctions, but the costs imposed on the target are also lower than the costs that could be imposed under multilateral sanctions. A solution to this public goods problem is proposed by all the authors just mentioned and will be discussed in the section about the involvement of international institutions. These authors suggest international organizations or institutions could solve the free-ride problem by providing leadership in the negotiation about sanctions and by reducing the leakiness of multilateral sanctions (Miers & Morgan 2002, 131). 5

Spatial theory The main argument of Miers and Morgan (2002, 118) is that multilateral sanctions fail because it can be difficult for a coalition to determine a common bargaining position, which can lead to a bargaining advantage for the target country. The main cause for this difficulty has to do with how many countries and how many issues are involved in the sanctioning as stated by Miers and Morgan (2002). There are two problems when spatial theory is applied to multilateral sanctions. First of all, as Miers and Morgan (2002) show, it is difficult to choose a common position in the bargaining space. Second of all, as the coalition in multilateral sanctions has to negotiate about the demand, they can move to any possible position, even to the most preferred outcome of the target state according to Bapat and Morgan (2009, 1080). When the coalition places a high value on staying together, it is especially likely that the coalition moves to the preferred outcome of the target (Miers & Morgan 2002, 119). Miers and Morgan (2002) argue that multilateral sanctions are more effective than unilateral sanctions if there is only one issue involved. When sanctions are imposed multilaterally and if there are multiple issues involved, the success rate is at its lowest (Miers & Morgan 2002, 129). This problem can be solved by the involvement of an international organization. Miers and Morgan (2002) show that multilateral sanctions can be effective just as what would intuitively seem correct, but this conventional wisdom should be modified by their finding that there should be only one issue involved. In contrast to the public goods problem, which could in any case be solved by institutions, in the spatial explanation institutions are only necessary if there is more than one issue involved in multilateral sanctions (Bapat & Morgan 2009, 1081). 2.2 WHY MULTILATERAL SANCTIONS ARE MORE EFFECTIVE Morgan and Bapat (2009, 1075) argue that policymakers often advocate the use of multilateral sanctions because a coalition of states can create stronger signals to a target government and impose greater costs if the target does not comply with the senders demands. However, HSE stated that unilateral sanctions are more effective and had empirical evidence to strengthen this claim. Bapat and Morgan (2009) found with their research contradictory evidence with the TIES dataset that in 2009 consisted of 888 cases, namely, that multilateral sanctions are more effective than unilateral sanctions even without the involvement of an institution. 6

Bapat and Morgan (2009, 1091) have also examined the theories just mentioned while doing their research and found only clear support for the spatial theory which states that multilateral sanctions will be more effective if the issue in dispute is unidimensional, or multidimensional with an international institution (Bapat & Morgan 2009, 1092). The reason for the lack of support for the selection effect and the public goods theory in the research by Bapat and Morgan (2009) could be that these theories, like the spatial theory, are based on the conclusions of HSE and are not empirically grounded. In other words, these theories are not strengthened by empirical research but by speculation. If the findings of HSE are actually flawed, as Bapat and Morgan claim, then this would mean that these theories are likely also flawed. It is important to note here that multilateral sanctions are different from sanctions with the support of an institution or an international organization when comparing HSE (2007) with Bapat and Morgan (2009). However, scholars are not always consistent with the definition of multilateral sanctions. This will be discussed in the next section about the difference between multilateral and institutionalized sanctions. 2.3 MULTILATERAL AND INSTITUTIONALIZED SANCTIONS There is a crucial distinction that has to be made when examining multilateral sanctions. Multilateral sanctions are different from sanctions that are supported by an institution. As mentioned in the introduction, this thesis focuses on multilateral sanctions without the support of international organizations or institutions because the disagreement between HSE and Bapat and Morgan is also about multilateral sanctions without the support of an institution. Unfortunately, scholars are not at all consistent with this distinction. HSE are not quite clear about the distinction between multilateral and institutionalized sanctions. HSE based their conclusion about the effectiveness of unilateral sanctions on the extent of international cooperation and the involvement of an institution (Hufbauer et al. 2007, 173). The involvement of an institution is determined by a variable that states whether the sender and the target were members of an organization that supported the sanctions (Hufbauer et al. 2007, 174). Yet, the variable states membership instead of involvement in the economic sanctions which is quite confusing and maybe not that useful to determine the effectiveness of the involvement of an international organization. 7

Bapat and Morgan (2009) explicitly distinguish multilateral sanctions from institutionalized sanctions when assessing the question whether unilateral or multilateral sanctions are more effective. Bapat and Morgan (2009, 1085) show that according to their empirical evidence, based on the TIES dataset, that multilateral sanctions actually appear more effective than do unilateral sanctions, even if an institution is not involved. It is clear that Bapat and Morgan (2009) tried to prove multilateral sanctions to be more effective without the specific involvement of an international institution. Morgan, Bapat and Kobayashi (2014, 550) also claim that multilateral sanctions are more effective than unilateral sanctions with the extended TIES dataset with 1412 cases. However, in this case, they did not make the distinction between multilateral and institutionalized sanctions. In this case, multilateral is thus defined by multilateral sanctions and institutionalized sanctions. This finding of Morgan, Bapat and Kobayashi (2014) does not create a problem, but the reference they make to the conclusion of Bapat and Morgan (2009) to strengthen their finding is confusing because Bapat and Morgan (2009) define multilateral in a different fashion. International institutions are seen as the key factor to make multilateral sanctions more successful than unilateral sanctions and furthermore solve the free-ride problem. Drezner (2000, 98) states for instance that international cooperation amongst the senders leads to greater coercive power and international organizations could prevent backsliding. Also, Drezner (2000, 98) claims that an advantage of international organizations is that countries that value the existence and maintenance of international organizations will be less willing to violate a previous commitment. Furthermore, Drury (1998, 507) states that an international organization not only creates a place for debate, but also gives legitimacy to the sanctioning effort. Early and Spice (2015, 358) discovered that the support of the UN or the support of the EU does not always affect sanctions in a positive way, but these institutions can constrain their members from undercutting the sanctioning. However, there exists variation in the ability of institutions to prevent their members from engaging in trade-based sanctions-busting (Early & Spice 2015, 356). In other words, when institutionalized sanctions are also considered multilateral sanctions, it is possible according to Early and Spice (2015), that these sanctions 8

are less effective because it is hard for large institutions (like the UN) to make sure the member states do not participate in sanctions-busting activities. 2.4 TWO HYPOTHESES The three theories discussed took the ineffectiveness of multilateral sanctions for granted and attempted to explain this ineffectiveness. These theories are not based on empirical research. If the conclusion by HSE is wrong, it is possible that these theories, which are based on that conclusion, are also partially wrong. To find out whether HSE (2007) or Bapat and Morgan (2009) were right and to answer the research question, there are two possible answers which are stated in the next two hypotheses. H1: The difference can be explained by the different coding systems for success and for the level of cooperation. One explanation for the different conclusions based on the two datasets can be that success has been defined in a different fashion. The first hypothesis shows that, if the same cases in the TIES dataset and the HSE dataset are examined and they provide different results, each dataset has a different system for the coding of success. If this hypothesis is correct, it would mean that certain cases have been wrongly judged as a success or failure by one of the two scholar groups. Next to the coding for success, it is also possible that the datasets disagree about how sanctions were implemented. Even though success and the level of cooperation should be consistent, considering the confusing definition of unilateral and multilateral in the HSE dataset, this could also be a possibility. H2: The difference can be explained by the selection of cases HSE and Bapat and Morgan have made. The second hypothesis states that there is a selection bias in the HSE dataset and in the TIES dataset. It is possible that HSE selected cases for their dataset that unintentionally prove unilateral sanctions to be more effective while a lot of other cases (which could be for instance incorporated in the TIES dataset) show multilateral sanctions to be more effective. Also, there could be selection bias in the TIES dataset. Perhaps Bapat and Morgan only observed a limited selection of cases that prove multilateral sanctions to be more effective while the other cases in the TIES dataset show that the level of cooperation does affect the effectiveness of sanctions in another way. 9

3. DATA & METHODOLOGY 3.1 DESCRIPTION HSE DATA The first dataset is called HSE and is named after the creators of the dataset; Hufbauer, Schott, Elliott and Oegg. This dataset consists of 204 cases. These cases of economic sanctions took place between 1914 and 2006. The main question HSE wanted to answer by using this dataset was Do sanctions work? (Hufbauer et al. 2007, 156). To answer this question HSE developed several variables to answer this question. Eventually, HSE found that sanctions were at least partially successful in 34% of the cases they examined. Considering the influence of the level of cooperation on the effectiveness of sanctions, it is interesting to note that HSE changed their conclusion between editions of their book. In the second edition, HSE (2007, 173) stated that the relationship between international cooperation and the probability of success was negative on average. In their third edition, HSE (2007, 173) show that international cooperation does not make a clear difference in successes and failures of economic sanctions, and the relationship may even be negative when the sanctions concern modest goals or demands for regime change. According to HSE (2007, 173), international cooperation may be helpful in cases when the sender country has ambitious goals but it is not essential for the success of economic sanctions. The authors conclude by stating that sanctions should be implemented unilaterally or designed in genuine cooperation with one s allies. In the latter case, the support of an international organization could be helpful, but not necessarily decisive (Hufbauer et al. 2007, 175). To compare these findings with the conclusion of Bapat and Morgan (2009) based on the TIES dataset, the variables that will be used for the analysis in this thesis are the final outcome variable, the international cooperation variable and the cooperating organization variable. The final outcome variable states whether the sanctions succeeded. This variable is a multiplication of the scores of two other variables. Whether sanctions were a success is determined by the extent to which the policy result sought by the sender country was in fact achieved and by the contribution to success made by the sanctions (Hufbauer et al. 2007, 49). Both variables are scaled from 1 to 4, to score each element. Hence, the multiplication of the policy result variable and the sanctions contribution variable define the eventual outcome. 10

The eventual outcome of the sanctions was scored on a scale from 1 to 16 (Hufbauer et al. 2007, 159). Note that not all 16 values exist because the score is created by a multiplication, thus the only possible values are 1, 2, 4, 6, 8, 9, 12 and 16. HSE considers a value of 1 until 8 a failure and a value of 9 or higher a success. HSE code international cooperation to determine the extent of international cooperation in economic sanctions. The international cooperation variable consists of four categories, (1) no cooperation, (2) minor cooperation, (3) modest cooperation and (4) significant cooperation which state the level of international cooperation (Hufbauer et al. 2007, 58). This variable does not automatically reveal whether sanctions were unilaterally or multilaterally imposed and it is not clear whether there was an institution involved. 3.2 DESCRIPTION TIES DATA The second dataset, called Threat and Imposition of Economic Sanctions (hereafter TIES), consists since 2014 of 1412 cases and these cases took place between 1945 and 2005. The dataset Bapat and Morgan (2009) used was an older version and that version consisted only of 888 cases and these sanctions took place between 1971 and 2000 (Morgan et al. 2009). Economic sanctions are defined as actions that one or more countries take to limit or end their economic relations with a target country in an effort to persuade that country to change its policies (Morgan et al. 2014, 543). Next to cases where sanctions were actually imposed, there are also cases in this dataset where only threats have been made, so called threat cases. There are only a few of such cases in the HSE dataset. In contrast to cases in the HSE dataset there are cases in the TIES dataset without a final outcome because the case was either ongoing or the creators of the dataset were unable to determine the outcome. The final outcome of a sanction case in the TIES dataset is indicated by a variable with ten categories. The first five categories are the same as the last five categories. The first five categories focus on the threat cases. The last five categories focus on the cases where sanctions were actually imposed. The possible categories are (1 or 6) partial acquiescence by target, (2 or 7) complete acquiescence by target, (3 or 8) capitulation by the sender(s), (4 or 9) stalemate, (5 or 10) negotiated settlement. In the cases where the final outcome is partial or complete acquiescence by the target state, the sender state succeeded in (partially) altering the behaviour of the target state and the target state agrees to all, or some, of the demands of the sender(s) (Morgan et al. 2013, 12). Capitulation by the sender(s) indicates that the sender(s) 11

removed the sanctions despite the refusal of the target to alter its behaviour (Morgan et al. 2013, 13). Stalemate indicates a situation where the issue remains unresolved and when the sanctions end in a negotiated settlement, the target alters some of its behaviour in exchange for actions taken by the sender(s). There are three different ways to determine the success of economic sanctions in the TIES dataset (Morgan et al. 2014, 546). These definitions of success apply to the threat cases as well as the cases where sanctions were actually imposed. The first two definitions of success are based on the final outcome variable with the ten categories. The restrictive definition of success considers only cases where the target partially or total acquiesced as successful cases. The negotiated definition of success considers next to partially or fully acquiescence also negotiated settlement as a success. The third definition is based on two other variables; the settlement nature for the sender state variable and the settlement nature for the target state variable. The settlement nature for both the sender and the target is scored on a scale from 1 to 10 in each variable. These scores indicate, according to the coder s perception, the proportion of the sender s or target s goals that were met in the case (Morgan et al. 2013, 13). When the value of the variable for the sender is greater than the value of the variable for the target, then the case can be considered successful (even though both values can be low). For this analysis, the definition which also considers negotiated settlements as successes will be used for this thesis. This definition is also used by Bapat and Morgan (2009) because this definition is closest to the definition of HSE. It should be noted here, that Bapat and Morgan excluded threat cases because the HSE dataset only consists mostly of cases where sanctions were actually imposed. While there are certain threat cases identified in the TIES dataset as HSE cases for this analysis, the negotiated definition will also include the threat categories. Using the definition that also considers negotiated settlements as successes, Bapat and Morgan (2009, 1082) got a success rate of 23% with the TIES dataset that consisted of 888 cases. To determine if cases were imposed unilaterally or multilaterally, there are several variables that should be examined in the TIES dataset. In the TIES dataset there are multiple variables that state which countries were the sender countries and there is an extra variable that states which country was the main sender. The countries are indicated by a Correlates of War (COW) country code. Cases with only one sender are considered unilateral cases. Cases with 12

more than one sender are considered multilateral cases. To make sure multilateral sanctions are distinguished from institutionalized sanctions, all cases are controlled for the institution variable. The institution variable indicates whether an institution was involved in the sanctioning case. When there was an institution involved, the case will be marked as an institutionalized sanctions case. 3.3 METHODOLOGY To find out why analysis based on the two datasets reaches differing conclusions concerning the effectiveness of unilateral or multilateral economic sanctions it is important to study the same cases in the different datasets. Since the TIES dataset is considerably larger, an attempt has been made to identify all of the 204 HSE cases in the TIES dataset. This was done by examining the information in the HSE dataset concerning sender and target countries and beginning and ending date of the sanctions. It appeared that it was not always easy to match the cases. While identifying the HSE cases in the TIES dataset, it came forward that it is impossible to match all the HSE cases in the TIES dataset. Eventually only 92 HSE cases were identified in the TIES dataset. Strangely, 13 cases that were identified only as threat cases in the TIES dataset were included as HSE cases while these are not all labelled as threat cases in the HSE dataset. 1 Next to that, there were three threat cases from the HSE dataset identified in the TIES dataset, but one of these cases was not labelled as threat cases in the TIES dataset. There are also three cases with the wrong score for success in the TIES dataset, while two cases where sanctions were not imposed got a score as if the sanctions were actually imposed and there is one case where sanctions were imposed but this case got a score as if the sender state only threatened the target state with sanctions. 2 Because there are 13 threat cases identified as HSE cases, the definition of success includes the threat categories of the final outcome variable. Next to that, HSE and the creators of the TIES dataset disagree about the end of certain cases. 3 There are 7 ongoing cases in the TIES dataset identified as HSE cases of which 4 are marked as completed in the HSE dataset. There are 2 ongoing cases that are identified as HSE cases in the TIES dataset which got a failure score. There is also 1 case in the selection of the 1 See Appendix A 2 See the remarks at Appendix A 3 See Appendix B 13

TIES dataset which is not ongoing, but in this case, the creators could not determine a fitting final outcome. This case (50-1) has an end year but as the creators of the dataset state in their manual if the outcome does not match the categories listed, the coder should code the variable as other and fill the outcome in the blank field at the bottom (Morgan et al. 2013, 12). There are also 8 ongoing cases in the HSE dataset that are identified in the TIES dataset of which 5 cases are marked as completed in the TIES dataset. Most of the ongoing cases in the TIES dataset did not get a final outcome score, while the ongoing cases in the HSE dataset were all marked as failures. A possible explanation for the disagreement about the end of several cases is that the TIES dataset is lastly modified in 2014 and the HSE dataset in 2007. Because the results will be more robust if there is little confusion about the final outcome of the cases, the cases with no final outcome in the TIES dataset will be left out of all the analyses. That leaves 86 cases which could be examined. Even though HSE and Bapat and Morgan do not agree about the ending of several cases and sometimes give these cases a final outcome score when the cases are still ongoing, these cases are included in the selection of cases for the analyses in this thesis. 3.4 DISCUSSION ABOUT SUCCESS In the TIES dataset with 1412 cases there are 388 cases with no final outcome. Six of these cases are identified as HSE cases. While these cases have no final outcome, these cases are excluded from the analyses. It is not clearly stated in the article by Bapat and Morgan (2009) whether they include cases with no final outcome in their analysis about the effectiveness of sanctions. When Bapat and Morgan (2009, 1082), for instance, use the definition of success which includes negotiated settlements as successes, they find 404 failed sanction cases and 118 cases of success which indicates an overall 23% success rate in the TIES dataset with 888 cases. When reproducing the analysis with the same dataset they used, it is only possible to come near this percentage when the restrictive definition of success is used and if the missing cases are labelled as failures! In the HSE dataset cases with no final outcome are also defined as failures in their analysis, but the main difference is that there are only 16 cases with no final outcome (these cases are marked as failures because they are ongoing) of the 204 cases in the HSE dataset and 353 cases of the 888 cases with no final outcome in the TIES dataset that was used by Bapat and 14

Morgan (2009). Thus, in the HSE dataset 7% of all cases are defined as a failure because of an absence of a final outcome and in the TIES dataset 40% of all cases are defined as a failure for the same reason by Bapat and Morgan (2009). These confusing findings cause doubt on the reliability of the research done by Bapat and Morgan. 15

4. ANALYSIS & RESULTS To explain the differences in the different datasets, the variables that will be used are transformed to make them more applicable to the selection of cases that is examined in this thesis. 4.1 HSE: INTERNATIONAL COOPERATION AND SUCCESS HSE (2007, 173) claim based on their international cooperation variable that unilateral sanctions are more effective than multilateral sanctions. This variable will be used to create a new variable. The new variable is created by turning category (1) of the international cooperation variable in the unilateral category. The other values, (2), (3) and (4) of the international cooperation variable are turned in the multilateral category. Then a third category, which states whether the sanctions were imposed through an organization, is added to the new variable. The third category is made by using the original HSE variable, cooperating international organization. This variable indicates the organization that supports the senders in the sanctions episode either by imposing sanctions itself or taking other supporting actions (HSE 2007, 77). The cases with the involvement of an organization were put in the category of institutionalized sanctions. As stated in the description of the HSE dataset, the success of economic sanctions is indicated by a 16 points scale. To use this variable in the analysis, it is transformed from a categorical variable to a dichotomous variable. The values (1 8) are considered a failure and (9 16) are considered a success. 4.2 TIES: INTERNATIONAL COOPERATION AND SUCCESS To create a variable that states whether sanctions were unilaterally or multilaterally imposed, the number of senders are examined. If there was only one sender involved, the sanctions fall in the unilateral category. If there was more than one sender involved, the sanctions fall in the multilateral category. A third category, which indicates whether it is an institutionalized sanctions case, is created by examining the original TIES variable, institution. The institution variable states if the sanctions were conducted through an international institution (Morgan et al. 2013, 2). If there was an institution involved in a certain case, independent of the number of senders, the case is labelled as institutionalized. 16

There are several possibilities to determine the success of economic sanctions in the TIES dataset. The one that will be used in this analysis includes the threat categories because there are several threat cases identified as HSE cases. The cases with a final outcome score of (1 or 6) partial acquiescence by target, (2 or 7) complete acquiescence by target and (5 or 10) negotiated settlement are considered success cases. The cases with a final outcome score of (3 or 8) capitulation by the sender(s) or (4 or 9) stalemate are considered failed cases. Cases with no final outcome in the TIES dataset are not examined because it is not possible to determine whether these cases were effective. In total, 86 cases are used for the analyses in this thesis. 4.3 BIVARIATE ANALYSIS: SUCCESS Table 1 shows the disagreement about the failures and successes of the selection of cases. Note that there are only 86 cases in this table because there are 6 cases where no final outcome has been coded in the TIES dataset. There is disagreement about the effectiveness of 37 cases which is 43% of the cases that are examined! There are especially many cases marked as a success in the TIES dataset that are marked as a failure in the HSE dataset. It is striking that both scholar groups only agree about 49 of the cases (57%) which is more than half of all cases. Even though this table does not provide us with enough information to base our conclusions on, it is possible to claim that one of the reasons the creators of the datasets conclude differently about multilateral sanctions has to do with their coding for success. This cautious claim would support the first hypothesis. Table 1. Failure or success in the HSE and TIES dataset TIES HSE Failure Success Total Failure 20 34 54 Success 3 29 32 Total 23 63 86 This surprising finding could be further explained by table 2 which examines the different coding systems for success more broadly. This second table shows the HSE scores and the TIES scores for success. The threat and imposed scores are combined in the TIES scores. There is definitely a problem with the validity of the operationalization of success! No wonder, the different scholar groups get different results when studying the effectiveness of multilateral economic sanctions. The discrepancy could perhaps partially be explained by the 17

way HSE has drawn a line between failures (1-8) and successes (9-16). If HSE would consider the final outcome score of 8 also a success, then there would be disagreement about only 31 cases which is 36% of the selection of cases. However, it remains remarkable that these percentages are this high when the same cases are examined. Table 2. Failure or success in the HSE and TIES dataset TIES Partial acquiescence Complete acquiescence Capitulation Stalemate Negotiated Total by target by target by sender settlement HSE 1 3 0 1 2 1 7 2 0 1 5 2 6 14 4 1 4 3 2 3 13 6 4 0 3 1 4 12 8 0 6 1 0 1 8 9 4 2 0 1 3 10 12 2 5 1 0 7 15 16 2 4 0 1 0 7 Total 16 22 14 9 25 86 4.4 BIVARIATE ANALYSIS: HOW WERE THE SANCTIONS IMPOSED Not only HSE but also Bapat and Morgan (2009) are not clear about their definition of unilateral, multilateral and institutionalized sanctions. Perhaps that is why cases are also coded differently when distinguishing unilateral cases from multilateral and institutionalized cases. As shown in table 3, both datasets differ in their opinion about a lot of cases. The creators of the different datasets especially do not agree about the number of multilateral cases. According to the TIES dataset, there are only 9 multilateral cases in the selection but according to HSE there are 31 multilateral cases. In the TIES dataset there are 33 cases that have got the support of an international organization while there are only 16 institutionalized cases in the HSE dataset. Table 3. How are the sanctions imposed in the HSE and TIES dataset? TIES HSE Unilateral Multilateral Institutions Total Unilateral 33 2 4 39 Multilateral 9 6 16 31 Institutions 2 1 13 16 Total 44 9 33 86 18

It is important to show this discrepancy because this will also come back in the next section about the effectiveness of sanctions. Table 4 shows the simplified version of the difference in cooperation as stated by HSE and Bapat and Morgan. When dividing the cases in only two categories, there is less disagreement about the cases, but there are still 17 cases (20%) that are differently judged. Table 4. How are sanctions imposed in the HSE and TIES dataset? TIES HSE Unilateral Multilateral Total Unilateral 33 6 39 Multilateral 11 36 47 Total 44 42 86 4.5 EFFECTIVENESS OF MULTILATERAL SANCTIONS Table 5 shows that the definition of success is crucial for the conclusion about the effectiveness of sanctions. In table 5A the definition of international cooperation by HSE is used in combination with the definition of success by HSE and in table 5B the HSE definition of international cooperation is used in combination with the definition of success by Bapat and Morgan. With the same cases in the same dataset and with the same definition of cooperation by HSE the success rate differs enormously. It seems as if the coding is in the wrong order, but it is not! Overall, with the definition of success by HSE (table 5A), sanctions are not very effective and institutionalized sanctions are the most effective. With the definition of success by Bapat and Morgan (table 5B), sanctions are overall much more effective and institutionalized sanctions are also the most effective. Note that the category with institutionalized sanctions has the lowest number of cases. When using the definition of success by Bapat and Morgan, it seems that the level of cooperation, does not make a clear difference for the effectiveness of sanctions. 19

Table 5A. HSE cooperation and HSE success Failure Success Total Unilateral 23 16 39 59% 41% 100% Multilateral 23 8 31 74% 26% 100% Institutionalized 8 8 16 50% 50% 100% Total 54 32 86 63% 37% 100% Table 5B. HSE cooperation and TIES success Failure Success Total Unilateral 11 28 39 28% 72% 100% Multilateral 8 23 31 26% 74% 100% Institutionalized 4 12 16 25% 75% 100% Total 23 63 86 27% 73% 100% When the definition of international cooperation by Bapat and Morgan is used in combination with the definition of success by HSE (table 6A), sanctions are overall not very effective. It is interesting to note that this combination shows multilateral sanctions to be the most effective. When the definition of international cooperation by Bapat and Morgan in combination with the definition of success by Bapat and Morgan is examined (table 6B), sanctions are overall far more effective and there is no big difference between the different cooperation categories. Also, it is notable that the definition of international cooperation by HSE or TIES does not seem to affect the effectiveness of sanctions when the definition of success by Bapat and Morgan is used. The success rate is almost equal among the different categories. 20

Table 6A. TIES cooperation and HSE success Failure Success Total Unilateral 28 16 44 64% 36% 100% Multilateral 5 4 9 56% 44% 100% Institutionalized 21 12 33 64% 36% 100% Total 54 32 86 63% 37% 100% Table 6B. TIES cooperation and TIES success Failure Success Total Unilateral 12 32 44 27% 73% 100% Multilateral 3 6 9 33% 67% 100% Institutionalized 8 25 33 24% 76% 100% Total 23 63 86 27% 73% 100% 4.6 COMPARISON SELECTION OF CASES IN BOTH DATASETS To see whether the second hypothesis, which states that the HSE dataset could be flawed due to an unintentional selection bias, could be examined by comparing the selection of 86 cases with the 112 other cases in the HSE dataset and with the 938 other cases in the TIES dataset. There are some notes that should be made before analyzing the results of the 112 cases. There are in total 16 ongoing cases in the HSE dataset. 4 These case are all marked as failures according to HSE. Eight of these cases are identified in the TIES dataset and are thus among the 92 identified HSE cases. Four of these are excluded from the analyses because they did not get a final outcome score in the TIES dataset. The other four are included in the analyses because they did get a final outcome score in the TIES dataset. The other eight cases that are ongoing in the HSE dataset could not be identified in the TIES dataset and are among the 112 cases. These cases are included in the analysis with the 112 cases because HSE got these cases a final outcome score. There are thus several ongoing cases included in this analysis, which got a failure score because they were ongoing at the time they were coded. 4 See Appendix C 21

Table 7 states that, in the HSE dataset, the 86 cases and the other 112 cases show quite the same numbers for unilateral and multilateral sanctions. Institutionalized sanctions are however far less effective for the 112 cases than for the 86 cases, but this category consists of the lowest number of cases. Table 7 shows that the selection of cases is quite representative for the whole dataset, except for the institutionalized cases. This information will be compared to the findings in table 8 to address the second hypothesis. Table 7. Effectiveness of sanctions in the HSE dataset Failure Success Total Unilateral 23 16 39 59% 41% 100% Multilateral 23 8 31 74% 26% 100% Institutionalized 8 8 16 50% 50% 100% Total 54 32 86 63% 37% 100% Failure Success Total Unilateral 21 17 38 55% 45% 100% Multilateral 36 14 50 72% 28% 100% Institutionalized 17 7 24 71% 29% 100% Total 74 38 112 66% 34% 100% As table 8 shows, there is quite a difference in the scores for success. The selection of the 86 cases got a very high success score, while the other 938 cases in the TIES dataset got overall a lower success rate. 5 When multilateral and institutionalized sanctions are combined for the 938 cases, multilateral sanctions (thus including institutionalized sanctions) are more effective than unilateral sanctions. In this case, it is possible to come near the findings about the effectiveness of multilateral sanctions by Morgan, Bapat and Kobayashi (2014, 550). Because both datasets have different coding systems for success and for the level of cooperation it is difficult to accept the second hypothesis. However, when analyzing both table 7 and 8, it seems as if the cases that are only in the TIES dataset show that multilateral sanctions (including institutionalized sanctions) are more effective. 5 There 938 cases while there are 1412 cases in total, minus 92 identified HSE cases, minus 382 cases with no final outcome 22

Table 8. Effectiveness of sanctions in the TIES dataset Failure Success Total Unilateral 12 32 44 27% 73% 100% Multilateral 3 6 9 33% 67% 100% Institutionalized 8 25 33 24% 76% 100% Total 23 63 86 27% 73% 100% Failure Success Total Unilateral 326 319 645 51% 49% 100% Multilateral 23 22 45 51% 49% 100% Institutionalized 76 172 248 31% 69% 100% Total 425 513 938 45% 55% 100% 23

5. CONCLUSION Overall, scholars agree about the finding that high target costs increase the chances for sanctions success, but even though it is possible to impose higher costs with a coalition, it is stated that unilateral sanctions are more effective. Based on empirical research, HSE argue that unilateral sanctions are more effective than multilateral sanctions; international cooperation is seldom decisive and the nature of the objective is critical in the achievement of success in the case of international cooperation (Hufbauer et al. 2007, 59). Bapat and Morgan (2009, 1085) found that, based on their findings in the TIES dataset, multilateral sanctions are actually more effective than unilateral sanctions. The condition for the success of multilateral sanctions has to do with how many issues are involved. To make sure no coalition member will have the incentive to free-ride, multilateral sanctions should focus on a single-issue or in the case of multiple issues, involve an international institution (Bapat & Morgan 2009, 1092). To answer the research question, there are two hypotheses addressed in this thesis. The first hypothesis is correct: The analyses of the the coding systems for success and for the level of cooperation led to the conclusion that HSE and Bapat and Morgan come to different conclusions because they code cases differently. Assuming most cases are identified in the right way, the scholar groups make their decisions for coding differently. The outcome of many sanction cases is judged differently which causes doubt on the validity of the operationalization for success in both datasets. Also, the coding systems for the level of cooperation leads to discrepancies. The level of cooperation is not clearly operationalized by either scholar group so perhaps this could be the explanation for the differences concerning the level of cooperation. The second hypothesis provides another explanation, but the results to strengthen this hypothesis are not very convincing. When the same selection of cases is compared to the other cases in the HSE dataset and TIES dataset, there are actually some differences. The selection of cases in the TIES dataset shows that the level of cooperation does not make a clear difference in the effectiveness of sanctions. The other 938 cases in the TIES dataset actually show unilateral sanctions to be just as effective as multilateral sanctions and institutionalized sanctions are more effective than both of them. 24