Improving comparative policy analysis through causal process-tracing: the case of oil nationalism in Latin America

Transcription

1 24th World Congress of Political Science RC30 Comparative Public Policy Panel Enhancing Analytical Frameworks through Comparative Public Policy: Methodological problems Improving comparative policy analysis through causal process-tracing: the case of oil nationalism in Latin America Guillaume Fontaine, FLACSO Ecuador Derek Beach, University of Aarhus 1

2 Abstract Causal process-tracing can help policy analysis to produce parsimonious theories through small-n comparison. In this paper we use a policy design framework to identify the causal mechanism linking a triggering factor to a policy outcome, underlying the value of policy instruments as empirically observable manifestations of this process. This theoretical causal mechanism can then be tested in various cases, using a method of agreement or differences. Our research design aims at explaining the effects produced by resource nationalism on political and social accountability in four Latin American oil-exporting countries (namely Venezuela, Ecuador, Brazil and Mexico). Key words: Small-N comparison; policy design; causal mechanism; accountability; resource nationalism Introduction The comparative turn undertaken by policy studies since the 1970 s (Heidenheimer, 1985: 445; Engeli and Rothmayr, 2014: 3) encompasses both applied and fundamental research (Etzioni, 2006). The experimental logic of comparison (Peters, 2013: 3) leaves enough room for both approaches, since it allows to manipulate the real world through the selection of cases, in order to determine regularities and causal relationships in the policy process. However, these traditions do not deal with causation problems likewise. On the one hand, fundamental studies are still influenced by comparative politics principles and methods, in search of law-like relations between policy inputs, outputs and outcomes (Lodge, 2007: 276; Caramani, 2014: 3-4; Rihoux et al, 2011). On the other hand, applied research still follows a pluralist tradition where thick descriptions serve contextual explanation aimed at providing insights on specific policy areas (Dodds, 2013: 13). This extremely stylized representation purposefully underlines the heterogeneity and conceptual stretching of comparative public policy, due to the differences in the treatment of governments decision-making, whether the focus is on the impact of a policy, on its outcomes or on its outputs (Peters and Pierre, 2006). Most of all, it calls for a common language, techniques and protocols that would make comparative studies actually comparable. This is particularly challenging to develop small-n comparison, which is the most common scale nowadays, after large-n and statistical research dominated the field until the 1980 s (Hassenteufel, 2005; Schmitt, 2013). Among the multiple inputs comparison can mobilize, causal process tracing has recently become quite popular in policy studies (Blatter and Haverland, 2014; Kay and Becker, 2015; Charbonneau et al, 2016). What makes process tracing attractive for policy studies as happened in international relations, political science and historical sociology (George and Bennett, 2005; Collier, 2011; Mahoney, 2012; Bennett and Checkel, 2015a) is the deterministic logics that underpins its fundamental techniques regarding case selection and the definition of the attributes of variables or factors. In this paper, we argue that causal process-tracing can help policy analysis to produce parsimonious theories through small-n comparison. We use a policy design framework to elaborate the causal mechanism linking a triggering factor to a policy outcome, underlying the value of policy instruments as empirically observable manifestations of this process. Our method focuses on systems of entities engaging in activities to produce an outcome. This stresses the importance of empirical observations of activities that are actually doing something, as evidence of the causal process linking causes and outcomes. The identified within-case mechanism can then be tested against other cases, using a method of agreement or differences. 2

3 The following section presents a policy design framework based on instruments selection. It starts by a discussion of the main typologies of policy instruments, then it explains how instruments are selected and why this process can actually be traced in order to explain policy outcomes. In the third section we deal with the question of causality, before explaining what process tracing brings out to the discussion on that issue. We explain how process tracing and comparison can be combined to produce a causal explanation of policy outcomes. In the fourth section we present a research design aimed at explaining the effects produced by nationalist oil policies on accountability in four Latin American countries (namely Venezuela, Ecuador, Brazil and Mexico). In the final section we conclude this paper by a discussion on the practical implications our method and the remaining challenges. 1. A policy design framework 1.1. Typologies of policy instruments A government s preference for a set of policy instruments reflects a governance mode, or the way a government defines policy problems, social factors and political objectives (Pierre and Peters, 2000: 201). Hence the multiplication of policy instruments has been coming along with the transformation of governance since the 1980 s, towards a major collaboration between the state and non-state actors (Salamon, 2000: 1640). Scholars used to seeing policy instruments as strictly functional and individual entities, like tools. However governments have at their disposal a set of instruments they utilize simultaneously when implementing a policy. This is key to understand that a policy design reflects in a mix of instruments, rather than in each one of them (Howlett and Rayner, 2007). Semantically an instrument is a useful artifact to execute an activity, a means or a set of means to an end (like satisfying a demand, solving a problem, etc.). However, aside from its constitutive dimension as an object (such as a law, a plan or a statistics database), a policy instrument is also a set of representations, translations and interpretations that vary and evolve across time and according to actors, just like happens with policy problems (Lascoumes and Le Galès, 2007a). Policy instruments are at the same time functional means for specific objectives and institutions that frames the way to formulate a problem (Lascoumes and Le Galès, 2007b). As such, they make possible policies through the stabilization of cooperation modes: they are technical and social devices (Lascoumes and le Galès, 2007b). Quite recently, scholars started to distinguish different types of policy instruments in order to identify their role in the general process of policy formulation, implementation and evaluation. These instruments could be classified in seven types of elements including direct provisions, transfers, taxes, contracts, authority, regulation and exhortation, to trace a process that is evaluated according to the aims and the public of a policy (Linder and Peters, 1998). Arguably the most complete typology classifies them according to their nature (product or activity), the way they reached the public (vehicle), and the administrative modality of their use (delivering system) (Salamon, 2000). For instance, instruments of direct government are materialized in goods and services, delivered by public agencies; social regulation materializes in prohibitions embodied by legal norms executed by control organizations; economic regulation materializes in prices and controls by regulatory commissions. This taxonomy is based on the instruments endogenous qualities such as efficiency, equity, versatility, legitimacy and their ability to make a difference degree of coercion, direct/indirect effects, visibility, etc. Most interventionist instruments, which tend to be less selective, are more expensive to manage but they allow for a major control over a policy; conversely, the less interven- 3

4 tionist ones are less expensive but more difficult to control than coercive instruments (Salamon, 2005). They are less expensive because they call for an indirect follow-up by private or local actors; but for the same reason they are more difficult to control and they imply educating those in charge of a policy implementation. However this diversity of criteria becomes an obstacle when it comes to analyze instruments selection. For instance, economic regulation is supposed to have a high degree of coercion, very direct but not very visible effects, while fiscal instruments have a low degree of coercion, direct and visible effects. Yet it is hardly possible to reach an objective definition and a precise measure of efficiency, equity (etc.), from these features, since they actually depend on the national context, the moment these instruments are used, and the way they are combined by a government. A more parsimonious alternative is the typology based on the state s resources of information nodality, authority, treasure and organization (Hood, 2007; Hood and Margetts, 2007). Nodality refers to instruments that produce information for the government (detectors) and by the government (effectors). The category of authority refers to legal norms and to the legal system regulating policies. Treasure includes fiscal, financial and monetary instruments that make possible or put a policy at risk. Organization aludes to people and formal institutions that are responsible for a policy or have an incidence on it. This typology encompasses all the instrumental aspects of public policies. As such, the combination of nodality, authority, treasure and organization instruments is a necessary and sufficient condition for a policy to be effective, regardless of its inputs and outputs. In the first hand, there is no policy without information, since one cannot take a decision without being informed about the problem to be treated; conversely the state produces information on policies, since state actors emit information throughout propaganda, awareness campaigns, accountability, etc. Secondly, there is no policy that is not regulated by legal norms or that is not part of a legal framework: this can be implicit (a policy does not necessarily rely on a dedicated law) or explicit (in certain regimes the law is the favored instrument of a policy). Third, there is no policy without an economic counterpart: a policy without a budget is just a statement of principle. Finally, there is not a single policy whose formulation, implementation and monitoring do not depend on a team and a state agency: although many policies are designed by others than those who implement them which is a commonly identified source of implementation gap no policy exists without an entity responsible to give it consistency. Eventually scholars have established a distinction between substantive and procedural instruments (Howlett and Ramesh, 2003; Howlett and Giest, 2013). With the categories of nodality, authority, treasure and organization, we could already understand empirically the formal instrumentation of a policy. In classical typologies, this substantive dimension commonly refers to the way a government changes or controls the production and distribution of goods and services through social conducts (Howlett, 2011: 43). The procedural dimension of policy instruments which is at best confused with the substantive dimension in other typologies refers to the way a government influences the relationships between the state and non-state actors (Howlett, 2011: 51-53). This dimension is related to the promotion or limitation of social interactions (among networks and policy communities, interest groups, associations, etc.) The selection of instruments The question of how the selection of policy instruments affects policy outcomes deals in particular with the consistency of a policy design, be it external (between policy aims and means) or internal (among the instruments) (Howlett et al, 2006; Howlett and Rayner, 2007). Although the selection of policy instruments begins at an early stage of policy formulation, this operation is re- 4

5 peated during implementation, through instruments calibration and technical adjustments. Therefore, a policy design includes the agenda-setting, the policy formulation and the socialization of the retained solutions, and their implementation, at least at an early stage (Eliadis et al, 2005; Howlett, 2011). What makes policy design complex is that a set of instruments brings out simultaneously multi-functionality, complex interplays and inter-dependence which determine the possibility to reform (or not) each instrument or to change their combination (Hill, 2005). Moreover many instruments have different functions and variable attributes and functions. Hence we cannot elaborate a map of instruments where each one occupies a definitive place (Peters, 2005). Each instrument is related to a context and it is the object of an evaluation and dynamic perceptions. The problem lays in defining the relative importance of these interplays and their results. This refers to the logic of appropriateness, according to which actors adapt their conducts and capacities to formal and informal rules, instead of adjusting means and ends according to their interests (March and Olsen, 2006). The way an instrument is akin to the logic of appropriateness and satisfies a social demand is similar to an institutional inertia, in which the coincidence of two a priori contradictory logics can generate a dynamic of creation and transformation of formal institutions, which in turn become causal factors of the policy design. Policy instrument are the product of a compromise between different actors of the decisionmaking process, even among the state s agencies. This refers to the role of interests, ideas and institutions (Kingdon, 1995; Muller and Surel, 1998), to which one can add individuals (Peters, 2015: ). The combination of these four I is key to explain the decision-making process, implementation gaps and policy failures, since they combine into endogenous and exogenous factors. Endogenous factors refer to the legitimacy of policy instruments, to the inheritance of existing instruments, and to cultural aspects regarding the routines and habits of the actors involved in the policy design. Exogenous factors refer to economic and political conjuncture, international relations and policy diffusion, etc. Last but not least, when identifying a problem and looking for a solution, actors start by muddling through (Lindblom, 1959) with what has already been made so far and what can be made, considering time, institutional capacity and political costs and benefits. Therefore policy instruments can also turn into a policy problem. A classical example that highlights this idea is the command and control of a state agency. Before effectively acting as an entity for a policy implementation, an agency may constitute a policy problem if it does not present the features of a reliable organization (for economical and technical reasons, or even for the lack of human resources) Policy instruments and policy design By policy design framework, we refer to the complementarity of policy analysis and policymaking. 1 In the broadest sense, this perspective refers to a model of causation to solve problems, a model of intervention and a model of evaluation (Peters, 2015: 5-6). The former is usually referred to as analytical frameworks in policy studies (Sabatier, 2007), while the latter is the object of an abundant literature that deals extensively with the impacts of public policies, often measured by econometric methods (Bellinger, 2007). As a matter of fact, the model of intervention through the selection of instruments is at the core of policy design. It allows to identify the core elements of a causal mechanism and to analyze the relations between these parts, in order to explain a policy outcome. Therefore it tells a lot about A rich account of the literature on policy design is presented by the Policy Design Lab, hosted by the Lee Kuan Yew 1 School of Public Policy (University of Singapore). Available [June 2016] at: 5

6 the rationale of a decision and the reasons of policy change or continuity. It also informs us about the way the environment is taken into account (or not) in order to produce an accurate response to the problem to be attended. Aside from highlighting the diversity of instruments and the complexity of their possible combinations, the typology of policy instruments by state s resources helps identifying possible sources of information (Howlett et al, 2006). The interpretation of this information brings out a rich material to policy analysis and is particularly useful to comparison. These sources complement other qualitative techniques like interviews and direct observation (by the way, organization charts are also a valuable input to identify qualified informants). Information on substantive instruments is available in public documents, departments plans, annual reports, interviews by civil servants and politics in the media, in registers of legislation by policy area and national legal databases, in public documents and annual reports by state agencies, and in civil service guides and organization charts by state agency. As for procedural instruments, the main sources of information may come from databases and assessments on specific topics, annual reports and databases of advising committees, statistical surveys and economical assessments by NGOs, research centers and think tanks, and annual reports, guides and organization charts of private entities involved in public-private partnership. Once information sources have been identified, policy instruments can be used threefold for process tracing. First, we can characterize the policy area under scrutiny, according to the number of agencies and programs involved (Howlett et al, 2006). This unit of analysis, which is defined by its programatic and instrumental dimensions (its aims and means), results from the definition of the limits (or sectorial dimensions) and components (or instrumental dimensions) of a concrete policy. A simple policy area includes only one program and one agency, a complex one involves multiple programs and/or multiple agencies. Complex policy areas can be inter-bureaucratic if a single program involves various agencies, or intra-bureaucratic if a single agency is involved in various programs. Second, we can evaluate the consistency between means and ends of the state s action, according to the degrees of abstraction of the decisions to be made (Howlett, 2009; Howlett and Cashore, 2009) (Table 1). The highest degree of abstraction of the policy design is the governance mode, which can be hierarchical or collaborative (Kooiman, 2002). At this level, policy aims are defined as long-term preferences and policy means refer to the state organization and the institutional design. The lowest level of abstraction is the public administration realm, where operational aims and short-term preferences are defined and give way to technical adjustments and administrative action. The intermediate level is the place of implementation styles, where strategic aims and medium-term preferences give way to a particular policy mix that characterizes the government s action. Implementation styles are relatively constant and sustainable, since they depend on variables that do not change for political will (such as capacity, legitimacy and complexity). Table 1. Policy instrumentation at three levels. Degree of abstraction of the policy design High Normal Low Level of government s action Governance mode Implementation style Public administration 6

7 Aims General aims and longterm preferences (ex: national sovereignty versus international integration) Strategic aims and medium-term preferences (ex: focus on correcting market failures versus state failures) Operational aims and short-term preferences (ex: focus on equity versus competitiveness) Means Institutional design, state organization (ex: resource nationalism versus neoliberalism) Policy mix, government action(ex: institutional voluntarism versus new public management) Technical adjustments, administrative action (ex: redistribution versus fiscal incentives) Source: Elaborated by the authors. Adapted from Howlett (2009) and Howlett and Cashore (2009). Third we can evaluate the coherence of the implementation style, which depends on the combination of substantive and procedural instruments, their degree of sophistication and their relative importance for a government (Howlett, 2005). On the one hand, the election of substantive instruments depends on the state s capacity (resources) and the degree of complexity of a policy area (given by the number of involved actors and the intensity of social relationships). On the other hand, the selection of procedural instruments depends on the degree of legitimacy of a government, at a sectorial or a global level. Practically speaking, public policies are ways to manipulate these instruments in order to correct or compensate the lack of legitimacy at both levels. The optimization of an implementation style depends on the coherence of the aims and means, as well as the consistency among the means (Howlett and Rayner, 2007). Implementation is optimum when the aims are coherent, the instrument mix consistent, and the policy integrated with former policies. Thus the expected outcomes are highly predictable. However, most policy designs are sub-optimum for one of the following reasons. A government may either add new instruments and aims without questioning the former, or change a policy aims without changing instruments. At worst, a government can define new aims and chose a new set of instruments but their action is hampered by a context where change is blocked. 2. Combining process tracing and comparative techniques 2.1. Causality in small-n comparison Comparison helps qualifying independent variables or causal factors, and defining their relative importance in a causal process (Peters, 2013). Comparative public policy is useful to treat separately these variables or factors, in order to define whether they are necessary and/or sufficient, and by this hierarchization of variables scholars try to get out of endogeneity and multi-causality pitfalls. Traditionally there are two ways of comparing, following the methods of agreement or differences (Mill, 1843: ). Both protocols are used to compare either similar cases (with the same hypothesized cause producing the same observed effect) or different cases (with different factors producing the same effects, or similar factors producing different effects) (Peters, 2013). Apart from seeking for a causal relation between two phenomena, comparison consists in isolating either variables or observations, to see their incidence on the policy outcome. A major difficulty is that deterministic causes cannot be established through a small number of cases including a large number of variables (due to the degree of freedom problem) (Lijphart, 2008: 250). To increase the degree of freedom it is possible to reduce the property-space of the analysis by combining two or more variables into a single one. More generally, scholars can focus on comparable cases, similar in a large number of important features but different regarding these variables (Lijphart, 2008: 250). This is where dimensions such as time, demography, economy, geography, quality of institu- 7

8 tions (etc.) intervene in controlled comparison and determine whether the scale of the analysis applies to small-n or large-n populations of cases (Van Evera, 1997: 56-57; George and Bennett, 2005: 151). The area approach, for instance to compare sets of countries, is of particular interest for small- N comparison. Large-N comparison, which dominated the field until the 1980 s (Heidenheimer, 1985; Lodge, 2007), is oriented towards the building or testing of general laws, typical of the neopositivist realm. It follows a probabilistic logic of causality (Hopkin, 2010; Przeworski and Teune, 1970), in search for patterns of regularity through quantitative methods, to predict causation. Small- N comparison, which is more common in contemporary policy studies (Steinberg, 2007; Blatter and Haverland, 2012; Schmitt, 2013), aims at finding the critical variables or factors that might explain an outcome, until we find something satisfactory we can measure or quantify. It follows a deterministic logic of causality (Beach and Pedersen, 2016a), in search for causal explanations through cross-case studies on specific policy areas, geographical regions or countries with comparable institutional systems. Hence this way of comparing is very different from the former and requires additional techniques, such as process tracing, to advance in theory-building and -testing Causality in process tracing Process tracing consists in the systematic examination of intervening variables (George and Bennett, 2015: 21), in order to confirm or disconfirm the hypothesized causes of a phenomenon, like a criminal investigation identifying the proofs that would demonstrate a person s guilt. Yet, to be more than a set of techniques and to produce or test theoretical generalizations, it requires to turn the mere observations of a pathway into proofs of an actual causal process (Bennett, 2010; Mahoney, 2012: 571; Bennett and Checkel, 2015b). The mechanistic conception of process tracing consists in unpacking what is going on between causes and effects, analyzing the way causal forces are transmitted through a set of articulated pieces of a causal mechanism, until they produce the observed outcome (Beach and Pedersen, 2013: 29). It is accurate for within-case study like policy analysis, either to engage in theory-building or theory-testing. In theory-building, process tracing aims at finding the causal mechanism through which a factor A produces an outcome B. This more inductive form of research is relevant when we know that A and B are correlated, but we do not know what links them, if anything (it can be spurious). Here we engage in a far-reaching empirical search for clues of a causal process, inspired by existing theories in related fields. Once we have a hunch about the key steps of a causal process, we can engage in more systematic empirical testing. In theory-testing, process tracing aims at verifying if a causal mechanism linking A and B together is present in a given case. It can be used to both confirm or disconfirm a deductively developed mechanism, or investigate whether a mechanism found in one case is present in other causally similar cases. When testing whether a theorized mechanism found in case 1 works in case 2, we might find that one or more parts do not work as expected in case 2. Here we would then engage in theory-building using both process-tracing and comparative methods, attempting to understand first what other parts were present in case 2, and then comparing the two cases in order to understand what contextual conditions differ between the two cases that might lead to the two different processes in the cases. At the theoretical level, conceptualizing mechanisms are broken down into distinct parts that are composed of entities engaging in activities (Machamer, Darden, and Craver 2000; Machamer 2004; Beach and Pedersen 2016a; Rohlfing 2012). Entities are the factors (actors, organizations, or structures) engaging in activities (the parts of the mechanism, i.e., toothed wheels), where the acti- 8

9 vities are the producers of change or what transmits causal forces through a mechanism (the movement of the wheels). What the entities and activities more precisely are will depend on the type of causal explanation, along with the level at which the mechanism works and the time span of its operation. The activities that entities engage in move the mechanism from an initial or start condition through different parts to an outcome. When theorizing the parts of a causal mechanism, the parts should exhibit some form of productive continuity, meaning that each of the parts logically leads to the next part, with no large logical holes in the causal story (Machamer, Darden, and Craver 2000: 3; Darden 2002: 283). At the empirical level, we utilize within-case, so-called mechanistic evidence to confirm or disconfirm to some degree whether parts of a mechanism were present and operated as we theorized (Russo and Williamson 2007; Phyllis McKay 2011). Mechanistic evidence which is more precise than causal process observations (Seawright and Collier, 2010: 318) relates to the empirical fingerprints, or observable manifestations, that the operation of parts of a mechanism leave in individual cases Testing causality What inferences pieces of mechanistic evidence enable us to make depends on the certainty and uniqueness of their existence in relation to a theorized part of a mechanism (Van Evera, 1997). 2 Certainty relates to the disconfirming power of evidence. If it was very certain that we had to find the predicted evidence based on our theory, and we do not find it in the case despite a thorough search in the empirical record, we can significantly downgrade our confidence in the existence of the part of the mechanism being studied. Uniqueness describes the expected probability of finding the predicted evidence in all other plausible explanations for the evidence, relating to the confirmatory power of evidence. In plain English, if the predicted evidence for a part of a mechanism is found, can we plausibly account for it with any alternative plausible explanation? If we find the predicted evidence and it was unique, we can make a strong confirming inference. The tests of the workings of parts of mechanisms based on predicted observables can have varying degrees of certainty and uniqueness (Van Evera, 1997; Bennett, 2010; Collier, 2011). A socalled straw-in-the-wind test has low certainty and uniqueness, meaning that finding or not finding the evidence tells us little by itself, although many independent tests of this type summed together can tell us something. Conversely, the most difficult test consists in establishing a doubly decisive test, which is highly certain evidence (it has to be found for our theorized part to exist) and highly unique (a signature fingerprint that cannot be accounted for with plausible alternatives). This would be both highly confirming if found, and highly disconfirming if not found, but unfortunately the empirical record in the social sciences is almost never so conclusive. However, many weaker tests that are independent of each other can be summed together, with the overall body of evidence enabling stronger confirmation or disconfirmation than individual tests. Between these extreme, a smoking-gun test is a piece of evidence (or more likely, a cluster of evidence) that is unique but not certain, meaning that if found it enables strong confirmation, but if not found, little disconfirmation takes place. Finally a hoop test is certain but not unique, meaning it is disconfirming if not found, but if found, little confirmation takes place. Certainty and uniqueness build on Bayesian logic. For an extended discussion on the use of bayesian logics in causal 2 process tracing, see Bennett,

10 2.4. A four-step method We propose a four-step process, with the first step involving theorizing an abstract causal process that can be potentially present within a population of causally similar cases, and a second step where clear observables for each of the key parts of the causal mechanism are developed that are expected to be present in all of the cases in the population. Third, we then map the population of potential cases to find either similar or dissimilar cases, depending on which research focus we have we have in the actual comparison in step 4. Step 1 - Conceptualizing mechanisms In order to theorize disaggregated causal mechanisms in a fashion that enables comparative analysis across cases, and in particular between cases where the same cause produces different outcomes because of different contextual conditions being present in the different cases, we need to develop robust theoretical mechanisms of policy processes that have a 'modular' property. Here we suggest thinking about theorized mechanisms of political processes as typological templates of common theoretical parts that explain policy processes. For example, a particular class of policy processes will always go through a part dealing with change, but the actual content of this part in a given policy process mechanism will depend on the context. Therefore, we can develop a typology of policy change parts that move the process forward. The actual content of these parts will vary depending on context, but the basic 'function' of a given part will be the same. Step 2 - Developing observable manifestations that can travel across cases After each part of a mechanism is theorized, we need to develop the observable manifestations they can be expected to leave in particular cases. This requires thinking carefully about the data-generating process of the working of each part of the mechanism, asking ourselves about what fingerprints have to be left in a case (certainty), and if we find them, whether the fingerprints can be accounted for with any plausible alternative explanation for the evidence (uniqueness). 3 Step 3 - Mapping the population After the theorized parts and their predicted observables have been developed, we engage in case selection of two or more causally similar cases. To do this requires that we first build a causally homogenous (or heterogenous) set of comparable cases. Building a population requires that we first draw on existing theoretical and empirical knowledge to determine a set of potentially causally relevant conditions, that might produce differences across what we initially believe might be a causally homogeneous set of cases (where both the cause and outcome are present). In studying public policy processes, this might relate to different institutional arrangements, or levels of resources that might result in causal differences across cases in the population. All cases in the initial population are then assessed to see the number of differences across the contextual conditions (Berg-Schlosser 2013). The goal is to select an initial small subset of positive cases where similarities are maximized to increase the chances that we have tapped into a causally homogeneous population (Ragin 2000: 59 62). In table 2, cases 1 and 2 share the most contextual conditions (4), and we would therefore select them for the initial comparison of causal processes. Table 2 Mapping cases for similarity Note that plausible alternatives at the level of within-case evidence typically do not come from alternative theoretical 3 frameworks (e.g. an alternative to an ideational approach would be a materialistic, rational actor model) because rival theories typically do not offer mutually exclusive and competing interpretations of individual observable manifestations of processes. For more, see Rohlfing (2014) and Beach and Pedersen (2016b). 10

11 Cause C1 C2 C3 C4 # of similarities with case 1 Outcome Step 4 - Case selection depending on the research purposes Comparison can have different purposes, as discussed earlier. The most basic one is to compare two cases that share both the cause, outcome and as many contextual conditions as possible to see whether the same mechanism works in both cases. This is a form of method-of-agreement comparison. Table 3 illustrates the core idea of comparing mechanisms across causally similar cases, defined as cases where the same cause is expected to launch the same mechanism that links it with the outcome because of shared contextual conditions. While the empirical observables should not be expected to be completely similar we should be able to argue that there is observational equivalence between the observables in the two different cases. The comparison can be depicted graphically as in the table below, we see two cases that are expected to be in similar contextual conditions, meaning that we should expect that cause and outcome are linked together through the same mechanism. The empirical analysis then assesses whether there is empirical evidence for each of the parts in both cases. Table 3 - A method-of-agreement comparison at the level of mechanisms. Cause Part 1 observables Part 2 observables Outcome Case 1 +?? + Case 2 +?? + If the analysis then finds confirming evidence in both cases, the inference can be made that the cause and outcome are linked together through similar mechanisms in the two cases. A cautious generalization can therefore also be made to other cases that share a similar set of contextual conditions. However, to be more confident we would then want to reduce the number of contextual similarities in our comparison, for example also assessing in table 2 (above) case 5 to see whether we find the same mechanism operative as in cases 1 and 2. If we find confirming evidence in case 5, we can then infer that the mechanism should be operative in the rest of the cases in the mapped population, whereas if we find negative results we would want to explore further what contextual conditions resulted in the mechanism not working as expected in case 5. In the analysis of cases 1 and 2, we might though find that the mechanism did not operate as theorized in case 2. This should lead us to compare the two cases in order to discover whether: 1) 11

12 case 1 is idiosyncratic, and in the rest of the population the relationship is spurious; 2) there is a different mechanisms in each case, which should lead us to revise the theory to account for the different process in case 2. If we find that the observables for part 2 are not present (see tables 4 and 5), this might be because the mechanism starts as theorized (part 1 is present in both cases), but there is an alternative pathway in case 2 from part 1 to the outcome, or the mechanism itself breaks down in part 2, resulting in another outcome. In the first situation, depicted in table 7, the cause and outcome is present in case 2, but there is an unknown alternative pathway linking part 1 to the outcome. Here we would then both engage in theory-building to discover the alternative part 2 in case 2, and compare the two cases to try to discover the contextual difference between both cases, that leads to different mechanisms. We speak about robustness when different parts of a mechanisms have roughly the same effects, which is actually a common situation in the natural world (Steel, 2008). Table 4 - A negative finding in case 2 Cause Part 1 observables Part 2 observables Outcome Case Case The second possibility is that the mechanism breaks down, resulting in another outcome. There can be two different reasons for a mechanism breakdown. First, there might be an omitted contextual condition that is present in case 1 but not in case 2 that can explain the mechanism breakdown that results in the negative outcome in case 2. Second, the cause might not be sufficient, meaning that there is an omitted causal condition that has to be present together with cause to enable the production of the outcome. Table 5 - A negative finding in case 2 - mechanism breakdown Cause Part 1 observables Part 2 observables Outcome Case Case Contrasting mechanisms across cases Here we are not comparing positive and negative cases per se. Instead, we are comparing two or more cases that exhibit the same cause but differ on contextual conditions and on the outcome. Here a cause A might produce an outcome B through a mechanism with parts E1 and E2 in one context, but the same cause A produces C through a mechanism with parts E3 and E4 in another. How then can we compare their workings of the same cause in different contexts? It is here that theorizing certain common causal elements that act as a template for a theorized mechanism enables comparison across dissimilar cases. We would want to develop a form of typological theory at the level of mechanisms that tells us about certain commonalities in a given form of policy process, where there are differences in what gets us from one part to the next, but there is a form of functional equivalence. This can be depicted visually as the situation in figure 1, where a cause triggers two different mechanisms depending on the contextual conditions present. In this fi- 12

13 gure, the cause (A) either triggers a mechanism that through parts E1 and E2 produces B, or through E3 and E4 produces another outcome (C), depending on which contextual conditions are present. Here comparison has two functions. First, we would want to compare cases where B and C are produced by the same cause in order to map the differences in the contextual conditions that might determine which of the outcomes is produced. Second, once we know the different contextual conditions, we can probe how the mechanism is different in each case, using either theory-building process-tracing if we have no knowledge of the parts operative, or theory-testing if we have good theoretical and/or empirical knowledge of the processes in the different cases. Figure 1 - Two dissimilar mechanisms triggered by the same cause. Element 1 Element 2 cause E1 E2 B E1->E2->B or E3->E4->C depending on context E3 E4 C In the theory-testing variant of the dissimilar mechanism comparison, the crucial step that enables us to make this form of comparison of mechanisms across cases in dissimilar contexts is the development of a theoretical template for the steps or elements that a causal mechanism of a particular type of policy process should share. In figure 1 above, depending on the context, the cause (A) will either lead to the different parts E1 or E3, both of which are the first theoretical steps in the causal mechanism linking A with either outcomes B or C. We attempt to theorize parts of the mechanism in terms of the functions that each part does, attempting to develop functionally equivalent mechanisms. 3. The case of resource nationalism affecting accountability 3.1. Step 1: Conceptualizing the causal mechanism Our research aims at explaining consistently the role of ideas and institutions in policy outcomes. We seek for a causal mechanism linking ideas and policy outcomes through institutional arrangements. In particular, we focus on policy design to describe the role of instruments selection and combination in improving or reducing social and political accountability. Drawing from the case of accountability in Latin American oil exporting countries, we intend to shed light on the relation between resource nationalism and democracy. Resource nationalism, refers to the maximization of public revenues; the assertion of strategic state control (ability to set political or strategic direction to the development of the sector); and enhancement of developmental spillovers from extractive activity (Haslam and Heidrich, 2016: 1). Mineral endowment has contradictory effects regarding economic development and democracy. For instance, while early works on Venezuela used to emphasize the negative effects of the 13

14 rentier state on democratic institutions (Karl, 1997; Ross, 2001), more recent studies have shown that oil rents might as well underlie democracy, at least until the 1990 crisis (Dunning, 2008; Ross, 2012). This controversy shows how much public policies and political institutions are key to explain the state capacity to cope with the effects of price shocks and rent booming, while attending social demands. In particular, social and political accountability are at the core of the discussion, for constituting the ultimate safeguards against patronage and corruption (Karl, 2007; Corrales and Penfold, 2011). Our hypothesis is that resource nationalism hinders democracy because it affects accountability at different levels when mineral prices are high (Corrales and Penfold, 2011). It affects vertical accountability because the incumbent benefits more from rents than outsiders, which affects their chances to be elected (Corrales, 2015). It also affects horizontal accountability since the executive concentrates power over the judicial and the legislative, which affects the effectiveness of balance agencies. It affects social accountability because the government controls information and social groups through dedicated agencies. 4 Drawing from the policy design framework, we argue that ideas and institutions interplay and reinforce one another at three levels (Table 6). At a normative level, ideas act as a trigger for a major change in the cognitive framework of a policy s stakeholders and the definition of general aims and means. This includes for instance that, to reach national sovereignty the state increases their control over the petroleum sector. At a strategic level, institutions act as a support for the ideas embodied by a policy paradigm. For instance, a government can pursue economic development through state-oriented policies. At an operational level, ideas and institutions act as an implementation mechanism of calibration and learning process. For instance, the oil policy can rely on the modernization of a national oil company in order to increase the government-take from oil production. Table 6. Policy content of nationalist oil policies by aims and means Normative level Strategic level Operational level Policy ends National sovereignty Economic development Increasing governmenttake from oil production Policy means State control over the petroleum sector State-oriented policy Modernization of national oil companies Source: Elaborated by the authors. Adapted from Howlett (2009) and Howlett and Cashore (2009). Once we identify the core element of oil policy design to describe the template of accountability in resource nationalist regimes, we will be able to build the causal mechanism linking radical resource nationalism to low accountability that is observed in some countries Policy instruments as observable manifestations To describe the role of policy design in improving or reducing social and political accountability, we utilize policy instruments as empirically observables, according to the taxonomy by the sta- Vertical accountability refers to electoral mechanisms of control by citizenship over the state; and horizontal accoun 4 - tability refers to the classic separation of the executive, the legislative and the judicial powers, while (O Donnell 1999: 38). Social accountability is a vertical but non-electoral form of pressure to have civil servants and elected representatives justify and inform on their decisions, and possibly be sanctioned when acting in a wrong or illegal way (Peruzzotti and Smulovitz 2002). 14

15 te s resources described above (see table 9). Taken as a whole, these instruments define the substantive and procedural contents of a policy design (Howlett, 2011). Nodality instruments of oil policies include information systems, technical assessments, national planning and international campaigns of advocacy, such as the EITI (Extractive Industries Transparency Initiative) and the NRGI (Natural Resource Governance Index). This includes any information produced by the state to legitimate oil policies by national sovereignty and development. This also includes the information required by the state from oil companies, in particular regarding oil exploitation and rent-seeking. Authority instruments refer to the norms regulating natural resource ownership and exploitation, contained in political constitutions, laws and regulations. They include any norm related to mineral rent management, as well as citizens rights to information, equity, environmental protection, etc. They also include contractual regimes framing the relationships between national oil companies and their foreign counterparts or private multinational consortia. At an international level, this also includes the World Bank s International Center for Settlement of Investment Disputes, which has been used by many companies during the nationalist reforms. Treasure instruments refer to specific contracts (which ought to be differentiated from the contractual regime), rent management systems, incentives to foreign direct investments, taxes, etc. Rent management systems include the way government-take is calculated, according to technical, economical and political factors. It also informs us about how the rent is distributed among the population, what proportion is to be collected by local governments. This category of policy instruments also includes macro-economic indicators such as the contribution of mineral rent in the GDP, the country s exports or the state s budget. Organization instruments refer to the state agencies (central and local administration, national oil companies, etc.) and international organizations involved in the petroleum sector, like the OPEC. The organization chart of the executive power informs us about who does what and how in oil policy. This category also includes information about balanced and dedicated agencies of political accountability, besides NGOs involved in advocacy campaigns and often at the core of the accountability process, such as EITI, the NRGI, etc. Table 4. Policy instruments according to the state s resources Information nodality Authority Treasure Organization Information systems Political constitutions Contracts National oil companies Technical assessments Ordinary laws and regulation Rent management systems State agencies National planning International laws Foreign direct investments Local administration Extractive Industries Transparency Initiative International Center for Settlement of Investment Disputes International prices International organizations Source: Authors elaboration. Adapted from Howlett and Ramesh (2003) Mapping the population of resource nationalist countries The total population of Latin American oil and gas exporting countries includes Argentina, Bolivia, Brasil, Colombia, Ecuador, Mexico, Peru, Trinity and Tobago and Venezuela. Among these, six countries have experienced resource nationalism at various degrees during the past two decades (namely Mexico, Brazil, Venezuela, Bolivia, Ecuador and Argentina) while the other three conti- 15