Orchestrating Experimentation in Non-State Environmental Commitments Kenneth W. Abbott 1

Transcription

1 Orchestrating Experimentation in Non-State Environmental Commitments Kenneth W. Abbott 1 Introduction: A striking recent development in global environmental governance is the emergence of arrangements by which non-state actors including business firms and associations, local governments, universities and international organizations submit voluntary commitments, in parallel to more traditional undertakings by states. Voluntary non-state commitments were central to the last two global sustainable development conferences, in 2002 and Since the 2014 UN Climate Summit in New York and the Lima Conference of the Parties (CoP) to the United Nations Framework Convention on Climate Change (UNFCCC), they have also become a significant part of global climate governance. The governance systems for non-state commitments in both sustainability and climate, however, apply only modest criteria and vetting procedures to commitments, provide commitment-makers with limited support, and have few means to hold commitmentmakers accountable. Perhaps most surprising, given the novelty of these systems, they have few mechanisms to promote learning from the experiences of commitment-makers. In many ways, then, the commitment approach to global environmental governance cries out for orchestration (Chan et al. 2015). One frequently recommended approach (which is in fact applied in some recent arrangements) is to strengthen the criteria for recognizing non-state commitments by incorporating so-called SMART principles: actors should make commitments that are Specific, Measurable, Achievable, Resource-based and Time-bound (e.g., NRDC 2013). SMART criteria, it is believed, would elicit stronger commitments ex ante and facilitate follow-up and accountability ex post. The implicit assumption underlying SMART is that we know what needs to be done to address climate and sustainable development problems; we simply need commitmentmakers to implement known solutions efficiently. As a result, SMART criteria, and governance arrangements that apply them, tend to elicit characteristic types of commitments: concrete, operational in character, based on recognized approaches, relatively narrow and not overly ambitious (so that they can be achievable on a timebound schedule). Almost certainly, however, we do not know all there is to know about how best to address complex environmental and sustainable development problems. Governance 1 Thanks to Ben Cashore, *** and participants in the panel on transnational environmental governance at ISA

2 arrangements that encourage diverse actions by diverse actors, across diverse scales and settings, offer unique opportunities to identify new approaches and to test those approaches against one another and against existing assumptions that is, to experiment helping stakeholders to learn. Current commitment systems based on SMART not only fail to take advantage of these learning opportunities, they may actually limit them. In a recent paper, Duncan Snidal and I argue that taking experimentation seriously as a device for exploration and learning would strengthen many areas of global governance (Abbott & Snidal 2016). Some would argue that the diverse voluntary commitments seen in sustainability and climate already constitute experiments; the growing literature on experimentalist governance suggests a more highly elaborated approach. We argue, however, that significant benefits could be realized by going further than either of these views: designing governance arrangements that incorporate the lessons of experimentation in the natural and social sciences. Doing so would require orchestration, a common feature of experimental programs; it could provide benefits even beyond learning. In this paper, I apply those lessons to voluntary non-state commitments. 1. Non-state commitments in transnational governance A. Sustainable Development Institutions addressing sustainable development have been particularly active in promoting non-state commitments alongside the largely soft-law undertakings accepted by states. Indeed, the softness of state undertakings provided the impetus for the turn to voluntary commitments. During preparations for the 2002 World Summit on Sustainable Development (WSSD), the Summit organizers saw little possibility for breakthroughs in the inter-state process. They therefore encouraged both state and non-state actors to form voluntary, multi-stakeholder partnerships for sustainable development, known as Type 2 partnerships, to implement agreed principles and goals. Over 200 partnerships were launched at WSSD itself; over a hundred more were formed thereafter. A substantial number are still in operation. While some Type 2 partnerships have produced solid results, as a whole they have been widely criticized as ineffective (Beisheim & Liese 2014; Bäckstrand & Kylsäter 2014). The criteria for partnerships, developed first by the vice-chairs of the former Commission on Sustainable Development (CSD) acting as WSSD preparatory committee, and later by the CSD itself, were very general; more importantly, they were not vigorously enforced. The focus, moreover, was on practical projects with measurable benchmarks; as a result, most partnerships undertook either concrete operational activities or broad programs of information exchange and dissemination. The CSD was designated as the focus of partnership activity in the UN, but it was poorly suited institutionally to orchestrate partnerships. It provided little support, coordination or review; its main approach to learning was to sponsor partnership fairs. The CSD did host an online 2

3 partnership database, but a subsequent review concluded that it never became a wellresourced and modern platform. 2 A recent study shows that a substantial number of partnerships did nothing at all (Pattberg et al. 2012). The 2012 UN Conference on Sustainable Development (Rio+20) adopted an expanded system for voluntary non-state commitments, no longer limiting them to partnerships. The non-binding eligibility criteria explicitly urged that commitments follow SMART criteria and identify tangible deliverables, defined as concrete policies, plans, programmes, projects and actions. 3 Over 700 commitments (including some partnerships) were announced at Rio, and more than 700 more were registered thereafter. The UN again operated an online registry and published a regular newsletter on partnerships, Sustainable Development in Action. The Rio+20 outcome document recognized that stakeholder involvement, amplified by networking and experience sharing, could generate valuable learning, 4 but little was done to implement this insight. The UN Secretary-General and Department of Economic and Social Affairs (UN-DESA), in some cases acting with partners, helped organize many voluntary commitments into action networks around substantive themes, such as Every Woman Every Child (EWEC) and Sustainable Energy for All (SE4All). Pre-existing networks elicited other commitments and continue to coordinate them. These include the UN Global Compact and the Partnership on Sustainable, Low-Carbon Transport (SLoCaT), which coordinates the Action Network on Sustainable Transport. The action networks have been responsible for much of the subsequent development and oversight of sustainability commitments; they have introduced some significant innovations, such as the accountability frameworks created by EWEC and SE43All. Their powers are uncertain, however, and their results have been uneven. States also agreed at Rio+20 to establish the High-Level Political Forum for sustainable development (HLPF) to replace the CSD. The HLPF is mandated to provide leadership, integrate the dimensions of sustainable development, encourage policy coherence, review progress and promote implementation of inter-state and non-state commitments (Abbott & Bernstein 2015). It is tasked with providing a platform for partnerships, but this has not yet been fully developed. The HLPF may also receive voluntary reports from some non-state initiatives as part of its review function. Many of the voluntary commitments made at Rio+20 were targeted at achieving elements of the Millennium Development Goals (MDGs). As a result, they were planned for 2 Lessons Learned from the Commission on Sustainable Development, Report of the Secretary-General, A/67, 21 February 2013, para UN DSD, Sustainable Development in Action: Special Report of the SD in Action Newsletter, Voluntary Commitments and Partnerships for Sustainable Development, July 2013, at 5. 4 The Future We Want, para. 64 3

4 completion by 2015, the conclusion of the MDG process. As 2015 arrived, then, many commitments were expiring; the system needed to be reinvigorated, if not reinvented. At Rio + 20, states launched a process to develop a set of Sustainable Development Goals (SDGs) to replace the MDGs and become part of the post-2015 development agenda. 5 The General Assembly meeting as the Sustainable Development Summit in September 2015 adopted the SDGs, along with the broader 2030 Agenda for Sustainable Development (Agenda). 6 Under SDG Goal 17 revitalizing the broad global partnership for sustainable development the Agenda calls for encouraging multistakeholder partnerships, including public, public-private and civil society partnerships, to support achievement of the SDGs. 7 The Agenda thus shifts the voluntary commitments system back toward partnerships rather than individual commitments. The Agenda did recognize that partnerships are important vehicles for mobilizing and sharing knowledge and expertise, as well as for generating concrete projects. 8 As UN- DESA notes in its 2015 report on partnerships and commitments, The effectiveness of multi-stakeholder partnerships will increasingly be tied to their ability to manage and share knowledge and expertise about the issues, processes, and solutions that they are promoting. 9 Later in 2015, ECOSOC agreed to convene a forum on improving partnership transparency, accountability and sharing of experiences. 10 In the run-up to the 2015 Sustainable Development Summit, UN-DESA launched a new online platform for multi-stakeholder partnerships Partnerships for SDGs replacing the registry from Rio In theory, at least, the platform is intended to facilitate collaboration and sharing of information, technology and other resources as well as transparency. It contains information on legacy partnerships and commitments as well as on new multi-stakeholder partnerships registered around the Sustainable Development Summit. As this is written, the platform lists well over 1900 commitments. Partnerships that register on the platform must identify concrete deliverables and indicate which SDG goals and targets they address; the UN again urges that they follow SMART criteria. 12 B. Climate change 5 The Future We Want, outcome document of United Nations Conference on Sustainable Development, A/RES/66/288, 11 Sept Transforming Our World: The 2030 Agenda for Sustainable Development, A/RES/70/1, 21 Oct Id. para Id. para UN DSD, Sustainable Development in Action: Special Report on Voluntary Multi-Stakeholder Partnerships and Commitments for Sustainable Development (2015), at 2 10 UN DSD, Partnerships for Sustainable Development Goals: A Legacy Review Towards Realizing the 2030 Agenda (2015), at (in beta version as this is written)

5 Negotiations under the UN Framework Convention for Climate Change (UNFCCC) have long been strongly inter-governmental in character. Outside of UNFCCC processes, however, non-state actors have initiated dozens of climate commitments, partnerships and other voluntary initiatives since the 1990s (Hoffmann 2011; Abbott 2014; Bulkeley et al. 2014; Hale & Roger 2014). 13 A substantial number of these are designed, in effect, to seek non-state commitments rather than to make them: private transnational regulatory organizations adopt standards of conduct and urge businesses and other actors to voluntarily adhere to them (Abbott, Green & Keohane 2016). In the last two years, the UNFCCC and related processes have begun to integrate some non-state commitments. In September 2014, Secretary-General Ban Ki-moon invited high-level representatives of governments, business and civil society to New York for Climate Summit 2014, designed to catalyze ambitious voluntary actions. The Summit was independent of UNFCCC negotiations, but was intended to mobilize political will for those negotiations by demonstrating the steps non-state actors in many countries were willing to take. Local governments, business groups, civil society organizations and multi-stakeholder coalitions even states made significant commitments at the Summit. 14 Many took the form of cooperative initiatives (that is, partnerships or networks) within eight action areas, including finance, energy, forests, transport and cities. For example, in the New York Declaration on Forests, numerous state and non-state actors committed to halt forest loss and promote reforestation. 15 At the Lima CoP in December 2014, the Peruvian presidency, the incoming French presidency and the UN Secretary-General launched the Lima-Paris Action Agenda (LPAA). 16 LPAA was designed to showcase existing commitments and action initiatives on mitigation and adaptation, help them scale up, and catalyze new ones for the crucial Paris CoP in December 2015 (where states were scheduled to adopt a new climate agreement) and beyond. 17 LPAA welcomes both individual commitments and cooperative initiatives like those from the Climate Summit. It focuses on similar action areas and particularly encourages transformational initiatives. LPAA does not strongly assess proposed commitments, enforce accountability or promote learning; its main goal is to demonstrate the breadth of non-state commitments. 18 It did, however, undertake to provide modest support to commitment- 13 Bulkeley et al. identify 60 transnational climate institutions; Abbott modifies their database to include nearly 70 organizations; Hale & Roger examine 75 transnational climate initiatives. 14 For a summary of major commitments, see Jennifer Morgan et al., World Resources Institute, Analyzing Outcomes from the UN Climate Summit, Sept. 23, 2014, available at 15 For a summary of the commitments made, see 16 The UNFCCC Secretariat joins them on the LPAA steering committee

6 makers. To increase the credibility of commitments, moreover, LPAA adopted explicit criteria, implicitly reflecting SMART. Under these criteria, commitments: Should be ambitious, supporting a resilient, 2 climate pathway and aiming to transform their sector Must be science-based and have concrete impacts within an action area Should demonstrate capacity for implementation Should monitor and report regularly on progress and results Must showcase implementation of past commitments Cooperative commitments must be inclusive. 19 To showcase the commitments of companies, investors, local governments and other non-state actors, the Peruvian presidency launched the Non-State Actor Zone for Climate Action (NAZCA) in parallel to LPAA. 20 NAZCA is an online portal, designed to demonstrate the groundswell of climate action 21 and build political momentum for the Paris CoP. 22 It does not assess commitments (against its own or LPAA criteria), provide support, enforce accountability or promote learning. Instead, it merely aggregates information on commitments provided by other organizations, although these were selected because of the quality of their information processes. Contributing organizations include the Carbonn Climate Registry, Covenant of Mayors and The Climate Group (for subnational government commitments); and the UN Global Compact, Carbon Disclosure Project, Investors on Climate Change and Climate Bonds Initiative (for business commitments). NAZCA currently lists nearly 11,000 commitments. The Paris CoP, in December 2015, represented a turning point for the UNFCCC: in parallel to the inter-governmental negotiations resulting in the Paris Agreement, LPAA sponsored a series of action days at which a wide range of state and non-state actors announced commitments, culminating in a high-level session sponsored by the French presidency and UNFCCC Executive Secretary to highlight the significance of voluntary commitments for achieving the goals of the Paris Agreement. 23 Important commitments were made on climate finance, forests, water, energy efficiency, buildings, local government mitigation plans, transport and other areas LPAA also encouraged interactions with national policy to facilitate compliance with national undertakings. 20 NAZCA represents the NAZCA Lines, ancient geoglyphs located in Peru. The initiative s rather awkward name was chosen to fit the acronym ; 6

7 The CoP decision accepting the Paris Agreement made unprecedented references to nonstate action. 25 It acknowledged the LPAA and Climate Summit, welcomed the non-state commitments already received, encouraged commitment-makers to register on NAZCA, encouraged governments to collaborate with non-state actors on new initiatives, and agreed to convene high-level events at subsequent CoPs to recognize new and strengthened commitments and take stock of progress. It also agreed to appoint two high-level champions to act for subsequent presidencies in supporting and eliciting commitments. 26 Over 1200 stakeholders also signed the Paris Pledge of Action, agreeing to act so as to meet or exceed the Agreement s 2 goal Non-state commitments as experiments How should we think of non-state commitments as experiments? Structurally, note first that voluntary commitments could be organized as experiments in at least two ways. A number of individual or cooperative initiatives e.g., a number of company and industry commitments in a particular domain could themselves be treated as experiments. Alternatively, an action network, multi-stakeholder partnership, cooperative initiative or other organization could make an overall commitment that included sponsoring and managing individual experimental actions under its aegis. In either case, what is substantively required for voluntary non-state actions to be regarded as experiments? Some observers would treat all voluntary commitments as experiments simply because they involve actors trying a variety of approaches (cf. Hoffmann 2011). This informal conception is legitimate: the first person to eat an oyster was undoubtedly experimenting (on him or herself). 28 But it fails to incorporate the welldeveloped understandings of experimentation in the natural and social sciences. Taking account of those understandings should increase the learning benefits of voluntary commitments. Charles Sabel, Jonathan Zeitlin and their co-authors have put forth the more highly developed concept of experimentalist governance, which has become an important lens for analyzing and prescribing governance arrangements. 29 Experimentalist governance emphasizes a process for managing experimentation, rather than individual actions. That process includes four stages: 1. A central authority provisionally establishes a policy goal, alone or together with the local authorities that will be responsible for pursuing it. In the first 25 Adoption of the Paris Agreement, FCCC/CP/2015/L.9/Rev.1, 12 December Id. para , See, e.g., Jonathan Swift, Polite Conversation in Three Dialogues (1892, London: Charles Whittingham & Co.), at 130: He was a bold Man, that first eat an Oyster. Available at 29 Sabel & Zeitlin 2012; 2010; 2008; de Búrca, Keohane & Sabel 2013,

8 structure described above, the central authority might be an institution such as the Sustainable Development Summit or Paris CoP, which adopts a goal such as the SDGs or 2 ; the local authorities would be commitment-makers. In the second structure, an initiative such as SE4All might be the central authority, adopting specific goals on energy efficiency; the local authorities would be firms and other participants in SE4All. 2. The local authorities are given discretion to pursue the goal however they see fit (as voluntary commitment-makers are). Local authorities are motivated to take meaningful action, in this theory, by their desire to avoid the adverse outcomes of inaction, referred to as penalty defaults. 3. The local authorities must, however, periodically report on their performance and participate in peer reviews, in which they compare actions and results. Where the actions of some authorities are seen as more effective, the others are expected to modify their own efforts by adopting the more successful approaches. 4. The central authority reviews the goal and may revise it in light of the local authorities experiences, and the process begins again. Such a system would be a significant improvement over current arrangements, which rarely go beyond the first two stages. Most importantly, experimentalist governance is designed to be a machine for learning, primarily through the peer interactions in stage Existing voluntary commitment systems, in contrast, have adopted much weaker learning mechanisms, such as partnership fairs and online registries. To be sure, there have been hopeful developments: the Rio+20 outcome document and 2030 Agenda recognize the importance of learning; the HLPF platform for partnerships and the LPAA s encouragement of reporting hold promise. Some action networks also include learning mechanisms: SE4All, for example, sponsors an Energy Efficiency Hub, operated by an outside institution, which is developing a curated knowledge base and learning platform. 31 Overall, though, learning remains a weak link in voluntary commitment systems. Beyond learning, a properly managed experimentalist governance system could also help address other problems with voluntary non-state commitments. It could modestly increase accountability for voluntary commitments. Strong enforcement measures are likely to deter potential commitment-makers, but periodic performance reports and peer reviews would provide a modicum of accountability while generating less resistance, especially if they were framed as learning mechanisms. 30 Sabel and colleagues refer to this approach as directly deliberative polyarchy. Cohen & Sabel 1997; Dorf & Sabel

9 It could similarly encourage commitment-makers to ratchet up and extend their commitments over time; this is especially valuable where SMART criteria generate narrow, time-bound undertakings. Reporting and peer review would also be central here, especially if paired with a clearly enunciated expectation of continuous improvement. It might even help to elicit new commitments, by making an implied promise to commitment-makers that learning will increase their impact over time. The Sabel-Zeitlin form of experimentalist governance could be applied to voluntary commitment systems with relative ease. But it too fails to incorporate the lessons of experimentation in the natural and social sciences. Snidal and I argue, therefore, that architects of governance should go further, taking those lessons more seriously. We call our approach experimentalist governance Experimentalist governance 2.0 Experimentalist governance 2.0 has two major components: first, conducting designed, controlled policy experiments in the scientific sense; and second even where such experiments are infeasible managing diverse initiatives through orchestrated experimental systems. Both features would enhance learning from non-state voluntary commitments. A. Conducting controlled experiments Oddly, the literature on experimentalist governance does not call for actors to conduct real experiments in the scientific sense: local authorities are free to choose any approaches they wish to pursue the agreed goal, whether or not those approaches are experimental; the central authority has no responsibility to conduct experiments either. Public policy scholars, in contrast, have long seen important benefits from testing proposed social policies and interventions through designed, controlled experiments (e.g, Berk, Boruch, Chambers, Rossi and Witte 1985). And social scientists are embracing experimental methods enthusiastically, thanks in part to the new opportunities presented by the Internet (Druckman et al. 2011). In addition, public authorities in the United States (Greenberg & Schroder 2004), the United Kingdom (e.g., the Behavioural Insights Team 32 ) and Europe 33 are actively conducting policy experiments. The World Bank and other development organizations, such as the UK Department for International Development, support policy experiments conducted in-house or by economists and other independent scholars and civil society groups (for examples, see Dupas & Robinson 2013; Viceisza 2008). 32 The Behavioural Insights Team, Update Report , at 5-6, available at 33 See Demos Helsinki, Design for Government: Human-centric Governance Through Experiments (Sept. 2015), available at 9

10 What would real experiments look like in the context of voluntary commitments? In the sciences, experiments are classified into three major categories: true, field, and natural experiments. True experiments are the standard for natural and social scientists. They require three elements: (1) a deliberate, controlled intervention by the experimenter (the treatment ); (2) a control group that receives no treatment or a different one; and (3) random assignment of subjects to the two groups. This randomized experimental design is intended to control for the influence of confounding factors causal factors other than those the experiment seeks to test. By controlling for confounding factors, the experimenter can appropriately infer that the intervention, not other influences, has caused the observed effects; this makes the experiment internally valid. The ability to demonstrate causal relationships is the most important benefit of true experiments. A laboratory setting maximizes control over the intervention and further isolates the experiment from confounders, strengthening causal inferences. True experiments are also transparent, allowing others to replicate the experiments (or attempt to), with different subjects and settings. Successful replication not only reinforces the original causal inferences, but also supports generalizing those results to other actors and circumstances; this makes the experiment externally valid. In the domain of voluntary commitments, physical laboratory experiments like those of physics and engineering would be appropriate for purposes such as the development of new technologies for energy efficiency or carbon capture and storage. But laboratory experiments are also used in psychology (Morawski 1988), economics (especially behavioral economics; Thaler 2015), political science (Druckman et al. 2011) and other social sciences, even in law (Chilton & Tingley 2013). True experiments, moreover, need not be physically conducted in a laboratory. Survey experiments, for example, can approximate the conditions of true experiments; they have become very popular, not only for studying public opinion (Sniderman 2011), but also for testing the impact of policy measures (e.g., Findley, Nielson & Sharman 2014). Experiments informed by economics might, for example, test the risk preferences of different classes of subjects; their preferences for different mitigation or adaptation measures (e.g., carbon taxes versus cap-and-trade); and their willingness to pay for different policy interventions (cf. Viceisza 2008). Some voluntary initiatives might serve as control groups and others as treatment groups; or particular initiatives might apply interventions to distinct treatment and control groups within the population. Experiments informed by political science might compare the impacts of private standards or legal enactments adopted by different 10

11 institutions or formulated and implemented in different ways. They might compare the effects of such norms on public opinion (Chilton 2014; Tingley & Tomz 2012); their uptake by business firms (in the case of voluntary standards); their levels of compliance; or public willingness to retaliate against violators (Tingley & Tomz 2014). Field experiments are randomized, controlled studies conducted in real-world settings; most policy experiments are field experiments. In the World Banksupported study cited above, for example, investigators randomly assigned owners of small informal businesses in Kenya to a treatment group that received access to bank accounts and a control group that did not, and then compared the behavior of different subjects (Dupas & Robinson 2013). 34 The downside of field experiments is that experimenters typically have to give up some degree of control; confounding factors may intervene. In addition, approval from governments or other actors may be needed, and there may be political, financial or even ethical barriers. The government of Finland has initiated a program of field experiments to assess alternative tax and regulatory policies. The program developers identify issues relating to health, wellbeing and education, along with consumer decisions on transportation and energy consumption, as especially appropriate for experimentation; all are relevant to sustainability. 35 In addition, field experiments are often used to test different development policies, as in the Kenya example above. An experimenter might, for example, randomly assign some local residents to a treatment group that receives cash benefits for increasing energy efficiency; others to a treatment group that receives non-cash benefits; and still others to a control group with no intervention (cf. de la O & Wantchekon 2011: 385). Field experiments like these would be especially appropriate and productive in the voluntary commitments context, as subjects would realize any benefits produced by the initiatives. Natural experiments are situations in which some phenomenon or some other actor not the experimenter has randomly (or almost randomly) assigned subjects to different groups, which then encounter different interventions. Examples include lottery systems for the military draft, birthdates that determine when children can enroll in school, and arbitrarily drawn boundaries. 36 The experimenter must, however, find situations that not only satisfy the criterion of 34 The study found, among other things, that female subjects in the treatment group used the accounts actively and increased their savings and business investment, in spite of substantial fees, while the male subjects did not. This suggests the presence of significant barriers to financial services for women. 35 Demos Helsinki The lottery is random by design; in the other two cases, there is no reason to expect children born a day apart, or people living in neighboring villages, to differ systematically. 11

12 randomness (or near randomness), but that involve interventions relevant to the questions of interest. These situations can sometimes be identified: for example, a government may randomly assign individuals or communities to different groups (de la O & Wantchekon 2011); a civil society organization may change the benefits it offers blood donors from cash to non-cash compensation (Niessen-Ruenzi et al. 2016). In such cases, natural experiments can provide valuable insights. Given their limitations, however, natural experiments are unlikely to be central to an experimental system. B. An orchestrated experimental system With proper encouragement and support, a significant number of voluntary non-state commitments could clearly conduct or be structured as true, field or natural experiments. But this is unlikely to be true of all commitments, for practical reasons or because the participants are unwilling to proceed in that fashion. Such commitments would then be experiments only in the informal sense. In both cases, however, the learning benefits of voluntary commitments can be enhanced by applying a general lesson of experimentation in the natural and social sciences: a program of experiments must be carried out within an organized experimental system, supervised and coordinated by a qualified, active manager. This is especially important when multiple experimenters are involved, as in medical clinical trials and in systems of voluntary commitments. Active coordination is important because decentralized experimenters (here, for example, companies, cities or other commitment-makers and initiatives) face incentives that militate against learning. For example: Individual experimenters have a natural incentive (often reinforced by SMART criteria) to conduct the types of experiments (formal or informal) that appear most likely to succeed or that are the most politically appealing to members, funders or other important audiences. 37 This may produce duplication of effort, with many initiatives taking similar actions. Even more important, it may lead initiatives to avoid approaches that would in fact prove to be superior, but that initially appear less promising. Both effects are antithetical to learning. The early stages of an experimental program, in particular, are opportunities for discovery; they should involve a wide-ranging search for new ideas. It is unlikely, however, that uncoordinated actors will conduct such a search on their own. Similarly, experimenters have a strong incentive to make their actions appear successful, because of the anticipated audience costs of failure. This may lead them to cover up failures, misrepresent results, and drop seemingly unpromising 37 The latter incentive can shade into corruption, as where experimenters select actions that provide material or political benefits to related actors. 12

13 experiments before they are completed. These effects too are antithetical to learning, which is often the product of failure (Firestein 2015). In some cases, experimenters may not wish to share their results with others, so they can retain useful findings for their own benefit. This incentive is especially strong where relevant audiences view actors as being in competition; this may be the case in the reputational markets that influence many local governments, businesses and other commitment-makers. This effect is antithetical to the diffusion of learning. In voluntary commitment systems, moreover, some commitment-makers are likely to be insincere; their interest is in greenwashing, not in effective interventions or learning. 38 The managers of experimental systems need real influence (and resources) to identify the effects of these incentives and to work against them. Yet the managers of transnational voluntary commitment systems are unlikely to be granted strong hierarchical powers. 39 They must therefore adopt the governance strategy of orchestration, relying on persuasion, support, disclosure and other soft inducements, rather than hard controls. Even an orchestrator, however, can contribute to the success of a formal or informal experimental program throughout the experimental cycle. Early stage: In the early stages of an experimental program, an orchestrator can promote participation in the program (making experimentally-oriented commitments, or commitments in general), as well as an experimental mindset, encouraging actors to design and carry out their commitments with an eye to experimentation and learning. To enhance the search for new ideas, the orchestrator should promote participation by a wide range of actors, not merely those that are well-known or have been involved in the past: new actors are more likely to contribute new ideas and to interact with new groups of subjects. To overcome the incentive to cluster around approaches viewed as likely to succeed, an orchestrator can promote varied experiments, which incorporate untested approaches as well as a range of known or hypothesized interventions. A manager with hierarchical authority could assign experimental approaches to different actors, ensuring variation and avoiding clustering. An orchestrator lacks this authority, but can encourage variation, and can incentivize it by tying future support to experimental variation. Where commitment-makers or initiatives pursue designed, controlled experiments, an orchestrator can guide them toward good experimental designs that maximize internal and external validity. It can also exercise some degree of supervision to ensure that 38 Thanks to Ben Cashore for emphasizing these incentives and suggesting some possible responses discussed below. 39 I discuss below the organizations that might play the managerial role. 13

14 actors carry out those designs in practice. Where multiple (formal or informal) experimenters are involved, an orchestrator will need to coordinate their actions for complementarity and comparability. As noted above, if experimenters cluster around a small number of approaches, the experimental program will include gaps: policy alternatives and theoretically informed hypotheses that go untested. Such gaps make it difficult to draw accurate conclusions, and increase the risk that valuable approaches will be missed. If actors design, implement, record, measure or report the results of their experiments in different ways, moreover, experimental data will be non-comparable, causing the machine for learning to break down. To discourage sham actions by insincere participants, the orchestrator can set ambitious goals, communicate expectations for meaningful action and discourage weak commitments. It can also disseminate information about commitments (through the existing registries or in other ways), facilitating monitoring and benchmarking by civil society, experts and other audiences. However, these are particularly difficult issues. A broad goal such as 2 C provides limited discipline on commitment-makers, while a highly specific goal may reduce variation. Criteria that discourage weak commitments, such as SMART, may produce excessively narrow actions. And adding elements of enforcement to an experimental program may discourage participation. Mid-cycle: While voluntary commitments are intended to draw on the capabilities of non-state actors, their impact can often be increased through modest forms of support (Abbott et al. 2015). Financial and other forms of material support are always welcome, but ideational support such as information, guidance, coordination and recognition can also be beneficial. Material and ideational support help make large-scale experiments feasible; encourage experimenters to try approaches that are costly or appear unlikely to succeed; and reduce the chance that potentially valuable approaches will be dropped, or appear to fail, only because of a lack of resources. Support is also one of the strongest incentives that a non-hierarchical orchestrator can deploy. Late stage: The late stages of an experimental cycle are where learning takes place, through analysis and comparison of experimental results. But learning too must be orchestrated. An orchestrator s first task is to ensure the quality of the data that experimenters provide, in part because of their incentives to massage data to make it appear successful and to withhold data that might provide competitive advantages. The orchestrator must then coordinate the analysis of that information. Pure peer review, the preferred technique of experimentalist governance theory, has appealing political qualities, but will often be insufficient for a proper analysis of experimental results. 14

15 The orchestrator must also disseminate experimental results, among the experimenters and beyond. Dissemination is crucial to learning. It can also produce other benefits. As noted above, dissemination facilitates benchmarking, helping to reveal which actors have undertaken serious actions and which have not. It can generate new political coalitions around actions shown to be beneficial. And it may help modify the preferences of reluctant actors, by identifying lower-cost approaches and showing the concrete and reputational benefits of environmental actions. An orchestrator must, however, carefully examine how actors have designed and carried out their experiments, and how they have measured and reported their results. It may be necessary to conduct independent analyses of reported data. To identify the causal mechanisms at work in particular experiments, it may be desirable to develop case studies that trace the processes involved. The orchestrator may need to commission experts to carry out these tasks. Frequently, moreover, the results of a set of experiments are not complete or conclusive; the orchestrator must then determine whether further experiments are desirable, how they should be designed, and who should carry them out. For all these reasons, an orchestrator should have some degree of independent authority and analytical capacity, rather than relying wholly on the experimenters. 4. Who can orchestrate non-state commitments? Many types of actors, governmental and non-governmental, could in theory carry out the functions of an experimental orchestrator, so long as they possess certain essential attributes (Abbott & Hale 2015): Because an orchestrator must influence the behavior of experimenters or commitment-makers through soft means, its effectiveness will increase to the extent it possesses authority and/or legitimacy in the eyes of those actors. Actors may derive authority and legitimacy from their institutional position, past successes, expertise, moral qualities and other attributes. An orchestrator should have strong connections to the actors it will seek to influence. Many successful orchestrators have not been dominant within their domains, but strong network relationships facilitate orchestration. Because support is a major tool of influence, an orchestrator must possess (or be able to mobilize) sufficient material and ideational resources. An orchestrator must be willing and able to engage in the activities associated with orchestration. At the least, it must have sufficient agency the ability to act intentionally as a unit to promote, support and coordinate formal or informal experiments and to manage learning processes. An entrepreneurial organizational culture (rather than a bureaucratic one) may also facilitate orchestration. To identify potential orchestrators, consider again the two potential ways to structure voluntary commitments as experiments. In the first approach, individual commitments, 15

16 partnerships and initiatives are themselves treated as experiments. Here, the institution that sponsors the relevant voluntary commitment system, or others acting on its behalf, are the natural orchestrators. Many of these institutions possess strong authority and legitimacy as well as ideational resources. Equally, however, many seem poorly suited institutionally to the demands of orchestration, much as the CSD was. This is especially true of conferences such as Rio+20 and the climate CoPs. While these institutions may sponsor voluntary commitment systems, they lack effective agency even during their sessions, and cease operation once adjourned. The secretariats and other agencies that manage these conferences have much greater agency. The office of the UN Secretary-General has had substantial success, both at the New York Climate Summit in 2014 and in organizing Rio+20 commitments into action networks. It has strong legitimacy through its institutional position and moral standing, and a close connection to the UN Global Compact, responsible for many business commitments. UN-DESA has also helped to organize action networks, maintains the registry of multi-stakeholder partnerships, and reports regularly on partnerships and commitments. The main difficulty for both organizations is resources; the UN Secretariat is generally under-financed, making it difficult to effectively deploy even ideational resources. The HLPF has significant strengths, including authority and legitimacy through its universal membership, and unparalleled convening power (Abbott & Bernstein 2015). It is, however, too large to orchestrate nimbly, and it meets only periodically. In climate, the Peruvian and French presidencies have orchestrated since 2014, in collaboration with the Office of the Secretary-General and UNFCCC Secretariat, with significant results. The intention is to continue this arrangement, with the past and current presidencies leading the LPAA. The continued success of this arrangement depends, however, on the attitudes and capabilities of future presidencies. The UNFCCC Secretariat is a natural climate orchestrator, but is constrained by a narrow legal mandate 40 and limited experience. The UN Environment Program (UNEP) has long been an important orchestrator in te environmental domain (Abbott & Snidal 2010); its activities originally inspired the concept of orchestration. Acting primarily through its Division of Technology, Industry and Economics in Paris, UNEP has helped to initiate and continues to support prominent business and public-private initiatives on environmental issues, including climate. These include the Global Reporting Initiative, UNEP Finance Initiative, Principles for Responsible Investment, Principles for Responsible Insurance, UN Global Compact (and its offshoots on climate, water and women s empowerment), and Sustainable Business and Climate Initiative, among others. UNEP possesses strong legitimacy through its position, expertise and track record; unusually strong connections in the private sector 40 UNFCCC Art. 8 provides only for narrow administrative functions. 16

17 and civil society through its past activities; and effective agency. Its mandate also focuses on coordination. While UNEP has not been central to the voluntary commitment systems in sustainability or climate, it could be an effective orchestrator in the future. Consider now the second way of structuring commitments as experiments, formal or informal. Here, an action network, multi-stakeholder partnership, collaborative initiative or similar organization makes a commitment (or interprets an existing commitment) to sponsor and manage formal or formal experiments by its members and other actors willing to participate. Here, the initiative itself would act as orchestrator. Consider, as one of many possible examples, United for Efficiency (U4E), an LPAA cooperative initiative, which aims to double the energy efficiency of electric appliances by 2030, in line with the goal of SE4All. 41 U4E is a public-private partnership. It is led by UNEP, and includes the UN Development Program and Global Environmental Facility (GEF), the International Copper Association and other private sector partners, and the Natural Resources Defense Council (NRDC). U4E builds on the work of the en.lighten partnership on efficient lighting, also led by UNEP and the GEF. It aims to help developing countries leapfrog inefficient appliances, moving directly to efficient products as they develop. Its strategies include promoting policies that encourage/require energy efficiency; promoting mechanisms to help consumers make the transition; developing monitoring and enforcement strategies; and promoting life-cycle management. In many of these areas, much is known about what works, and UNEP plans to promote best practices based on that knowledge. Yet virtually all of these areas would benefit from greater knowledge, especially about what works in developing economies with limited government capacity, poor communications, weak educational systems and the like; and about the best ways to promote and facilitate the political and economic strategy of leapfrogging, which could have many broader applications. U4E could contribute to these forms of knowledge by sponsoring and managing formal and informal policy experiments conducted by participating governments, private sector firms, civil society groups and social scientists and by constantly updating best practices as it learns. Could U4E be an effective orchestrator? UNEP s lead role brings its considerable strengths to the organization; UNDP and GEF bring similar strengths (Graham & Thompson 2015), although less orchestration experience. U4E itself possesses authority and legitimacy, stemming from its tripartite, high-level membership, LPAA role, expertise, and experience with en.lighten. It possesses both material resources (provided primarily by the GEF) and ideational resources. Its members have wide connections in both the North and the South

18 As a multi-stakeholder partnership, U4E faces some challenges of agency, but UNEP leadership should help address this problem. U4E was also selected by SE4All to manage the appliances program of the Global Energy Efficiency Accelerator Platform. 42 In that capacity it will develop policy and investment roadmaps for developing countries, and help implement them through multi-stakeholder networks. These activities should expand U4E s authority, connections and experience, and force it to organize in ways that enhance its agency. They will, in fact, require U4E to act as an orchestrator, and the skills it gains there could easily be transferred to promoting experimentation. Much of what has been said about U4E is equally applicable to other action networks, partnerships and cooperative initiatives. Many bring similar strengths including the participation of international organizations and have at least as many opportunities for experimentation. Some have also adopted mandates or goals for which experimentation is highly relevant. For example, Every Woman Every Child (EWEC) has established an Innovation Working Group made up of development innovation organizations; part of its mission is to strengthen the pipeline of new innovations. 43 The group s strategy does not mention experimentation, but it could and should. SE4All has committed itself to promote innovative solutions as well as known best practices. 44 The Higher Education Sustainability Initiative, another action network, has adopted missions to promote research on sustainability, develop green campus programs, and share information with international networks; 45 it could become a hub of experimentation. Other organizations have also taken active roles in promoting and managing voluntary commitments, and could function in the same ways as action networks, multi-stakeholder partnerships and cooperative initiatives. For example, organizations of subnational governments such as ICLEI Local Governments for Sustainability, the Covenant of Mayors, the Compact of Mayors and the Compact of States and Regions have elicited and managed many commitments by local governments. There is, if anything, an over-abundance of potential orchestrators in the voluntary commitments space. One final innovation, then, would be valuable: empowering an existing body, or creating a new one, to orchestrate the orchestrators. Such a body could encourage action networks, multi-stakeholder partnerships, cooperative initiatives and other organizations to initiate programs of experimentation; provide guidance and support to them; and engage in all the other functions of an experimental orchestrator, including coordinating experimentation across networks. While agency could be an issue for such a meta-orchestrator (Beisheim & Simon 2015), an appealing approach would be to establish a multi-stakeholder body or one for each issue area or sector that brings together international organizations including UNEP, the UN Secretariat and/or the