Outcome Evaluation of the UNDP Project. Building the Post-2015 Development Agenda: Open and Inclusive Consultations

Transcription

1 Outcome Evaluation of the UNDP Project Building the Post-2015 Development Agenda: Open and Inclusive Consultations

2 Authors: Mike Zuijderduijn, Francine Egberts, Ella Krämer Published by MDF Training & Consultancy BV Bennekomseweg 41, 6717 LL Ede, The Netherlands telephone: website: ISO 9001:2008 certified Copyright 2016 Design by Sebastian Villar

3 Outcome Evaluation of the UNDP Project Building the Post-2015 Development Agenda: Open and Inclusive Consultations Ede, the Netherlands, April, 2016

4 Contents Executive Summary 1 1. Introduction Delineation of the Evaluation Structure Evaluation Report Project Description Methodological Approach Methodological Choices and Nature of the Evaluation Evaluation Standards and Principles Evaluation process Observations Concerning the Evaluation Process Findings and Analysis Introduction Effectiveness Effectiveness of the global consultation processes Effectiveness of national consultation processes Relevance and Inclusiveness Relevance in light of other global processes Inclusiveness at the global level Inclusiveness at the national level Positioning of the Project Positioning of project in the global process Positioning of project in national processes Innovation 39

5 4.6 Analytical Findings of What Went Well and What Could Be Improved Concerning project design Concerning process management Concerning content / Results of the consultations Conclusions Effectiveness Relevance and Inclusiveness Positioning of UNDP in the Process Innovation Overall Conclusion Recommendations and Dilemmas Concerning the design of future consultative processes Concerning the management and implementation of future consultative processes 59 Annexes 63 Annex 1. References 63 Annex 2. Informants 71 Annex 3. Visual Overview Evolution of the Project 82 Annex 4. Description of project Outputs 84 Annex 5. Terms of Reference 87 Annex 6. Overview of Consultations 92 Annex 7. Evaluation matrix 95 Annex 8. sense-making event 98

6 Abbreviations ASG CSO DESA EOSG HLP HQ IGN INGO LGBT MDG MoI NY OECD OWG PAC P15A QCPR SDG SDSN SG TOR TST UNCT UNDAF UNDG UNDP UNGC UNRC UNRCO UNTT WWW Assistant Secretary-General Civil Society Organization Department for Economic and Social Affairs Executive Office of the Secretary-General High Level Panel Headquarters Inter Governmental Negotiations International non-governmental organization Lesbian, Gay, Bisexual, Transgender Millennium Development Goals Means of Implementation New York Organisation for Economic Cooperation and Development Open Working Group Project Appraisal Committee Post-2015 Agenda Quadrennial Comprehensive Policy Review Sustainable Development Goals Sustainable Development Solutions Network Secretary-General Terms of Reference Technical Support Team United Nations Country Team United Nations Development Assistance Framework United Nations Development Group United Nations Development Programme United Nations Global Compact United Nations Resident Coordinator United Nations Resident Coordinator s Office United Nations Task Team World We Want

7 Executive Summary With the world entering a new millennium, 189 Member States adopted the Millennium Declaration in September 2000, containing the values, principles and objectives for international development in the 21st century. Building on various global summits and international conferences of the 1990s, the Millennium Development Goals (MDGs) comprised eight global goals. The MDGs provided important development objectives that were seen by many as a powerful tool for global action. However, apart from significant achievements made on various targets, the MDGs were also critiqued. Because the goals were not subject to intergovernmental agreement, they lacked ownership by Member States and many development actors. It also took a while for the MDGs to gain momentum. In terms of content the critique focused on reducing the Declaration's overarching principles into relatively simplistic targets focusing too narrowly on the social and human dimensions of poverty at the expense of development in its broader sense and not addressing the root causes of the overarching challenges to development faced worldwide. Despite the critique, it was felt that an overarching global framework would continue to be of essence for the post-2015 period. During the 2010 MDG Summit, Member States requested the UN Secretary- General to initiate a process towards shaping a post-2015 development agenda, which would have to take into account the lessons learned and limitations of the MDGs. It would have to be broader, more ambitious and covering the global challenges of the 21st century and be actionable at country level, while taking into account the complexity of integrated development challenges, and most important, be owned by Member States from the very beginning. In his 2011 MDG progress report, the Secretary-General stated that the post-2015 development framework was likely to have the strongest impact if it emerges from an inclusive, open and transparent consultation process. In response to this, several work streams were initiated, including the establishment of the UN System Task Team to coordinate system-wide preparations for a Post Development Agenda (P15A), and a High Level Panel of Eminent Persons to advise on the global development framework. The United Nations Development Programme (UNDP), working with agencies in the United Nations Development Group (UNDG), initiated preparations to realize the Secretary-General s vision to make the discussions of the post-2015 agenda open, inclusive and in line with the UN s principles and values. On a parallel track, UN Member States convened in 2012 for the Rio+20 Conference, where the request to develop an open and broad consultation process with technical support from the UN system formed part of the outcome document. This outcome document also tasked the General Assembly to set up an Open Working Group (OWG) to define and to establish an inclusive and transparent intergovernmental process on the Sustainable Development Goals (SDGs). It is in this context that in 2011 the project Building the Post-2015 Development Agenda: Open and Inclusive Consultations was conceptualized and launched. The overall objective was to build and lead a strategic coalition of partners that can shape the post-2015 development agenda through 1

8 Building the Post-2015 Development Agenda - Open and Inclusive Consultations global broad mobilization and engagement of government, grassroots, academics, CSOs and other organizations and to ensure that the post-2015 development agenda is (1) guided by UN norms, values and commitments, (2) informed and shaped by the arising challenges of the 21st century, including sustainability and equity and (3) built on the momentum and lessons learned from the MDGs. 1 At the end of the project, almost 100 national consultations, 11 global thematic consultations, 6 global consultations on implementation and pilot exercises with illustrative goals in 10 countries had taken place. In addition, the project initiated the MY World survey with almost 10 million respondents and organized numerous (side-)events and advocacy efforts to communicate the results of the consultations and feed into the OWG. A large part of the project focused on building a bridge between the people who normally do not participate in this type of agenda setting and the institutional mechanisms in place responsible for a global process like the formulation of the SDGs. Although it was administered and hosted by UNDP, the project was led by UNDG, to allow for a coherent and coordinated approach in which several UN agencies could work together on the development of the P15A. In recognition of its complex context, the project was designed to be flexible, without pinning down an explicit Theory of Change or detailed results framework, so it could evolve organically and adapt to changes in the policy context by adjusting the project s outputs and activities. In this way it was expected that the shaping of the post-2015 agenda could benefit optimally from a global inclusive consultation process, making the Sustainable Development Goals relevant to national contexts and allowing for an unprecedented diversity of stakeholders to contribute to the post-2015 agenda. By opening up the global policy making space to virtually everyone, this consultation and advocacy process consumed a significant amount of time and resources within and beyond the UN system. Given the unprecedented scale and unique nature of this project - that has now come to an end - a learning-oriented evaluation exercise was commissioned with two main objectives: 1. Assess the significance of this global consultation process for the shaping of the Sustainable Development Goals and the wider P15A in all its qualitative dimensions both at global and country level; 2. Draw lessons from this global consultation process for ongoing / future policy and programme development within UNDP and the UN system at large. In pursuit of these objectives, the evaluation started by mapping out how the global consultative processes had played out in reality. This led to the distinction of three complementary tracks of influence : (1) national processes focusing on national consultations and creating buy-in of multiple stakeholders among which the Member States were considered crucial; (2) UN-driven processes which to a large extent were meant to feed the Secretary-General s guidance to the process; and (3) supporting the intergovernmental process by informing and supporting the Open Working Group. 1 UNDP Project Document Building the Post-2015 Development Agenda: Open and Inclusive Global Consultations (2011). 2

9 Executive Summary On this basis, the evaluation investigated the following key evaluation criteria / questions; Effectiveness of the project interpreted as the extent to which the project achieved its ambitions in contributing to the 2030 Agenda by: (1) increased understanding and ownership among Member States and the people at large; (2) creating a platform to inform the formulation of goals and targets; and (3) the creation of new sustainable partnerships; Relevance in terms of (1) the uniqueness / added value of the project; and (2) inclusiveness (i.e. the extent to which the different stakeholders, notably vulnerable groups, had their voices heard in the development of the P15A); Positioning in terms of the degree and type of leadership provided by the project, including its contribution towards more unified UN support in shaping the P15A; Innovation related to the extent new methods and tools for global policy making have been introduced and the added value of those methods and tools in building the P15A. Findings in response to these evaluation questions were collected through a mixed-method approach, which included document analysis, web research, online surveys among stakeholders in New York and at country level, interviews with key informant in New York and Geneva, and visits to seven countries in which national consultations took place. The interviews were done face-to-face or through Skype with representatives from Member States, UN system, civil society organizations (CSOs), and academic institutes and findings were disaggregated per stakeholder group. The data collection process was concluded by a sense-making workshop attended by multiple stakeholders in New York in December In light of the findings of the evaluation, the following conclusions concerning the key evaluation criteria were drawn. Effectiveness In terms of increasing understanding and ownership, notable achievements were made, particularly when compared to the MDGs. The project has been credited for opening up the global policy making process, paving the way for Member States, civil society and other stakeholders to take part in shaping the SDGs. The 100 national consultations and the MY World survey in particular provided unprecedented opportunities, grasped by people all over the world, to contribute to the 2030 Agenda. Member States and multiple stakeholders took an active interest and made contributions through networks, organizations or as individuals. At the same time, Member States had started preparing for the integration of the 2030 Agenda into their own national policy framework through various processes at national level. The timeliness and the unprecedented scale and openness of the global consultations, which gave legitimacy to the wide variety of inputs, have been important contributing factors in this. At the same time, some challenges were noted in the synergy between national UN-initiated consultations and existing national processes, the involvement of sub-national government (although addressed in the second round of consultations), and the links between the delegations in New York that took part in the intergovernmental negotiations and their government counterparts at country level. 3

10 Building the Post-2015 Development Agenda - Open and Inclusive Consultations Effectiveness in creating a platform to inform the content of the 2030 Agenda was assessed primarily by looking at the project s effect through the three intermediary tracks of influence. In this process, the effectiveness on the OWG (i.e. the intergovernmental track) was the most direct and the strongest, mainly due to the delivery and promotion of issue briefs formulated by many of theun staff playing a key role in the thematic consultations. The effect through national processes has been less powerful at the global level but more visible in the way consultation results were picked up in national policy development processes. The effect on the content of the High Level Panel (HLP) on the Post-2015 Development Agenda has been modest, mostly due to its timing. Beyond the project s sphere of control, effects of the consultative processes can be seen in the way the 2030 Agenda has taken shape. The framing of the consultations contributed to the comprehensiveness of the Agenda and to the inclusion of goal 10 (reducing inequality) and 16 (promote peaceful and inclusive societies). Besides, the consultation can be partially credited for the inclusion of various qualitative targets and non-financial systemic implementation issues. Scope for improvement was found in providing more clarity to country teams about the intended use of national consultations in influencing global and national policy, the recognition of countrylevel differences to optimize the use of UN resources and the potential contribution of the national consultations to the thematic ones. Effectiveness in creating new partnerships was particularly successful within the UN system and with civil society, although the sustainability of these partnerships remains a challenge. The project was a positive example of interagency cooperation, facilitated by the UNDG leadership, the composition of the post-2015 team and by offering space to many UN agencies to lead thematic consultations or Dialogues on Implementation. Also, at country level the project demonstrated an inclusive and unified UN approach. In addition, the global consultation process contributed to a renewed collaboration between Member States and the UN development system primarily stimulated by the co-hosting principle. However, broad and substantial engagement of the private sector entities proved to be difficult. Relevance The project was a relevant and important addition to other consultative processes by ensuring a timely, adequate and unified UN response to the request for support from Member States to inform the P15A formulation process based on an open and inclusive consultation process. The uniqueness of the project was most obvious in the scale and diversity of inputs it mobilized though added value as well by offering a channel for inputs through which the content of the 2030 Agenda could be, and was, influenced. The project successfully linked up with emerging institutional arrangements and in particular has been convincing in demonstrating the UN s convening power in a multilateral policy debate. In terms of inclusiveness, the project has come far in realizing its ambitions of unprecedented outreach. This was achieved by making use of a wide variety of participation modalities, providing space to a large diversity of networks, organizations and individuals representing many (often vulnerable) groups that are normally not part of such processes. At the same time, it has to be acknowledged that certain groups continue to remain out of the picture due to practical or political limitations, and it was a challenge to ascertain whether participants were representative of their groups. The only dimension of inclusiveness on which the projects has not performed very well is getting feedback on the use of the consultation results, in particular off-line at country level. 4

11 Executive Summary Positioning of UNDP The positioning of UNDP in terms of providing leadership by hosting the project was in particular appreciated in the conceptualization, resource mobilization and implementation arrangements of the project that enabled a joint UN effort. In addition, the project showed leadership in its power to convene and synthesize the consultation results, even though this was (naturally) met with some controversy. UNDP deliberately concentrated on its convening power while keeping a lower profile on content, by providing space for other UN agencies to lead the thematic consultations. At the country level, leadership was taken not by UNDP, but the UN Resident Coordinator s Office, which facilitated the inclusiveness of the project within and beyond the UN system. The delicate choice of positioning yourself as leader on process or content is illustrated by comments from agencies: some remarked on UNDP s limited presence in providing subject matter inputs, while others pointed out UNDP s dominance in the governance consultations (one of the thematic areas in which UNDP shared the lead on content). The majority of respondents, however, expressed appreciation for the position UNDP took in this project, which leads to the conclusion that overall UNDP seem to have succeeded in performing this balancing act. A different example of the project s leadership was found in the fact that the project inspired a number of countries from the Organisation for Economic Co-operation and Development (OECD) to organize self-financed consultations under the project s umbrella. Innovation The innovativeness of the project is rated as high in terms of its unprecedented scale, ambition and transparency of results. It stimulated innovation at the country level through the guidelines for national consultations and the support of the post-2015 team, and at the global level by rolling out the MY World survey; the largest online survey ever. The thematic consultations and the Dialogues on Implementation in comparison were less innovative, which does not diminish their relevance and effectiveness given that innovation is not an end in itself. The most innovative element of these global consultations can be found in the way they were framed around new universal issues like: Inequality, Governance and Population Dynamics, and around non-financial means of implementation issues. Overall conclusion Looking at the extent to which the project met its general ambitions, the overall conclusion has to be positive. First the project did mobilize and engage a broad coalition of stakeholders of unprecedented scale and diversity to contribute to the shaping of a post-2015 agenda. Second, the project s inclusive approach did contribute to increased understanding and ownership among Member States, which was one of the most important lessons from the MDGs. Third and finally, the project did enable the provision of inputs that influenced a 2030 Agenda that is of universal relevance and does justice to the complexity of integrated development and sustainable poverty eradication. In other words, the project seems to confirm the hypothesis that open and inclusive engagement helps in getting universal agreement on an agenda of global relevance, that is widely understood, owned by several stakeholders and ambitious. This achievement can be recognized as a remarkable first step; however, now an even bigger challenge will be sustaining this broad coalition to help translate the 2030 Agenda into actionable plans at the country level and to take charge of its implementation. The project has set a new standard for global consultation processes in the future, and many lessons can be learned from it. Care has to be taken that such multilateral processes in future will be practised by keeping what went well and adapting what can be improved. To facilitate this learning process, the evaluation identified a number of specific lessons related to project design, process management and the results of the consultations (see table below). 5

12 Building the Post-2015 Development Agenda - Open and Inclusive Consultations Practices to be kept (what went well) Opportunities for improvement Project design Seize the opportunity to make open and inclusive global consultations happen in time to be of use to the OWG Follow the principle of co-hosting and coleading an interagency post-2015 team Allow and use space to adapt project along the way Provide guidance to national consultations Ensure that thematic and national consultations feed into each other Synergize with existing national consultative mechanisms Prioritize resources where UN added value is highest Process management Stimulate innovative approaches in seeking participation Ensure a large diversity of civil society involvement Ensure transparency of consultation results Recognize like-mindedness and identify allies Be prominent in consultations, but step back during intergovernmental negotiations Provide guidance on thematic consultations and Dialogues on Implementation Ensure uniformity in interpreting the purpose of national consultations Link national consultations to the global policy debate Ensure a feedback mechanism on the (use of ) results of the consultation process Results of consultations in shaping the 2030 Agenda Frame the global debate to reflect the demand for a relevant and universal agenda. Perceived legitimacy of consultation results. Put lessons learned from the MDGs into practice. Comprehensive, widely owned human rights based global agenda Influence the content position of middle / higher income countries Influence the opinions of experts Create a basis for a commonly understood and actionable agenda at country level Given the enormous complexity of the context in which the project pursued its ambitions, the evaluation would like to be modest in its recommendations while at the same time providing some guidance for future multilateral policy processes. In recognition of this complexity, some of the lessons are translated into concrete recommendations while others are presented as dilemmas as they have no apparent, easy or straightforward solutions but are likely to be faced again in future multilateral policy processes. Recommendations and related dilemmas concerning the design of future consultative processes include: 6

13 Executive Summary 1. Look, think and act ahead to secure the necessary time and an ambitious well-resourced plan of action to provide high-quality timely inputs that can be meaningful in future multilateral policy processes. Related dilemma: How to ensure the right balance between open and framed consultations will be a recurring challenge in shaping such plans. 2. Design flexibly with a matching resource facility to allow for adaptations to a changing policy context. This requires the creation of funding facilities and project design and participatory monitoring methods that are more outcome and less output oriented. 3. Guide the UN system and Member States in delivering as one. The arrangement of co-hosting countries and co-leading agencies with an interagency implementation team has proven to be successful. This decentralized set up requires strong overall leadership providing direction on role distribution, clarifying expectations and setting performance standards, while recognizing the difference between content and process leadership. Related dilemma: What is the optimal added value / role of the UN development system (technical advisor, convener and / or guardian of UN norms and values) and / or how to find the best balance between those roles? 4. Pursue complementary consultations while retaining diverse methods and modalities to maximize outreach and diversity of inputs. In doing so, consider the way these consultations can enrich and reinforce each other to become complementary rather than additional to each other. 5. Synergize where possible with existing national processes to optimize the added value of UN efforts in organizing consultations at country level. In this way, perceived overlap of consultation efforts can be avoided, while maximizing the sense of national ownership and therefore the chance that consultation results will be taken seriously in national policy processes. 6. Off-line data collection demands off-line feedback: this is to be considered as integral part of the consultative process. Transparency and accessibility of consultation results have to be equal for all who participated; they are crucial factors in motivating people to engage in future consultations and to contribute to implementation. Related dilemmas: How to best manage expectations about the use of inputs, and retaining the diversity of consultation results, given the need for synthesis and the response time for action on those inputs, especially those concerning urgent needs? Another dilemma in this context relates to how meaningful partnerships can be sustained, bridging the significant time gap between consultation and implementation, and knowing that this takes scarce time and resources without immediate obvious results. Recommendations and dilemmas concerning the management and implementation of future multilateral consultations include: 7. Balance between influencing policies and staying out of politics. Clear guidance in this regard is difficult to give, but it is important to continue this balancing act as a deliberate project management strategy, to ensure that conscious choices are made. Related to this, sharing experiences within the UNDG in manoeuvring between providing normative technical support and politics may help in strengthening the UN s institutional capacity to do so. 8. Stimulate innovation to reach unprecedented results especially in light of the ever-changing technological possibilities. This can be done by the project leadership demonstrating its own efforts to innovate, sharing and encouraging the use of innovative approaches and providing incentives for innovation in the roll-out of consultation processes. Innovation, however, needs to serve a clear purpose (e.g. reaching new levels of outreach) and not become an end it itself. 7

14 Building the Post-2015 Development Agenda - Open and Inclusive Consultations 9. Stimulate progressive integrity and quality of consultations evolving from collecting multiple inputs to advanced inclusive consultations in which vulnerable groups are empowered to contribute, experts are able and willing to listen and challenge their own truths while conveners are capable of neutral synthesis that do equal justice to all inputs. Related dilemma: How to create and maintain a balanced platform that allows for advanced consultations while ascertaining truly representative inclusiveness, given practical time and resource limitations? 10. Create an actionable integrated agenda at country level. Going beyond the scope and capacity of this evaluation, this is an obvious yet highly complex recommendation, challenged by existing institutional arrangements. Nevertheless a possible way forward may consider: a. developing a national Theory of Change, mapping out interrelations among SDGs at the country level as important step towards setting local priorities; b. (re)establishment of one dedicated interagency team under UNDG leadership to help ensure the UN continues to deliver as one in support of the implementation of the 2030 Agenda; c. engage in a dialogue with funders to establish more integrated and less earmarked financing; and d. establish a learning and communication system to continuously capture and promote best practices, while alerting against initiatives that stimulate fragmentation. 8

15 1. Introduction 1. Introduction 1.1 Rationale and Objectives of the Evaluation With the world entering a new millennium, 189 Member States adopted the Millennium Declaration in September 2000, containing the values, principles and objectives for international development in the 21st century. Building on various global summits and international conferences of the 1990s, the Millennium Development Goals (MDGs) comprised eight global goals. The MDGs provided important development objectives that were seen by many as a powerful tool for global action. However, apart from significant achievements made on various targets, the MDGs were also critiqued. Because the goals were not subject to intergovernmental agreement, they lacked ownership by Member States and many development actors. It also took a while for the MDGs to gain momentum. The United Nations Millennium Declaration was criticized for reducing its overarching principles into relatively simplistic targets with a narrow focus on the social and human dimensions of poverty at the expense of development in its broader sense, and without addressing the root causes of the overarching challenges to development faced worldwide. Despite the critique, it was felt that an overarching global framework would continue to be of essence for the post-2015 period. During the 2010 MDG Summit, Member States requested the UN Secretary- General to initiate a process towards shaping a post-2015 development agenda, which would have to take into account the lessons learned and limitations of the MDGs. It would have to be broader, more ambitious and covering the global challenges of the 21st century and be actionable at country level, while taking into account to the complexity of integrated development challenges and most important, be owned by Member States from the very beginning. In his 2011 MDG progress report, the Secretary-General stated that the post-2015 development framework was likely to have the strongest impact if it emerges from an inclusive, open and transparent consultation process. In response to this, several work streams were initiated, including the establishment of the UN System Task Team to coordinate system wide preparations for a Post UN Development Agenda (P15A) and a High Level Panel (HLP) of Eminent Persons to advise on the global development framework. The United Nations Development Programme (UNDP), working with agencies in the United Nations Development Group (UNDG), initiated preparations to realize the Secretary-General s vision to make the discussions of the post-2015 agenda open, inclusive and in line with the UN s principles and values. On a parallel track, UN Member States convened in 2012 for the Rio+20 Conference, where the request to develop an open and broad consultation process with technical support from the UN system formed part of the outcome document. This outcome document also tasked the General Assembly to set up an Open Working Group (OWG) to define and to establish an inclusive and transparent intergovernmental process on Sustainable Development Goals (SDGs). 9

16 Building the Post-2015 Development Agenda - Open and Inclusive Consultations Besides an investment of over US$ 22 million, 2 this consultation and advocacy process took a substantial of the time and energy of countless people within and beyond the UN system. Given the unprecedented scale and unique nature of this project that has now come to an end an evaluation exercise was commissioned with two main objectives: 1. Assess the significance of this global consultation process for the shaping of the Sustainable Development Goals and the wider P15A in all its qualitative dimensions both at global and country level; 2. Draw lessons from this global consultation process for ongoing / future policy and programme development within UNDP and the UN system at large. 1.2 Delineation of the Evaluation The evaluation focused primarily on assessing the significance of the consultative processes, deliverables and events that took place as part of the UNDP-administered project 3 Building the Post Development Agenda: Open and Inclusive Consultations, November 2011-December Numerous other efforts and initiatives took place to influence and shape the P15A at global, regional and country levels, many with substantial involvement of staff that were also part of the UNDP post team. However, these efforts have been considered as external to the project. Where relevant, they were used in the project s assessment as evidence of its causes or effects. The evaluation used the second project document revision (22 November 2013) with nine outputs as main reference document. In the evaluation, no attention is paid to output 6 (the set-up of the HLP secretariat) and 7 (support to the post-2015 team) as these outputs relate to the creation of practical organizational arrangements and not directly to the consultation or advocacy processes. 1.3 Structure Evaluation Report The evaluation report provides a factual project description (Chapter 2) and an explanation of the methodological approach (Chapter 3). Subsequently the findings of the evaluation are presented and structured according to the main evaluation criteria, including a set of analytical findings about what worked well and less well in designing and implementing the consultative process (Chapter 4). Finally the conclusions of the evaluation are presented in answer to the key evaluation questions (Chapter 5), followed by a set of recommendations and related dilemmas that are meant to guide future multilateral consultation processes. Note on terminology In the evaluation report the terms Sustainable Development Goals, P15A, post-2015 agenda and 2030 Agenda have been used interchangeably. 2 According to the budget of the second project document revision (virtual Project Appraisal Committee/ PAC date; 22 November 2013). 3 UNDP provided most of the financing for the project. 10

17 2. Project Description 2. Project Description 2.1 Background of the Project The Building the Post-2015 Development Agenda: Open and Inclusive Consultations project aimed to contribute to the formulation of an integrated set of widely owned and understood Sustainable Development Goals, which are of global relevance and actionable at country level. The project is based on the premise that the design and formulation of the post-2015 agenda would benefit from a global inclusive consultation process. Such an inclusive process would help making the Sustainable Development Goals more relevant to national contexts and at the same time pave the way for involvement of key stakeholders in the implementation of the Post-2015 Agenda. A large part of the project was focused on building a bridge between the people who normally do not participate in this type of agenda setting and the institutional mechanisms in place responsible for a global policymaking process, such as the formulation of the SDGs. Though administered and hosted by UNDP, the project was led by the UNDG, which allowed for a coherent and coordinated approach in which many UN agencies could work together in support of the development of the P15A. The project was designed in an open way to allow for the process to evolve organically and it left space to address changes in the context by adjusting the focus of the project and subsequent activities. Because a Theory of Change of the project was not explicitly put on paper at the start of project implementation, the evaluation team used the inception mission to create a de facto Theory of Change with the post-2015 team. This reality map is visualized below. The next section provides a description of how the project unfolded, including information on the underlying assumptions that have guided it. 11

18 Figure 2.1 Theory of Change - Reality Map of the project SDGs that are: widely understood actionable at coun Multilateral bodies (HLP/SDSN/SG) provide clear and convincing guidance to P 15A National Governments inform their missions based on consultative processes Multilateral bodies have access to high quality reliable data Peer pressure among Member States National Governments base their positions on consultative processes Pilots Illustrative goals Output 9 How consultations Output 8 National Consultations Output 1 Micro grants Output 1 Resources and infrastructure in pla (Building the Post-2015 Develo Sphere of Concern Sphere of Influence National Processes UN D Proce Sphere of Control

19 try level doing justice to reality address real needs owned by Member States/peolpe Heads of State formally adopt P15A OWG Proposal for SDGs (2014) Member States are able and willing to negotiate for key priorities in P 15A Intergovernmental Negotiations take note of outputs of consultative processes Open working groupg provides direction to P15A UNTST - 29 issue briefs Lobby & advocacy by others Advocacy efforts (side events) Output 4 Direct lobby & advocacy by UN agencies Synthesis and dissemination of findings from consultative processes Thematic consultations Output 3 Output 2 Consultative processes that involve the right people at the right time and place World we want/my World survey Output 5 ce to conduct analytical processes pment Agenda - UNDP project) riven sses Intergovernmental Processes

20 Building the Post-2015 Development Agenda - Open and Inclusive Consultations 2.2 Description of the Project 4 The overall project objective was to build and lead a strategic coalition of partners that can shape the post-2015 development agenda through global broad mobilization and engagement of government, grassroots, academics, CSOs and other organizations and to ensure that the post-2015 development agenda is (1) guided by UN norms, values and commitments, (2) informed and shaped by the arising challenges of the 21st century, including sustainability and equity and (3) built on the momentum and lessons learned from the MDGs. 5 The overall project objectives are presented at the top of the Reality Map. In order to work towards this goal, three streams under the project have been identified, i.e. national, multilateral and intergovernmental processes. Below is a description of the activities under each results area. National processes The efforts conducted towards informing national processes and building coalitions took for a large part place during the first phase of the project ( ). The national consultations have been designed to increase the participation of people from around the world and from all backgrounds. These national consultations were designed to inform Member States about issues and priorities in their countries and the underlying assumption was for national governments to inform their missions in New York based on the outcomes of the consultative processes. In addition, the project organized a range of lobby & advocacy activities in New York to communicate the results of the consultation processes directly to the Permanent Missions of Member States from both the North and the South. During project implementation, it was observed that in some countries, the democratic space between governments and their citizens increased. Although this outcome had not been formulated in the project documents, it was included in the evaluation as an area of investigation. UN-driven processes Part of the activities under the project were centred around consolidating the outcomes of the various global and national consultations, MY World Survey, the World We Want website and advocacy efforts. This was done by publishing synthesis reports and providing this information to relevant fora feeding into the High Level Panel 6 and directly into the UN SG Office (and to a lesser extent into other UNdriven processes like the Sustainable Development Solutions Network and UN Global Compact). The underlying assumption was that a global, inclusive consultation process that reaches out to people from different backgrounds across the world would yield information that is relevant to actors who provide input and/or shape the post-2015 agenda. Intergovernmental processes In the second phase of the project ( ), changes in context took place, which made UNDP switch the focus of the project from a UN-driven consultation process towards supporting the intergovernmental processes. One of the outcomes of the Rio+20 Conference in June 2012 was that the formulation of the Sustainable Development Goals was mandated to an Open Working Group (OWG) led by Member States. In the OWG, 30 seats were shared among 70 countries. As the intergovernmental negotiations became more central in the formulation of the SDGs, the focus of the post-2015 project also shifted its attention towards influencing and supporting the Open Working 4 For a description of the separate outputs, please refer to Annex 4 on page UNDP Project Document Building the Post-2015 Development Agenda: Open and Inclusive Consultations" (2011). 6 The project also provided the secretariat to the HLP, but this output is considered outside the scope of this evaluation. 14

21 2. Project Description Group. During this phase, the 29 technical issue briefs as requested by the OWG became an important piece of information. The briefs were delivered by the Technical Support Team (TST) under the UN System Task Team, involving many staff who had also been involved in the thematic consultations. The People s Voices Issue Briefs also provided important information to the OWG. These briefs conveyed key messages from the consultations and the MY World Survey results. Beyond the immediate scope of the project, lead agencies and host countries of the various thematic consultation processes convened briefings and side events to feed information about the results into the intergovernmental negotiations. In the second phase of the project, the attention shifted also more towards the how. When the preliminary ideas were defined and the SDGs started to take shape, attention needed to be paid to how to make the goals work and to test good practices. In response, two outputs were added to the project design (output 8 and 9 in the Reality Map). These outputs were part of the national and multilateral processes, and the results of these processes were also shared with the OWG. Spheres of control, influence and concern Besides the intervention logic of the project, the evaluation team also defined what was, and what was not, within the power of the project to control or influence. Understanding the three spheres of control, influence and concern helped with evaluating the strategy and direction of the project by defining what was being evaluated and how it was understood. The sphere of control is the area where the project has complete control over its actions. It refers to the strategy, tactics, inputs, activities and outputs that were delivered by the project. The second layer is the sphere of influence, the intersection between the sphere of control and the sphere of concern. Within the sphere of influence fall all the effects that the post-2015 project actions could have but which are beyond the direct control of the project (team). These range from raising awareness on the post-2015 agenda, to influencing the language of the SDGs or steering their implementation. The sphere of concern (i.e. impact level) represents changes in societal level which would relate to the actual implementation and subsequent benefits of the P15A. This sphere has not been taken into account in this evaluation as the implementation of the SDGs started in Project Design The UNDG-led project Building the Post-2015 Development Agenda: Open and Inclusive Consultations was housed at UNDP at the Bureau for Policy and Programme Support 7 and was approved in December The project was managed by the post-2015 team as part of the One UN Secretariat with the mandate to coordinate across work streams towards developing the post-2015 agenda. With an initial budget of US$ 7,775,250 for two years, the project took off in early 2012 and focused on the organization of consultations to facilitate an open, inclusive global dialogue with the aim of harnessing a collective buy-in from all stakeholders and ensuring that the voices and ideas of the disadvantaged and marginalized are heard in the debate on the post-2015 agenda. 8 In response to the evolving context and the need to deepen the multi-stakeholder consultations by focusing on the means of implementation and informing the intergovernmental processes with information from country level, the project was reformulated in 2013 and additional outputs were added. The total budget amounted to US$ 22,358,160 for December December In the early years, the project fell under UNDP Bureau for Development Policy. 8 UNDP Project Document Building the Post-2015 Development Agenda: Open and Inclusive Consultations (2011). 15

22 Building the Post-2015 Development Agenda - Open and Inclusive Consultations Output Description Timeframe Inclusive national consultation processes to stimulate the debate on the post-2015 agenda Global and regional consultations with academia, media, private sector, private foundations, employers and trade unions, civil society and decision makers on the current central challenges to the post-2015 development agenda and to broad coalition of change** Preparatory conferences/meetings by partners to discuss the post-2015 development agenda informed by UNDG Discussion and position papers on the post-2015 development agenda*** Leverage ICT tools to facilitate an open interaction and information exchange among a range of stakeholders about post * May June Support to set up HLP Secretariat 7 Support to the post-2015 team 8 Inclusive dialogue around the Means of Implementation Prototyping future goals / Illustrative Goals *** Some countries already started consultations before the project took off. These consultations are outside the scope of the project. A few national consultations (Burundi, Kenya and Samoa) ran through till *** In the 2013 Project Document Revision, Output 2 is phrased as Thematic Consultations. *** In the 2013 Project Document Revision, Output 4 is phrased as Knowledge Products and Communication. 16

23 3. Methodological Approach 3. Methodological Approach 3.1 Methodological Choices and Nature of the Evaluation The overall design of the evaluation is based on the Theory of Change of the Building the Post Development Agenda: Open and Inclusive Consultations that was reconstructed and discussed during the inception mission. This Theory of Change describes the intervention logic that underpinned the project and provides insights into the critical assumptions that have guided the project. The evaluation team used this Theory of Change to distinguish and assess the different tracks through which the consultations were expected to influence the shaping of the P15A. During the inception it also became clear that the creation of new partnerships and increased understanding and ownership over the 2030 Agenda were an important part of the project s ambitions that should be assessed. These dimensions were therefore included in a final set of questions formulated under the evaluation criteria that were part of the Terms of Reference; 1. Effectiveness related to the significance of the project in contributing to the various qualitative dimensions of the P15A at country and global level; 2. Relevance related to the extent needs of different stakeholders, notably vulnerable groups, to have their voices heard in the development of the P15A have been addressed (i.e. inclusiveness); 3. Positioning and partnership related to the (perceived) role of UNDP in providing thought leadership and the creation of new partnerships for the formulation of the P15A; 4. Innovation related to the extent new methods and tools have been introduced and the added value of those methods and tools in building the P15A; 5. Efficiency related to the timeliness and cost-consciousness with which the project outputs have been produced. For answering the key evaluation questions and criteria, an evaluation matrix (see Annex 7) was developed, including a rating scale of 1 to 4 (a lot, reasonably, moderately, not at all) to judge the progress made on each criteria. The evaluation relied on a mixed-method approach using various data-collection methods including desk study; qualitative interviews both in person and over Skype; focus group discussions; a web-based survey; and country visits. 17

24 Building the Post-2015 Development Agenda - Open and Inclusive Consultations The main emphasis of the evaluation was on effectiveness and relevance/inclusiveness of the various project outputs in light of the development of the 2030 Agenda at global and national levels. As was already stressed in the Terms of Reference and reconfirmed during the evaluation process, the main purpose of the evaluation was to learn from this project for the benefit of future multilateral consultation processes in terms of substance and the way the project was designed, managed and implemented. 3.2 Evaluation Standards and Principles The evaluation team operated according to the following standards: Triangulation of findings: making sure that data collected was cross-checked from different sources, also using relevant scientific research published on the consultation processes. Mixed-method approach: making sure that quantitative data (collected primarily through a survey and desk review) would be given qualitative meaning through interviews with diverse stakeholders (UN staff, CSOs, Member States) and reflective discussion with the project team. Learning oriented and utility focused, recognizing the complexity of the project: this means that the evaluation not only documents findings, conclusions and recommendations but also a set of dilemmas that emerged as further food for thought. To add depth and stimulate learning from the evaluation, a one-day sense-making workshop was organized with key stakeholders in New York. The feedback, reflection on strategic dilemmas and forward thinking, has been integrated in this report. 3.3 Evaluation process The evaluation involved three phases. The inception phase was initiated with a mission of the evaluation team to the post-2015 secretariat in New York from July 2015, to validate the Terms of Reference, clarify the expectations and information needs and to develop a draft Theory of Change. The inception mission was furthermore used to agree on the boundaries of the evaluation, determining the intended users of the evaluation and obtaining relevant documents, including a list of people to be interviewed. Part of this phase included drawing up a draft inception report that was shared with a group of key reference persons in New York before it became final in August The second phase of the evaluation involved an extensive data collection process: 1. In-depth analysis of key documents which reviewed linkages between key documents in the process of shaping the Sustainable Development Goals. 2. Desk study on the key programme documents identified by the post-2015 team, such as programme proposals, annual reports, country consultation reports, thematic consultation reports, strategy documents, online resources and relevant external academic research (see Annex 6). 3. Key stakeholder interviews with UN representatives involved in the consultations both at global and national level, key personnel of Permanent Missions to the UN in New York, CSOs, and associated donors and partners including media, academia, and youth (see Informants Annex 2). 4. Two online surveys that were designed to validate and triangulate the findings at the national level were distributed to 93 people with the request to distribute further among relevant stakeholders in their countries, resulting in a total of 232 responses. Out of these respondents, 154 were involved in national consultations (66%), others made reference to the MY World Survey, Means of Implementation (MoI) dialogues and advocacy efforts, such as launching the One Million 18