INCEPTION PAPER INSPECTION AND EVALUATION DIVISION. 8 July Programme Evaluation of the United Nations High Commissioner for Refugees (UNHCR)

Transcription

1 INSPECTION AND EVALUATION DIVISION INCEPTION PAPER Programme Evaluation of the United Nations High Commissioner for Refugees (UNHCR) 8 July 2014 Assignment No.: IED

2 INSPECTION AND EVALUATION DIVISION FUNCTION The Office shall evaluate the efficiency and effectiveness of the implementation of the programmes and legislative mandates of the Organisation. It shall conduct programme evaluations with the purpose of establishing analytical and critical evaluations of the implementation of programmes and legislative mandates, examining whether changes therein require review of the methods of delivery, the continued relevance of administrative procedures and whether the activities correspond to the mandates as they may be reflected in the approved budgets and the medium-term plan of the Organisation; (General Assembly Resolution 48/218 B). Project team members include: DEMETRA ARAPAKOS, CHIEF OF SECTION MONA FETOUH, PROJECT LEADER YUN JAE CHUN, TEAM MEMBER MARIA SINGER, TEAM MEMBER CONTACT INFORMATION OIOS/IED Contact Information: phone: ; fax: ; DEMETRA ARAPAKOS, CHIEF OF SECTION Tel: , Fax: DEBORAH RUGG, DIRECTOR Tel: , Fax:

3 Contents Paragraph Page I. Introduction II. Background A. UNHCR History, Mandate and Normative Framework B. UNHCR Structure and Governance C. UNHCR Global Strategic Priorities and Results Framework D. UNHCR Persons of Concern E. Life Cycle of Displacement and UNHCR response F. UNHCR Resources III. Evaluation scoping and M&E assessment A. Evaluation scoping process and criteria B. UNHCR M&E Capacity and Practice C. Evaluation coverage D. Evaluation topics considered E. Evaluation topic selection and rationale IV. Evaluation Terms of Reference A. Evaluation Objective B. Evaluation Approach C. Evaluation Scope D. Evaluation Questions E. Evaluation Methodology F. Evaluation Stakeholders G. Gender and Human Rights Perspective H. Evaluation Timeline and Work Plan I. Evaluation Resources J. Evaluation Deliverables K. Plan for Dissemination and Report Follow-up 3

4 I. Introduction 1. The Inspection and Evaluation Division of the Office of Internal Oversight Services (OIOS-IED) is undertaking an evaluation of the United Nations High Commissioner for Refugees (UNHCR). Selection of UNHCR for evaluation followed a risk assessment undertaken by OIOS to identify Secretariat programme evaluation priorities. The Committee for Programme and Coordination (CPC) selected the programme evaluation of UNHCR for consideration at its 55 th session in June The General Assembly endorsed the selection in its resolution 68/ The general frame of reference for OIOS is General Assembly resolutions 48/218B, 54/244, and 59/272, as well as ST/SGB/273 and Article 97 of the United Nations Charter, which authorizes OIOS to initiate, carry out and report on any action that it considers necessary to fulfill its responsibilities. The general frame of reference for OIOS-IED is provided in the Regulations and Rules Governing Programme Planning, the Programme Aspects of the Budget, the Monitoring of Implementation and the Methods of Evaluation (PPBME). 2 II. Background A. UNHCR History, Mandate and Normative Framework 3. Every year, millions of people are forced to leave their homes to escape persecution, conflict or natural disasters, and find themselves facing new vulnerabilities, security threats and lack of access to basic goods and services. Providing protection and services for these populations, while working towards solutions, is a global responsibility which is led, in many cases, by UNHCR. 4. UNHCR was founded on 14 December 1950 with a mandate to lead and coordinate international efforts to protect the rights and well-being of refugees and find solutions to refugee situations. Since then, its mandate has expanded to include stateless people. UNHCR has a shared responsibility for internally displaced persons (IDPs) within an inter-agency approach. 5. UNHCR s main objective is to ensure international protection to refugees and others of concern to the Office of UNHCR and to seek permanent solutions to their problems in cooperation with States and other organizations, including through the provision of humanitarian assistance. 3 Its mandate was extended in 2003 by the General Assembly until the refugee 1 Draft report, Addendum Programme questions: Evaluation of the Office of the United Nations High Commissioner for Refugees (E/AC.51/2013/5), Committee on Programme and Coordination E/AC.51/2013/L.4/Add.4 2 ST/SGB/2000/8, p. 12, Regulation 7.1 notes that the objective of evaluation is: (a) To determine as systematically and objectively as possible the relevance, efficiency, effectiveness and impact of the Organization s activities in relation to their objectives; (b) To enable the Secretariat and Member States to engage in systematic reflection, with a view to increasing the effectiveness of the main programmes of the Organization by altering their content and, if necessary, reviewing their objectives. 3 Proposed strategic framework for the period : Part two: biennial programme plan, Programme 21: International protection, durable solutions and assistance to refugees, February 2012 (A/67/6 (Prog. 21) 4

5 problem is solved. 4 UNHCR s role has grown significantly as the nature of conflict and crises has grown more complex and resulted in diverse and growing populations of concern. In 2013, UNHCR was responsible for 35.8 million people under its mandate. 5 Partnerships with governments, United Nations entities, non-governmental organizations and displaced and host communities are UNHCR s key modality in the delivery of basic assistance, and contracts to implementing partners represent 39 per cent of its total expenditures in The primary instruments governing the rights of refugees and the legal framework underpinning UNHCR s work are the 1951 Convention Relating to the Status of Refugees (1951 Convention) and its 1967 Protocol, with 145 and 146 state parties, respectively. 7 The international legal framework regarding stateless people stems from the 1954 Convention relating to the Status of Stateless Persons and the 1961 Convention on the Reduction of Statelessness, with 79 and 55 state parties to each respectively. 8 These are supplemented by regional level instruments. 7. In the past two decades, UNHCR has undergone a series of reforms aimed at responding to new challenges in displacement and the evolving landscape of conflict, as well as improving efficiencies. A key outcome was to improve efficiency by reducing administrative and support costs and to reduce the number of staff located in Geneva. Reforms also led to the establishment, in 2008, of a Global Service Centre in Budapest, Hungary, which has responsibility for most administrative functions and supply chain management. B. UNHCR Structure and Governance 8. UNHCR is headed by the High Commissioner, who is responsible for the direction of the organization, and is supported by a Deputy High Commissioner and two Assistant High Commissioners one each for Operations and Protection. 9. The Office is governed by the General Assembly and Economic and Social Council and reports annually to both bodies. An Executive Committee (ExCom), formed in 1959 and comprised of 94 Member States, meets every October and approves the biennial programme and budget. The Standing Committee of the ExCom meets throughout the year. 10. UNHCR s headquarters are located in Geneva, Switzerland. Geographically, it is organized by five Regional Bureaux, which direct operations in each region and liaise between field offices and headquarters. The five regions are: Africa, the Americas, Europe, Asia and the Pacific, and the Middle East and North Africa. UNHCR is located in 449 locations in 123 countries. 9 4 General Assembly Resolution 428 v, Statute of the United Nations High Commissioner for Refugees, (A/RES/428 (V)) 14 December 1950 and General Assembly Resolution 58/153 (A/RES/58/153), February UNHCR Mid-Year Trends 2013, Table 1, page 17, UNHCR, UNHCR Global Report 2013, calculations based on figures on pages 84, 113, UNHCR, United Nations Treaty Collection online database, accessed by IED on 5 June UNHCR Global Report 2013, Annex, page 214, UNHCR, UNHCR Global Appeal , UNHCR,

6 11. UNHCR headquarters is comprised of the following seven divisions, each led by a Director: Division of External Relations (DER) Division of Emergency, Security and Supply (DESS) Division of Financial and Administrative Management (DFAM) Division of Human Resources Management (DHRM) Division of Information Systems and Telecommunications (DIST) Division of International Protection (DIP) Division of Programme Support and Management (DPSM) 10 C. UNHCR Global Strategic Priorities and Results Framework 12. UNHCR s goals are articulated as Global Strategic Priorities (GSPs) in its key reports and results framework. The operational GSPs correspond to the Expected accomplishments reported in the General Assembly-approved Strategic Framework. Table 1 below shows the operational GSPs and EAs in relation to each other. Table 1: UNHCR Operational Global Strategic Priorities and Expected accomplishments Global Strategic Priority Expected accomplishment Favourable protection environment - Ensuring access Improved overall protection environment for refugees to territorial protection and asylum procedures; and others of concern protection against refoulement; and the adoption of nationality laws that prevent and/or reduce statelessness Fair protection process and documentation - Securing birth registration, profiling and individual documentation based on registration Security from violence and exploitation - Reducing protection risks faced by persons of concern, in particular, discrimination, sexual and gender-based violence (SGBV) and specific risks faced by children Basic needs and services - Reducing mortality, morbidity and malnutrition through multisectoral interventions; Meeting international standards in relation to shelter, domestic energy, water, sanitation and hygiene Community empowerment and self-reliance - Promoting active participation in decision-making of persons of concern and building coexistence with hosting communities; Promoting human potential through increased opportunities for quality education and livelihoods support Durable solutions - Expanding opportunities for durable solutions for persons of concern, particularly those in protracted situations, including through strengthening the use of comprehensive approaches and contributing to sustainable reintegration, local settlement and successful resettlement in third countries Refugees and others of concern are treated fairly and efficiently when seeking protection, and receive adequate documentation Increased safety from violence and exploitation for refugees and others of concern, especially women and children The basic needs of refugees and others of concern are met and essential services are provided without discrimination and with specific consideration to age, gender and physical condition Refugees and others of concern, both men and women, participate equally within their communities and their self-reliance is promoted Progress in finding durable solutions for refugees supported by sustained international cooperation 10 Biennial Programme Budget of the United Nations High Commissioner for Refugees (A/AC.96/1125). 6

7 UNHCR effectively prepares for, and responds to, emergencies 11 Strengthened partnership and emergency response capacity to provide the fullest possible coverage of the needs of persons of concern Source: UNHCR Biennial Programme Budget and Strategic Framework UNHCR intensified efforts to implement results-based management processes during its reforms in 2006, resulting in a comprehensive Results Framework, noted in section III below. D. Lifecycle of Displacement and UNHCR Response 14. UNHCR operations and response can be categorized according to the lifecycle of displacement faced by the population served by UNHCR, as shown in Figure 1 below. UNHCR s responsibility starts from the moment people flee their homes until and, often even after, durable solutions are found. It is mandated to facilitate access to protection space, provide essential services for displaced people, facilitate and negotiate durable solutions, and aid in integration or reintegration in the post-solutions phase. Figure 1: Lifecycle of displacement Source: OIOS-IED F. UNHCR Persons of Concern 15. The 42.9 million people who fall under UNHCR s mandated responsibility, referred to as persons of concern (PoC), are categorized as follows: Refugees persons who are outside their country of nationality (or former habitual residence) owing to well-founded fear of being persecuted, as further defined in Article 1A(2) of the 1951 Convention. 11 Categorized under the support and management GSPs rather than as an operational GSP. 7

8 Returnees persons who were of concern to UNHCR when outside of their country of origin and who remain so, for a limited period after returning to their country of origin; also, IDPs who return to their previous place of residence. Stateless people persons who are not considered as nationals by any State under the operation of its law. Asylum-seekers persons who are seeking international protection; in countries with individualized procedures, their claims have not yet been finally decided on. Internally displaced people (IDPs) persons who have been forced or obliged to flee from their home or place of habitual residence and have not crossed an internationally recognized State border. UNHCR works primarily in conflict-related internal displacement, though a coordinated, inter-agency approach. 16. With the recent chain of crises since 2011, the numbers of PoCs have increased steadily in the last two biennia, from 25.8 million in 2012, to a projected 42.8 million by the end of Table 2 below shows where these populations were located by region and type in The greatest growth was in the Middle East and North Africa, where there were 1.3 million refugees in the first half of 2013 alone due to the Syria crisis. The second largest increase during this period was in Africa, due to crises including South Sudan, Mali, and Central African Republic. UN major regions Table 2: Populations of concern by region and population type, 2013 Total refugees and people in refugeelike situations Of whom assisted by UNHCR Asylumseekers (pending cases) Returned refugees IDPs protected/ assisted by UNHCR, incl. people in IDP-like situations 8 Returned IDPs Persons under UNHCR's statelessne ss mandate Various Total population of concern Africa 2,932,854 2,679, , ,479 7,633, , , ,616 13,007,250 Asia 3,547,077 2,764,496 94,924 43,600 1,910, ,148 1,422, ,870 7,770,813 Pacific Middle 2,630,787 2,333,988 68, ,655 7,835, , ,237 3,706 11,346,066 East and North Africa Europe 1,786, , , , , , ,590 4,225,688 Americas 805,960 97, , ,368, ,032 12,318 6,526,765 Total 11,703,179 8,519,364 1,171, ,554 23,925,555 1,356,182 3,469, ,100 42,876,582 Source: UNHCR Global Trends Trends in UNHCR populations of concern reflect the changing nature of conflict and displacement in recent decades. As old models of interstate conflict have been overtaken by conflict driven by non-state armed groups, the numbers of IDPs have been steadily rising. More people are also fleeing due to environmental factors, or a combination of such factors and state failures, rather than conflict alone. Another trend is the growth of urban refugees, who in 2013 represented the majority of refugees under UNHCR s responsibility. E. UNHCR Resources 18. In 2014, UNHCR s budget was US$ 5.3 billion, a significant increase from 2012 due to funding contributions related primarily to the Syria crisis, as shown in Table 3 below. Two per

9 cent of UNHCR s budget comes from United Nations regular budget contributions, whereas the remaining 98 per cent is funded by voluntary contributions. 12 Figure 2 below shows expenditures by region for Table 3: 2012 expenditures, 2013 current budget, and proposed budgets 2012 (expenditure) 2013 (current budget) 2014 (proposed budget) 2015 (proposed budget) Field 1,990,083 4,586,289 4,460,693 4,360,243 (+131%) (-3%) (-2%) Global programmes 170, , , ,453 (+21%) (+6%) (-8%) Headquarters 188, , , ,611 Sub-total 2,349,104 4,987,946 4,880,915 4,762,307 programmed (+112%) (-2%) (-2%) activities Reserve and Junior Professional Officer funds 8, ,971 (+3176%) 426,928 (+51%) 417,156 (-2%) Total 2,357,710 5,269,917 5,307,843 5,179,464 Source: UNHCR Biennial Programme Budget Figure 2: 2013 UNHCR Expenditure by Region Source: UNHCR Global Report 2013, page The budget is divided by four pillars corresponding in part to the key groupings of PoCs. The refugee pillar represents by far the highest proportion, with 81 per cent of the total requirements for 2014, and 76 per cent of actual expenditures in When divided by Global Strategic Priority rights group (noted in Table 1), basic needs and essential services comprise 56 per cent of total 2014 requirements, while the remaining areas range between 1 and 8 per cent. The proposed 2014 budget allocates 8 per cent for durable solutions. However, this figure is not fully representative of the range of activities that promote work towards solutions, which 12 Biennial Programme Budget of the United Nations High Commissioner for Refugees (A/AC.96/1125) 9

10 includes, for example, education and livelihoods work that is classified as basic needs and services, and advocacy work that falls under the favourable protection environment GSP. Solutions generally represent the culmination of protection and assistance activities. 20. In 2013, UNHCR had 9,087 staff, 89 per cent of whom were located in field offices. Overall, staff numbers have increased 7.5 per cent from 2012 to 2013, although there has been a doubling of the budget during the same period. 13 III. Evaluation scoping and M&E assessment A. Evaluation scoping process and criteria 21. In order to determine the focus of this evaluation, IED undertook a scoping process which entailed the following steps: a review of numerous documents, including UNHCR Strategic Frameworks and Results Frameworks, programme budgets, reports on UNHCR s work, and internal and external reviews of UNHCR (including audits, evaluations and recent reviews of UNHCR s evaluation capacity); the development of a Programme Impact Pathway (PIP), shown in Figure 3 below, which details the logic model of inputs, outputs, outcomes, and impacts of UNHCR programmes; the conduct of 18 semi-structured interviews, including with the Deputy High Commissioner, Directors of UNHCR Divisions and Bureaux, and several country representatives, donors and key partners; ongoing discussions and consultation with the focal point at UNHCR as well as the Head of the Policy Development and Evaluation Service (PDES); and coordination with staff conducting reviews of UNHCR from OIOS Internal Audit Division (IAD) and the Board of Auditors (BOA). 13 Ibid. 10

11 11

12 22. Through this process, IED s selection of an evaluation topic considered the following criteria: coverage of a critical strategic area of UNHCR operations; criticality to the achievement of results in the field in relation to UNHCR s mandate and objectives; balance of geographic coverage and diversity; portion of UNHCR s resources; relevance with regard to addressing a high priority issue; complementarity to recently conducted and ongoing OIOS audits and evaluations, UNHCR self-evaluations and external reviews; and added value of IED evaluation as an independent oversight body. B. UNHCR M&E capacity and practice 23. Since its establishment in 1972, UNHCR s evaluation function has evolved to its current configuration as part of the Policy Development and Evaluation Service (PDES), located in the Executive Office and consisting of 5 posts. The Head of the Service reports to the High Commissioner. UNHCR s current evaluation policy states that PDES is responsible for the management and development of UNHCR s evaluation function, which entails undertaking, commissioning and managing evaluations of UNHCR s activities, among other responsibilities. According to UNHCR s website (where PDES posts all of its evaluation reports), PDES has managed 66 evaluations since PDES produces its work plan on a rolling basis, updating it every 6 months, and presents an annual report to the Executive Committee According to three recent reviews that IED conducted (two on the overall Secretariat evaluation capacity for the biennium and a third specifically on UNHCR evaluation capacity), areas for improvement included: having an evaluation function that was more independent and not merged with policy development; having a more robust evaluation plan; having procedures for the review of evaluation reports and follow-up of recommendations; and allocating sufficient resources to monitoring and evaluation (e.g., UNHCR allocated per cent in ). 16 In addition, despite its strong field presence, UNHCR does not conduct systematic or regular evaluations at the regional or country level. UNHCR has started to implement these reports recommendations, including introducing a management response requirement for evaluations, to be overseen in select instances by the newly established Internal Compliance and Accountability Committee. 25. In addition, UNHCR s internal oversight includes the Inspector General s Office, which conducts regular inspections of UNHCR operations and offices to ensure conformance with 14 According to the IED report on the Review of the evaluation capacity of UNHCR, of the 28 reports from , it was not clear what standard PDES used to categorize them as evaluations given that some of the reports were policy papers and academic research papers not directly related to evaluation of UNHCR interventions. 15 See A/AC.96/1128, Report on Policy Development and Evaluation. 16 Strengthening the role of evaluation and the application of evaluation findings on programme design, delivery and policy directives, 2 April 2013, A/68/70; United Nations Secretariat Evaluation Scorecards , December 2013, IED ; and Review of the evaluation capacity of the Office of the United Nations High Commissioner for Refugees, 25 March 2013, E/AC.51/2013/5. 12

13 organizational rules and regulations. One of the Office s core functions is to conduct inspections of the quality of management of UNHCR operations. 17 Moreover, IAD provides internal audit services to UNHCR under a Memorandum of Understanding (the first MOU was signed in April 1997). 26. UNHCR extensively monitors its operations through different systems. In relation to the outputs, indicators of achievement and expected accomplishments set forth in its Strategic Framework, UNHCR collects data for the Secretariat-wide Integrated Monitoring and Documentation Information System (IMDIS). It also has developed two software systems, Focus and Managing Systems Resources and People to help plan and monitor performance information tied to its Results Framework and financial information on expenditures. In May 2013, UNHCR completed development of a new tool, Global Focus Insight, which provides new dashboard views and key analytical reports, both at the field operations and global corporate levels. C. Evaluation coverage 27. Many entities review UNHCR s work, including PDES, OIOS (IED and IAD), BOA, JIU, and donors. 18 With regard to PDES: From mid-2102 to mid-2013, it reviewed the use of refugee protection and solutions in urban areas, building upon earlier evaluations of the implementation of UNHCR s urban refugee policy in select countries, and surveyed 24 country offices. During the same period, PDES also undertook evaluations of UNHCR s emergency response and preparedness. In the area of solutions, PDES has commissioned or initiated reviews of the strategic use of resettlement, integration of resettled refugees in labour markets, engagement with assisted voluntary return programmes, and efforts to link humanitarian relief and development. In , UNHCR evaluated the overall progress of the Protracted Refugee Situation Initiative, resulting in reports for 4 out of the 5 selected situations. 19 Due to an overlap in strategic interests, a joint evaluation with the Danish Ministry of Foreign Affairs was conducted for the protracted refugee situation (PRS) of Burundian refugees in the United Republic of Tanzania. 17 See A/AC.96/1127, Report on the activities of the Inspector General s Office. 18 See, e.g., evaluations by the 1) United States Department of State s Bureau of Population, Refugees, and Migration: Evaluation of Humanitarian Assistance to Colombian Refugees in Ecuador and Somali Refugees in Kenya: Supporting Protracted Refugees Livelihoods, Protection Space, and Self-Reliance; Evaluation of Reintegration and Recommendations for Program Strengthening in Afghanistan; and Impact Evaluation of PRM Humanitarian Assistance for the Repatriation and Reintegration of Burundi Refugees; 2) United Kingdom s Department for International Development Multilateral Aid Review, March 2011 and March 2013 update; 3) Australian Multilateral Assessment, March 2012; and 4) Multilateral Organisation Performance Assessment Network Assessment of UNHCR 2011, and ongoing 2014 assessment. 19 A review of UNHCR s response to the protracted situation of stateless Rohingya refugees in Bangladesh, December 2011; A review of UNHCR s response to the protracted refugee situation in Serbia and Croatia, December 2010; A review of UNHCR s response to the protracted refugee situation in eastern Sudan, November 2011; Evaluation of the Protracted Refugee Situation (PRS) for Burundians in Tanzania, October 2010; and Joint Evaluation between PDES and Evaluation Department in the Danish Ministry of Foreign Affairs. 13

14 Also in , UNHCR conducted a joint impact evaluation with the World Food Programme (WFP) on the contribution of food assistance to durable solutions in PRS, focusing on Bangladesh, Chad, Ethiopia and Rwanda With regard to audits, from July 2012 through June 2013 IAD completed 31 audits with opinions (i.e., satisfactory, partially satisfactory or unsatisfactory) given on governance, risk management and internal control processes. 21 Ongoing audits of note in 2014 include a Summary report on recurrent audit results on implementing partners identified in country audits and an Audit of emergency preparedness and response. 29. Reviews conducted by other United Nations entities include annual financial reports from the BoA and an ongoing Joint Inspection Unit report on the Review of the management of implementing partners (IPs) in United Nations system organizations, which includes UNHCR. 22 D. Evaluation topics considered 30. Resulting from the scoping process and evaluation coverage noted above, IED identified the following gaps and possible topics for its evaluation focus: 1) UNHCR s role in emergencies and humanitarian coordination: The number of simultaneous complex emergencies, most notably in Syria, the Central African Republic, the Philippines, and South Sudan, has affected the management of UNHCR s operations. In some of these emergencies, UNHCR has also had challenges of working with integrated missions. Questions arise on UNHCR s effectiveness in the larger inter-agency humanitarian framework and cluster approach and in the roll-out of the Transformative Agenda (adopted in late 2011). 2) Effectiveness of relationships with implementing partners: With over 700 implementing partners and 39 per cent (US$ billion) of UNHCR s expenditures implemented through them, the effectiveness of arrangements between UNHCR and its partners warrant further evaluation. This includes UNHCR s role in monitoring partner activity and the changing nature of the partnerships. 3) Refugees outside of camp settings: Only one-third of the world s refugees now live in camps, and more than half of the refugees that UNHCR serves live in urban areas, with the remainder outside camps living in rural areas. 23 The number of urban refugees is on the rise and presents different challenges in terms of providing protection and assistance, local integration and relationships with host communities. 20 Synthesis Report of the Joint WFP and UNHCR Impact Evaluations on the Contribution of Food Assistance to Durable Solutions in Protracted Refugee Situations, conducted through in Bangladesh, Chad, Ethiopia and Rwanda. 21 A/AC.96/1126, Internal Audit in the Office of the United Nations High Commissioner for Refugees ( ). 22 A/68/5/Add.5, Report of the Board of Auditors: Financial report and audited financial statements; JIU/REP/2013/4. 23 UNHCR website, accessed by IED on 16 June

15 UNHCR has a specific policy on refugee protection and solutions in urban areas, adopted in ) Normative framework: The 1951 Convention and the 1967 Protocol define refugee as a person being persecuted for reasons of race, religion, nationality, membership of a particular social group or political opinion, is outside the country of his nationality or who, not having a nationality and being outside the country of his former habitual residence as a result of such events 24 Questions arise on the effectiveness of UNHCR s normative work, particularly with the causes for displacement changing, and persons of concern not necessarily falling under the legal definitions. 5) UNHCR s work related to statelessness: For mid-2013, UNHCR s offices reported 3.5 million stateless persons, but with underreporting it estimates their actual number to be at least 10 million. 25 In its Biennial programme budget for , UNHCR proposed allocating approximately 1.3 percent of its total budget on its Global stateless programme. 26 With its multi-faceted mandate, UNHCR s work in this area has received less evaluation attention. Issues include UNHCR s work in assisting governments ratify the 1954 Convention Relating to the Status of Stateless Persons and 1961 Convention on the Reduction of Statelessness, and the effectiveness of UNHCR s operations in reducing the number of stateless persons. 6) UNHCR s work related to durable solutions for protracted situations: OIOS selected this topic, in the context of UNHCR s effectiveness in achieving its overall mandate, further to the rationale described below. The topics not selected may warrant evaluations in the future. E. Evaluation topic selection and rationale 31. Recent emergencies have resulted in a shift of UNHCR s resources and attention towards emergency contexts, and away from other parts of its mandate, including durable solutions. This has led to questions on whether or not the appropriate balance exists between its different activities, and about UNHCR s core roles and responsibilities vis-à-vis its intended goals and results. Senior staff interviewed reported that the Office has been more successful in its emergency response interventions, and less successful in achieving solutions, with durable solutions for protracted situations not having been attained at the expected rates. 32. At the end of 2013, some 6.3 million refugees under UNHCR s mandate were part of a protracted refugee situation (PRS), defined as those who have been refugees for five years or more after their initial displacement, without immediate prospects for implementation of durable solutions. 27 UNHCR has identified 33 major PRS involving 6.3 million refugees an increase Convention Relating to the Status of Refugees, Article 1A(2). 25 UNHCR Mid-Year Trends 2013, p. 11, UNHCR A/AC.96/1125, Table I UNHCR Global Appeal , p. 49, UNHCR

16 from 5.5 million in UNHCR defines major PRS as refugee populations of 25,000 persons or more who have been in exile for five or more years in developing countries. As of 2013, there are also over 20 million IDPs assisted by UNHCR who are in protracted situations, although UNHCR does not have sole responsibility for these populations Furthermore, the issue of durable solutions for protracted situations has been identified as critical for UNHCR, as exemplified by the attention it has received through several high-level initiatives over the past several years. These include: 1) The Dialogue on Protection Challenges convened by the High Commissioner in 2009 on the topic of PRS which led to the implementation of a Special Initiative on PRS focusing on five situations: Afghan refugees in the Islamic Republic of Iran and Pakistan; refugees from Myanmar in Bangladesh; Bosnian and Croatian refugees in Serbia; Burundian refugees in the United Republic of Tanzania; and Eritrean refugees in eastern Sudan. Four of the five situations have been assessed. 2) The Transitional Solutions Initiative in 2010, in collaboration with the World Bank and United Nations Development Programme (UNDP), which aims to help prioritize displacement needs on the development agenda of governments and international development donors and other actors. 30 An assessment has been initiated. 3) The Policy Committee meeting of 4 October 2011, in which the Secretary-General endorsed the Framework on Ending Displacement in the Aftermath of Conflict, establishing priorities and responsibilities to support the delivery of durable solutions for IDPs and refugees returning to their countries of origin. 31 An assessment has not been undertaken. 4) The internal Solutions Steering Group established by UNHCR in July 2013 with a mandate to review its approach to solutions and to provide technical support and advice on resource allocation for projects which are most likely to yield results in moving a particular refugee group towards solutions. 34. While prior evaluations have covered specific countries or issues, a gap exists in the overarching question of UNCHR s role and effectiveness in durable solutions in general, and how results in this area compare to those achieved in other parts of its mandate. Moreover, UNHCR s work in durable solutions is multi-faceted and linked with other parts of its mandate on protection and assistance. In this regard, the evaluation will assess not only the GSP entitled Durable solutions but also the other GSPs related to protection and assistance activities that support solutions. The selected topic speaks to UNHCR s ultimate impact of reducing the number of persons of concern through durable solutions, resonates with UNHCR senior staff and stakeholders, and allows for a broad-based strategic view of UNHCR s effectiveness, thus 28 UNHCR statistics provided by PDES, Internal Displacement Monitoring Centre Global Figures, Being piloted in Colombia and Sudan. 31 Decision No. 2011/20 Durable Solutions: Follow up to the Secretary-General s 2009 report on peacebuilding; being piloted in Afghanistan, Kyrgyzstan and Côte d Ivoire. 16

17 ensuring utility and added value of OIOS work. With this evaluation, IED will also be able to complement some of the more narrowly-focused reviews on components of solutions or country and region-based studies, and review UNHCR s monitoring data. IV. Evaluation Terms of Reference A. Evaluation objective 35. The overall evaluation objective is to determine, as systematically and objectively as possible, the relevance, efficiency, effectiveness and impact of UNHCR towards achieving its mandate in protection, assistance and solutions in emergency and non-emergency contexts, with a focus on durable solutions for protracted situations. B. Evaluation approach 36. The evaluation will have a two-fold purpose in terms of accountability and learning. With regard to accountability, it will provide evidence of results achieved in durable solutions for PoC, as well as evidence of the effectiveness of emergency and assistance operations, while examining the strategies, tools and processes that contribute towards those results. With regard to learning, the evaluation will explore what the main factors affecting the achievement of the mandate and key goals have been, and will provide lessons learnt and recommendations for improvement. 37. In order to ensure that the evaluation has maximum utility for UNHCR and responds to priorities facing PoC, the evaluation team will work in close consultation with the evaluand, while still maintaining its independence. The evaluation team has and will continue to work with the PDES focal point in arranging meetings and contacting respondents for data collection. The evaluation team will debrief UNHCR through the focal point at key points in the evaluation process, and will share early results to facilitate timeliness and relevance of evaluation outcomes. C. Evaluation Scope 38. IED will focus its primary data collection on UNHCR s activities, resources and results in the three primary solutions options: voluntary return, resettlement to a third country, and local integration into the host country. IED will assess UNHCR s effectiveness in achieving its protection and assistance mandates through secondary sources. Planning for and implementing solutions covers the full lifecycle of displacement, and is conducted by UNHCR through both its work with partners and governments, and through its assistance and protection activities for PoC. UNHCR has also started to consider comprehensive solutions strategies that involve all or elements of all three options, while working to promote greater self-reliance. Figure 4 below details the logic model of inputs, outputs, outcomes, and impact of UNHCR s work in achieving durable solutions for PoC. 17

18 18

19 D. Evaluation Questions 39. The evaluation will seek to answer a number of key questions and sub-questions, which may be adapted during data collection, covering the selected evaluation criteria of relevance, efficiency, effectiveness and impact. 32 These questions were developed as a result of the scoping process interviews and document reviews, as well as a review of the PIP. The PIP also helped in identifying the factors affecting performance, and tailoring evaluation questions around the most critical areas and processes. 1) What key results has UNHCR achieved in protection, assistance and solutions for persons of concern? a. What results has UNHCR achieved compared to expected targets in each Global Strategic Priority according to monitoring data? b. What results has UNHCR achieved in protection, assistance and solutions according to recent assessments by UNHCR and others? c. What areas is UNHCR doing less well in? What are the resource allocations and funding for these areas? d. What is the comparison of targets achieved in emergency versus protracted situations? e. How have UNHCR s results in durable solutions compared to results achieved in other parts of its mandate? f. Does UNHCR have the necessary portfolio of key activities and functions to achieve its overall objectives? 2) What results has UNHCR achieved in durable solutions for protracted situations? a. What results have been achieved in resettlement, return and integration? b. How has progress on solutions compared with expected results/targets? c. What are the trends in resettlement, return and local integration? d. What is UNHCR s role in promoting self-reliance and resilience of persons of concern, including women and vulnerable groups? e. Where has UNHCR closed operations and what were the criteria? f. What are the constraints on reaching solutions and how has UNHCR met them? g. What are the unintended consequences of UNHCR s presence and assistance in protracted situations? 3) Does UNHCR have the necessary strategies, tools, processes and resources to achieve its long-term objectives? a. What are the current strategies in achieving solutions and to what extent are they appropriate? b. Does UNHCR have the right tools and processes, in the right place at the right time, for the achievement of solutions? c. How, if at all, are solutions strategies incorporated into planning for emergency operations? 32 The United Nations Evaluation Group Norms and Standards for Evaluation in the UN system note the following key evaluation criteria, which are in line with OECD/DAC Evaluation criteria: relevance, effectiveness, efficiency, impact, and sustainability. 19

20 d. What is the resource allocation process (e.g., financial resources, human resources, and fundraising appeals) for each Global Strategic Priority and is allocation commensurate with needs? e. How have key high-level initiatives on solutions facilitated the achievement of results? f. How have strategies been adjusted to meet new realities (e.g., urban refugees, climate change, etc.)? Have these adjustments been adequate? 4) How well is UNHCR working with stakeholders to achieve its long-term objectives? a. What is UNHCR s role in engagement with governments and how effective has it been? b. What is UNHCR s role in engagement with operating and implementing partners and how effective has it been? c. How does UNHCR involve persons of concern in decisions on solutions, including women and vulnerable groups? d. How effective is UNHCR s work with United Nations Country Team (UNCT) partners in the transition to development and (re)integration? e. How effective has advocacy with governments been in creating or increasing acceptance of asylum-seekers and refugees? E. Evaluation Methodology 40. The evaluation methodology seeks to answer the evaluation questions through a combination of qualitative and quantitative data collection and analysis, and with information collected at UNHCR headquarters and several field locations. Results will be based on a triangulation of evidence collected through these methods, in order to ensure high validity. The evaluation team will develop a data collection plan to ensure that the evaluation questions will be covered through the intended methods, which will include the following: Document review: structured content analysis (descriptive and/or comparative, as relevant) of a selected sample of key reports and documentation, including but not limited to UNHCR strategic and financial documents, UNHCR performance documents, narrative reports, publications, academic reviews, audits and evaluations, journal articles, internal directives and papers, and reports to governing bodies; Meta-analysis of existing assessment and evaluation reports: tabulation and comparative analysis of results from a sample of evaluation reports and reviews by PDES and external entities, to provide aggregated evidence of UNHCR s effectiveness in emergency and non-emergency contexts; Secondary data analysis: Analysis of monitoring and performance data, including trends of solutions (i.e., resettlement, repatriation, and integration), assistance, protection, and associated resources; Interviews: structured and semi-structured interviews with UNCHR staff, partners, donors, governments, ExCom, persons of concern and other stakeholders, both at headquarters and in the field, as per the stakeholder categories noted in section E; Focus groups: small focus group discussions with persons of concern both in host countries and in post-solutions contexts; 20

21 Surveys: local population survey of persons of concern in one site, to obtain their perspectives and experiences regarding solutions options and UNHCR s effectiveness; web-based survey of a purposive sample of UNHCR staff, focused on those with responsibility for solutions; and a web-based survey of operating and implementing partners; Direct observations and site visits: visits to countries of operations, including to camps and communities where persons of concern are settled, and observation of conditions, communications, and community structures and governance processes, as well as UNHCR services and operations. 41. In order to answer the evaluation questions, the evaluation team will visit UNHCR headquarters in Geneva as well as five countries of operations to gather country and site-level documentation, conduct interviews and focus groups with key informants, and carry out direct observation of implementation in the field. The proposed countries are Afghanistan, Bangladesh, Ethiopia, Pakistan, and Sudan. One site will be chosen as the location of the local population survey, which will be conducted with local academic partners. The data collection missions will provide the perspectives of different stakeholders including: UNHCR staff; government officials; staff of the UNCT partners; international and local NGOs and civil society organizations; and local populations. 42. The suggested mission locations were selected through a purposive sampling of the 42 countries which host protracted refugee and IDP situations, on the basis of the following key criteria: size of population of concern, budget, geographical representation, the proportion of PoCs in protracted situations, parties and non-parties to the 1951 Convention and 1967 Protocol, duration of displacement, previous evaluation coverage, and the nature and context of displacement. 43. The locations chosen represent a cross-section of UNHCR work in different contexts. In the selection of the missions locations, the team also considered the opportunity and costeffectiveness of a visit in the scheduled time period. F. Evaluation Stakeholders 44. In order to provide a comprehensive assessment of UNHCR results, OIOS-IED will consult with a range of stakeholders during data collection, in addition to UNHCR HQ and field staff. Partners are critical in achieving solutions, which require structures for self-reliance, integration, and livelihoods opportunities. In emergency contexts involving IDPs, UNHCR normally operates within the cluster system of coordination established by the Inter-Agency Standing Committee (IASC), while maintaining its key mandate, and recent agreements with 21

22 OCHA have clarified UNHCR s role in mixed contexts. 33 Key stakeholders and partners for the organization as a whole include the following: Persons of concern and community governance structures such as camp committees and community-based organizations. Host communities: local communities in host countries, amongst whom persons of concern are temporarily settled. States, including host and asylum governments and countries of origin. State-level partners include line ministries, legislative bodies, immigration services, high courts and agencies related to migration and asylum issues. Governing bodies including the General Assembly, the Economic and Social Council, and ExCom. Donors, including bilateral and multilateral agencies such as the United States State Department, Government of Japan, European Commission, and World Bank, as well as private donors. Ninety-eight per cent of UNHCR s budget is made up of voluntary contributions. United Nations partner entities including: o United Nations Office for the Coordination of Humanitarian Affairs (OCHA) o UNDP o United Nations Office of the High Commissioner for Human Rights (OHCHR) o WFP o United Nations Children s Fund (UNICEF) o Other UNCT members o Other IASC members International, national and local non-governmental organizations and other civil society organizations which comprise the majority of UNHCR s operating and implementing partners totalling 733 in Media organizations, through which UNHCR promotes its advocacy initiatives. Academic institutions, such as the Refugee Studies Centre at Oxford and the Feinstein International Center at Tufts, which provide advice, studies and thinking on forced displacement issues. G. Gender and Human Rights Perspective 45. According to General Assembly resolutions A/RES/53/120, A/RES/60/1, and A/RES/60/1, gender and human rights perspectives must be mainstreamed into all United Nations policies and programmes. Guidance from the United Nations Evaluation Group (UNEG) states that an evaluation that is Human Rights and Gender Equality responsive addresses the programming principles required by a human rights based approach and gender mainstreaming strategy. Gender and human rights concerns, including a rights-based approach, are also fundamental to the work of UNHCR, and the evaluation will incorporate this perspective throughout all stages of the project. 33 Joint UNHCR-OCHA Note on Mixed Situations Coordination in Practice, UNHCR Global Report 2013, p.84, UNHCR

23 46. The key evaluation questions include specific considerations of gender and human rights issues. Moreover, data collection will examine issues specific to women and girls in displaced contexts, who face specific challenges and concerns, particularly related to sexual and genderbased violence. IED will also consider UNHCR s Accountability Framework for Age, Gender and Diversity Mainstreaming and Targeted Actions and progress achieved, when assessing results. IED will ensure that the perspectives of rights holders and duty bearers are incorporated into data collection and analysis by obtaining perspectives of displaced groups through surveys, interviews and focus group discussions. IED will also analyse and present gender-disaggregated data where available. H. Evaluation Timeline and Work Plan 47. The evaluation will be implemented in five phases, as follows: Phase 1: Scoping/planning (February June 2014). Document review, development of PIP, initial consultations, scoping interviews, and contracting of subject-area expert(s). This phase has resulted in this inception paper and evaluation terms of reference. Phase 2: Field and HQ data collection (July October 2014). Mixed method data collection, including document review, meta-analysis, secondary data analysis, interviews, surveys, and mission travel. Phase 3: Data analysis (October December 2014). Consolidation and analysis of data. Phase 4: Report writing (November 2014 January 2015). Development of evaluation results and recommendations and drafting of report. Phase 5: Report finalization (February March 2015). Dissemination of draft report to UNHCR for comments, inclusion of comments into final report, and finalization of report. 48. A provisional timeline is presented below. While the timeline is subject to change as evaluation planning progresses, the report will be submitted to the Department for General Assembly and Conference Management (DGACM) by its slot date of 23 March Table 5: UNHCR Evaluation Timeline, Desk review Scoping interviews Inception report Data collection tools developed Data collection in HQ and field Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Jun 23