Validation Report. Pakistan: Access to Justice Program. Independent Evaluation Department

Similar documents
People s Republic of China: Judicial Reform: Using Big Data to Improve Delivery of Justice

Diversity of Cultural Expressions

ASIAN DEVELOPMENT BANK

UNDAF Results Matrix Sri Lanka

Central Asia Regional Economic Cooperation: Working with the Private Sector in Trade Facilitation

Shireen Lateef Senior Advisor (Gender) MDB Sponsored Gender, Voice and Agency Workshop Manila, June 2014

SECTOR ASSESSMENT (SUMMARY): TRANSPORT (ROAD TRANSPORT) 1. Sector Performance, Problems, and Opportunities

NATIONAL ANTI-CORRUPTION STRATEGY

PRE-CONFERENCE MEETING Women in Local Authorities Leadership Positions: Approaches to Democracy, Participation, Local Development and Peace

Governing Body Geneva, March 2009 TC FOR DECISION. Trends in international development cooperation INTERNATIONAL LABOUR OFFICE

JICA S APPROACH TO GOOD GOVERNANCE AND ANTI-CORRUPTION. Chie Miyahara *

MARYLAND STATE RETIREMENT AND PENSION SYSTEM GOVERNANCE CHARTERS. Adopted by the Board of Trustees

Reference: CU 2017/96/DTA/CEB

KEYNOTE SPEECHES Keynote speeches.p /16/01, 10:33 AM

9. What can development partners do?

Informal debate of the General Assembly Promotion of gender equality and the empowerment of women 6 8 March 2007

OPERATIONS MANUAL BANK POLICIES (BP) These policies were prepared for use by ADB staff and are not necessarily a complete treatment of the subject.

National Cooperative Policy in Rwanda. Revised Version [1]

The Government Owned Entities Bill, 2014 THE GOVERNMENT OWNED ENTITIES BILL, 2014 ARRANGEMENT OF CLAUSES

TECHNICAL ASSISTANCE COMPLETION REPORT 1 Division: CWRD AFRM

Technical Assistance People s Republic of China: Urban Poverty Strategy Study II (Financed by the Poverty Reduction Cooperation Fund)

Executive Board of the United Nations Development Programme and of the United Nations Population Fund

SCR Local Enterprise Partnership Terms of Reference

UN VOLUNTEER DESCRIPTION OF ASSIGNMENT

Project Title: Strengthening Transparency and Integrity in the Civil Service. Project Number: Project Duration:

DPA/EAD input to OHCHR draft guidelines on effective implementation of the right to participation in public affairs May 2017

Social Dimension S o ci al D im en si o n 141

REGULATION (EU) No 439/2010 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 19 May 2010 establishing a European Asylum Support Office

POLICY SEA: CONCEPTUAL MODEL AND OPERATIONAL GUIDANCE FOR APPLYING STRATEGIC ENVIRONMENTAL ASSESSMENT IN SECTOR REFORM EXECUTIVE SUMMARY

Country programme for Thailand ( )

INTERNAL AUDIT DIVISION REPORT 2017/149

EU proposal on State-owned enterprises, enterprises granted special rights or privileges, and designated monopolies. Article x (Delegated Authority)

World Bank s Country Partnership Framework

Executive Board of the United Nations Development Programme, of the United Nations Population Fund

Report on the Implementation of the Public Information Interim Policy (November 2017 to September 2018)

WTO TRADE FACILITATION NEGOTIATIONS SUPPORT GUIDE

The Potential Role of the UN Guidelines and the new ILO Recommendation on the Promotion of Cooperatives

SIGAR SEPTEMBER. Special Inspector General for Afghanistan Reconstruction. SIGAR Audit SIGAR Audit 13-17/Health Services in Afghanistan

Project Proposal. i) Women, Peace and Security. Final draft of 9 May 2017

PHARMAC s implementation of Trans-Pacific Partnership (TPP) provisions and other amendments to application processes September 2016 Appendix two

CASE STORY ON FIJI S TRADE POLICY FRAMEWORK AID FOR TRADE CASE STORY: FIJI

Implementation of Security Council Resolution 1325 (2000) on Women, Peace and Security in Peacekeeping Contexts

SECTOR ASSESSMENT (SUMMARY): PRIVATE SECTOR AND SME DEVELOPMENT

Commission for Gender Equality (CGE) Opinion Piece: Women s Political Representation and Participation

India: Delhi Meerut Regional Rapid Transit System Project

Food and Agriculture Organisation of the United Nations (FAO) Consumer Unity and Trust Society (CUTS) International

Sanctuary and Solidarity in Scotland A strategy for supporting refugee and receiving communities

The Influence of Conflict Research on the Design of the Piloting Community Approaches in Conflict Situation Project

Women, gender equality and governance in cities. Keynote address by Carolyn Hannan Director, United Nations Division for the Advancement of Women

Adopted by the Security Council at its 8360th meeting, on

Good Governance for Economic Growth

Self-Reliance through Mutual Accountability Framework (SMAF) Second Senior Officials Meeting Kabul, Afghanistan, 5 September Co-Chairs Statement

REPORT ON INTERNAL AUDIT IN UNHCR I. AUDIT ACTIVITIES

Reviving Trade Finance for Corporation and SMEs. Presenter: Dr. David S. Hong President of TIER

JOURNALIST LEGAL AID FUND MANUAL

Overview Paper. Decent work for a fair globalization. Broadening and strengthening dialogue

RUNO ANNUAL PROJECT REPORT TEMPLATE 4.4

Governing Body 334th Session, Geneva, 25 October 8 November 2018

SEMINAR ON GOOD GOVERNANCE PRACTICES FOR THE PROMOTION OF HUMAN RIGHTS Seoul September 2004

1. IDENTIFICATION Support for Municipal Finance in Lebanon CRIS number ENPI 2011/22758 Total cost Total estimated cost: EUR

Greater Mekong Subregion: Mekong Tourism Development Project (Cambodia, Lao People s Democratic Republic, and Viet Nam)

Premise. The social mission and objectives

CORPORATE GOVERNANCE

Letter dated 20 December 2006 from the Chairman of the Peacebuilding Commission addressed to the President of the Security Council

POLICY TYPE: EXECUTIVE LIMITATIONS. June 28, 1994; March 13, 1997 (R); March 27, 2001(R); March 25, 2004 (R), March 29, 2007 (R)

Conclusions on the former Yugoslav Republic of Macedonia

UNITED NATIONS COMMISSION ON INTERNATIONAL TRADE LAW (UNCITRAL)

RESETTLEMENT FRAMEWORK. Supplementary Appendix to the Report and Recommendation of the President to the Board of Directors. on the

RIGHT TO FOOD ASSESSMENT CHECKLIST Assessing the Right to Food in the National Development Context

ENERGY SECTOR ACT. Chapter one. GENERAL

AFRICAN DEVELOPMENT BANK GROUP

Recommendation of the Council for Development Co-operation Actors on Managing the Risk of Corruption

Proxy Voting Procedures

The Government Performance and Accountability Act. The People of the State of California hereby find and declare that government must be:

I assent. AN ACT TO AMEND THE INTERNATIONAL FINANCIAL ORGANISATIONS ACT 1971

INTERNAL AUDIT DIVISION REPORT 2017/016

COMPILATION OF BACKGROUND HISTORY AND INFORMATION U.S. FEDERAL DEFENDER PROGRAM December 2005

Oxfam believes the following principles should underpin social protection policy:

MODEL CONSTITUTION OF A REGIONAL CHAPTER. The Preamble

GOVERNANCE MANUAL FOR COUNTRY COORDINATING MECHANISM (CCM), BHUTAN THE GLOBAL FUND TO FIGHT AIDS, TUBERCULOSIS AND MALARIA

Peacebuilding Commission

KAPSTONE PAPER AND PACKAGING CORPORATION. CORPORATE GOVERNANCE GUIDELINES (the Guidelines )

THEMATIC COMPILATION OF RELEVANT INFORMATION SUBMITTED BY SIERRA LEONE ARTICLE 6 UNCAC PREVENTIVE ANTI-CORRUPTION BODY OR BODIES

Iraq Mood Improving Despite Divisions General Overview January-March 2014 Survey Findings. Page 1

THE GLOBAL ECONOMIC CRISIS DEVELOPING ECONOMIES AND THE ROLE OF MULTILATERAL DEVELOPMENT BANKS

CAMBODIA SYSTEMATIC COUNTRY DIAGNOSTIC Public Engagement

Centrality of Protection Protection Strategy, Humanitarian Country Team, Yemen

MOPAN. Synthesis report. United Nations High Commissioner for Refugees (UNHCR) Multilateral Organisation Performance Assessment Network D O N O R

UNCITRAL Model Law on Cross-Border Insolvency with Guide to Enactment

NATIONAL ANTI-CORRUPTION STRATEGY POLICY PAPER

REPORT 2015/168 INTERNAL AUDIT DIVISION. Audit of the operations in Thailand for the Office of the United Nations High Commissioner for Refugees

Summary of the Indigenous Peoples' Consultation with the Asian Development Bank, November 27 th 2007

COUNTRY REPORT. by Andrei V. Sonin 1 st Secretary, Ministry of Foreign Affairs

ASIAN DEVELOPMENT BANK

Basic Polices on Legal Technical Assistance (Revised) 1

ASEAN Law Association

The Way Forward: Pathways toward Transformative Change

REPORT 2016/063 INTERNAL AUDIT DIVISION. Audit of the operations in Nepal for the Office of the United Nations High Commissioner for Refugees

EAST ASIA AND PACIFIC REGION CAMBODIA Portfolio

Indonesia: Enhanced Water Security Investment Project

Transcription:

Validation Report Reference Number: PCV: PAK 2011-52 Program Number: 32023 Loan Numbers: 1897/1898/1899(SF) December 2011 Pakistan: Access to Justice Program Independent Evaluation Department

ABBREVIATIONS ADB Asian Development Bank AJP Access to Justice Program CPLC citizen police liaison committee DMF design and monitoring framework IED Independent Evaluation Department MOL Ministry of Law, Justice, Human Rights, and Parliamentary Affairs PCR program completion report PMU program management unit TA technical assistance NOTE In this report, $ refers to US dollars. Key Words access to justice development fund, asian development bank, citizen police liaison committee, dispute resolution, independent evaluation department, judicial administration, law reform, law and justice committee, lessons, legal empowerment fund, pakistan, performance evaluation, police reforms, poverty reduction, program completion report validation Director Team leader Team members W. Kolkma, Independent Evaluation Division 1, Independent Evaluation Department (IED) R. Vasudevan, Evaluation Specialist, IED A. Anabo, Senior Evaluation Officer, IED V. Melo, Evaluation Assistant, IED The guidelines formally adopted by IED on avoiding conflict of interest in its independent evaluations were observed in the preparation of this report. To the knowledge of the management of IED, there were no conflicts of interest of the persons preparing, reviewing, or approving this report. In preparing any evaluation report, or by making any designation of or reference to a particular territory or geographic area in this document, IED does not intend to make any judgments as to the legal or other status of any territory or area.

PROGRAM BASIC DATA Program Number: 32023 PCR Circulation Date: January 2010 Loan Numbers: 1897/1898/1899(SF) PCR Validation Date: December 2011 Program Name: Loan 1897/1898: Access to Justice Program Loan 1899(SF): Institutional Development for Access to Justice Country: Pakistan Approved ($ million) Actual ($ million) Sector: Public Sector Total Program Costs: 350.00 353.79 Management (SDR equivalent) ADB Financing ADF: $106.80 million Loans: Loan 1897 Loan 1898 Loan 1899 243.90 a 86.80 b 20.00 257.76 a 92.71 b 3.31 OCR: $243.20 Borrower: 0.00 0.00 million Beneficiaries: 0.00 0.00 Others: 0.00 0.00 Cofinancier: None Total Cofinancing: 0.00 0.00 Approval Date: 20 Dec 2001 Effectiveness Date: 21 Mar 2002 24 Dec 2001 Signing Date: 21 Dec 2001 Completion Date: 31 Dec 2004 2 Jul 2009 c Program Officers: Validator: Name: D. Porter D. Porter S. Nicoll W. ul Hasan D. Kertzman H. Soliman Consultant Closing Date: Loan 1897/1898 Loan 1899 Location: Headquarters Resident Mission Resident Mission Resident Mission Headquarters Team Leader: 30 Jun 2005 30 Jun 2006 From 2001 Jan 2005 Oct 2005 Dec 2006 Jan 2008 30 Sep 2007 2 Jul 2009 To 2004 Sep 2005 Nov 2006 Dec 2007 Dec 2009 R. Vasudevan Evaluation Specialist, IED1 Quality Reviewer: N. Bestari Advisor, IEOD Director: W. Kolkma, IED1 ADB = Asian Development Bank; ADF = Asian Development Fund; IED1 = Independent Evaluation Division 1, Independent Evaluation Department; IEOD = Office of the Director General, Independent Evaluation Department; OCR = ordinary capital resources; PCR = progam completion report; SDR = special drawing rights, SF = special fund. Notes: a Equivalent to 29.685 million. b Equivalent to SDR 67.914 million. c Loans 1897 and 1898 were closed on 30 September 2007, while Loan 1899 was closed in 2 July 2009. A. Rationale I. PROGRAM DESCRIPTION 1. The Access to Justice Program (AJP) 1 sought to (i) provide security and ensure equal protection under the law to citizens, in particular, the poor; (ii) secure and sustain entitlements and thereby reduce the poor s vulnerability; (iii) strengthen the legitimacy of state institutions; and (iv) create conditions conducive to pro-poor growth, especially by fostering investors 1 ADB. 2001. Report and Recommendation of the President to the Board of Directors: Proposed Loans and Technical Assistance Grant to the Islamic Republic of Pakistan for the Access to Justice Program. Manila.

2 confidence. The AJP envisaged that vulnerability, justice, and entitlements were closely linked. Vulnerability was due to insecure access to key material, social, political, and environmental assets. Justice was defined as a function of the relationship between the institutions responsible for delivering entitlements (public goods and services) predictably, affordably, and accountably, and the citizens who needed to secure and sustain their access to key assets. The AJP posited that efforts to limit the poor s vulnerability to the vagaries of administrative, political, civil, and criminal justice were at least as important for poverty reduction as macroeconomic performance. In addition, the prevailing legal framework and the performance of judicial institutions significantly constrained market-based economic growth, hindering foreign direct investment, in particular, as well as the growth of small and medium-sized enterprises in Pakistan. B. Expected Impact 2. The expected impact was to contribute to the nationwide efforts to influence the exercise of political, administrative and judicial powers in order to improve accessibility to public entitlements for all citizens, in particular the poor, women, and minorities. It was expected to prompt public and professional debate about important social issues. The AJP intended to tackle gender issues through the creation of competent legal, judicial, and police institutions through which existing basic and constitutional rights could be implemented fairly and accountably. The AJP was expected to promote research and debate on the quality and equality of existing laws through the reformed Law Commission, as well as the Women's Commission, and help build constituencies for change and elicit consensus for reform. C. Objectives or Expected Outcomes 3. The AJP s key development objective was to assist the government in improving access to justice by supporting five inter-related governance objectives: (i) providing a legal basis for judicial, policy, and administrative reforms; (ii) improving the efficiency, timeliness, and effectiveness of judicial and police services; (iii) supporting greater equity and accessibility to justice services to the vulnerable and poor; (iv) improving predictability and consistency between fiscal and human resource allocations and the mandates of reformed judicial and police institutions at the federal, provincial, and local levels; and (v) ensuring greater transparency and accountability in the performance of the judiciary, the police, and administrative justice institutions. 4. The program was expected to contribute to the government s efforts to transform the performance of the judiciary and the police. It was anticipated that the citizenry would regard these institutions as important to the assertion and protection of their rights, while the poor would feel confident about resorting to them. Although the benefits of the improved rule of law were indispensable to improving economic performance, the AJP targeted a decrease in the vulnerability of the poor by (i) alleviating the vulnerability arising from everyday harassment, underperformance, exclusion, and the denial of their basic rights by public officials; (ii) easing the vulnerability of informal sector activities to rent-seeking activities by officials responsible for licensing and regulating the people s access to economic and environmental assets; (iii) improving the accountability of public officials; and (iv) enhancing the accessibility and affordability of public goods and services on which the poor rely disproportionately. D. Components and Outputs 5. The program was expected to deliver benefits in three components policy, institutions, and budget and expenditure management. Actions in the first component, i.e., policy and legal

3 provisions, were to secure (i) greater judicial independence, transparency, and accountability; (ii) better citizen-state relations; and (iii) more durable public institutions responsible for the delivery of justice. Actions in the component on institutional performance were to result in (i) greater judicial independence, fostering the predictable, timely, and impartial implementation of the rule of law to boost investor confidence; (ii) a decline in political interference in police services; (iii) enhanced institutional arrangements for holding public servants and officials accountable for violations of citizens rights and entitlements; and (iv) greater public engagement in planning and monitoring the performance of judicial and police service delivery. Actions in the third component budget and expenditure management aimed at creating consistency between allocations of human and fiscal resources and institutional responsibilities for the delivery of justice at federal, provincial, and local levels. 6. The AJP focused on judicial reforms (including those reforms relating to administrative justice institutions) and police reforms. Key outputs were produced under the program under five categories (i) providing a legal basis for judicial, policy, and administrative reforms; (ii) increasing the efficiency, timeliness, and effectiveness of judicial and police services; (iii) improving equity and accessibility to justice services for the poor; (iv) improving predictability and consistency between fiscal and human resource allocations and mandates of reformed judicial and police institutions; (iv) enhancing transparency and accountability in the performance of justice sector agencies. E. Provision of Inputs 7. The program was approved in December 2001, and comprised two loans in the amount of $330 million, a technical assistance (TA) loan for $20 million, and a TA grant for $900,000. The capacity development activities were designed to be accomplished through the TA loan and grants. The loan funds were to be disbursed in four tranches. The first tranche of $63.2 million was to be released upon loan effectiveness. An incentive tranche of $25 million was to be released on 30 June 2002. The second tranche of $75 million was scheduled for release in June 2003, and the final tranche of $80 million was due for release on 31 December 2004. The financial terms thus built in a marginally increasing incentive to promote sustainability. F. Implementation Arrangements 8. The Ministry of Law, Justice, Human Rights, and Parliamentary Affairs (MOL) was the executing agency. The Program Management Unit (PMU) was established in the MOL. The program completion report (PCR) adequately described the complexity of the implementation arrangements between different branches of government (judiciary and the executive branch) and across different layers of implementation (federal, provincial, and district offices). The AJP s initial design did not provide for PMUs at the provincial level; however, four provincial PMUs were set up subsequently. The PCR also noted a high degree of compliance with the loan conditions, with the exception of the setting up of citizen police liaison committees (CPLCs), which was waived due to a lack of ownership. In all, there were 64 policy actions: 58 of these were fully complied with, 5 were substantially complied with, and 1 was waived. 9. The TA grant of $900,000 was meant to support the PMU, and to develop and implement independent monitoring and evaluation systems. Subsequently, the TA grant amount was increased to $1.74 million. The procurement and consultants performance were satisfactory. Under the TA loan, 55 national consultants were engaged for a total of 524

4 person-months; under the TA grant, 29 national consultants were engaged to support the implementation of the AJP. The performance of consultants was satisfactory. II. EVALUATION OF PERFORMANCE AND RATINGS A. Relevance of Design and Formulation 10. The realization of the vision for a strong legal foundation that provided predictable rules, allowing the poor and disadvantaged to participate in their legal entitlements, remained elusive. The basic law was suspended in 1977 and 1999 by military governments, and replaced with provisional constitutions under the doctrine of necessity. In 2000, the Supreme Court again validated the fourth military intervention, allowing a Legal Framework Order to serve as the basic law of the land (PCR, p. 2). Prior to 2000, eight judicial commissions recommended changes in the substantive and procedural law, but these were only partially implemented because law and justice reforms were never a major part of the policy agenda. Poverty was increasingly equated with growing insecurity and vulnerability arising from sectarian violence, communal clashes, and declining law and order (PCR, p. 3). 11. Given this backdrop, the PCR noted that the Asian Development Bank (ADB) entered the law and justice sector in 1997 with a TA project (TA 2979) 2 at the request of the then Minister of Law, Justice, Human Rights, and Parliamentary Affairs. After 6 months, the project was followed-up by a larger TA project (TA 3015). 3 Law and justice reform was consistent with ADB s recognition that a pro-poor legal and institutional framework was essential for economic development. The AJP s innovative features included (i) proceeds from the AJP program loans being earmarked to increase budgetary allocations for key implementing agencies; (ii) the use of 60% of counterpart funds generated from policy loan proceeds for capital works to improve judicial and police facilities; and (iii) the government s agreeing to monitor the use of these funds regularly and to conduct a performance audit during the AJP s implementation. 12. The AJP was relevant at the appraisal stage, and remained so. The long-term objectives of the AJP and its outcomes were consistent with the strategic objectives of ADB s country operations strategy and the government s development plan. While the design was innovative and comprehensive, as noted in the PCR, it turned out to be too ambitious based on its broad scope and formidable implementation risks (PCR, p. 15). Intervening in two broad areas of reform proved too difficult, due to the various layers of changes that had to be managed, especially in the police sector. The fact that the police force was a provincial function should have been known from the very beginning. This oversight, combined with the fact that no PMUs were envisaged at the provincial level, shows that the program design had some information gaps and shortcomings. There was insufficient analysis of the possible risks and necessary risk mitigation measures. Also, steps to strengthen the capacity of government institutions were provided in the TA loan, but were not fully implemented. Despite these shortcomings, which have been acknowledged by the PCR, this validation concurs with PCR s assessment of relevant. B. Effectiveness in Achieving Program Outcomes 13. The PCR concluded that the program was effective in improving access to justice (PCR, n. 5), but that it was less effective in improving police performance and accountability. This 2 ADB. 1997. Technical Assistance to the Islamic Republic of Pakistan for Strengthening Government Legal Services and Subordinate Judiciary, Manila (TA 2979-PAK, for $150,000, approved on 31 December) 3 ADB. 1998. Technical Assistance to the Islamic Republic of Pakistan for Legal and Judicial Reform Project. Manila (TA 3015-PAK, for $995,000, approved on 7 May)

5 validation rates the overall effectiveness of the program loan as less effective because an important component of the program could not be delivered as intended. Due to the limited use of TA loans, capacity development could not be undertaken as envisaged in the AJP. Further, there was no program-wide monitoring and implementation system, and the PCR could not provide supporting evidence against the DMF indicators. This is a major factor that conceals actual AJP performance. The rating on effectiveness is less effective. 14. The PCR listed a number of the program s achievements. These included judicial and police reforms, starting with the amendment of the Law Commission Ordinance of 1979 to expand the composition and mandate of the country s Law and Justice Commission. It also expanded the mandate in the field of judicial administration to recommend improvements in legal education standards; administer and manage the Access to Justice Development Fund, 4 an endowment fund created under this program loan; prepare schemes for access to justice and legal aid; and develop human resources for court administration and case management purposes. The National Judicial Policy Making Committee was created to promote a uniform judicial policy and provide leadership in the sector. Judicial independence was strengthened by separating the judiciary from the executive and giving only the judiciary the mandate to try crimes. A framework was also institutionalized to ensure the judiciary s capacity to prepare and negotiate budgets with the Department of Finance. And that budgetary allocation increased both in real terms and as a percentage of federal and provincial budgets. 15. The program contributed to improvements that were made in the case-flow management reforms, reducing the age of cases from 15 years to as low as 3 years for some district courts. The Supreme Court was also able to cut its backlog from 80,000 cases to 19,055 cases between 2005 and 2008. Legislation to introduce small causes court (handling claims involving up to PRs100,000 and minor criminal offenses) was adopted. The Freedom of Information Ordinance 2002 was enacted to increase accountability, but it included a number of exemptions and applied only to the federal government. Limited progress was recorded in the area of judicial education and training. Although the Federal Judicial Academy developed training handbooks and adopted a 10-module training curriculum, the judiciary has not yet finalized and signed-off on the training requirements for the courts. 16. The PCR concludes that the reforms for professionalizing the police and strengthening police accountability proved inadequate (PCR, p. 9). The Police Order of 2002 was promulgated under the AJP, which mandated the transformation of the police force into a more serviceoriented and depoliticized organization. The law, however, was ineffective because under the constitution the police force was a provincial function and resistance to many provisions of the law at the provincial level was considerable. Moreover, only a few public safety and police complaint commissions have been fully functional, which indicates a lack of ownership of this reform program. Also, the police order envisioned the creation of CPLCs to develop liaison and coordination mechanisms, and to provide relief, where warranted. The experience with the CPLCs was limited, largely because they were established only in some capital cities and districts. The program acknowledged that this reform measure could not be legislated. Consequently, this tranche condition was waived. C. Efficiency of Resource Use in Achieving Outputs and Outcomes 17. This validation concurs with the PCR that the AJP, as a whole, was less efficient. The major reasons for this rating were the 5 year delay in implementation, and the fact that 4 This was an extra-budgetary facility and proceeds were intended to provide a sustainable source of funding for subordinate courts, educational and research institutions, bar associations, and the Federal Judicial Academy.

6 the policy component was not fully delivered. Implementing agencies faced tremendous capacity constraints. The AJP had anticipated some difficulties and provided for the TA loan to make up for the deficiency, but the TA loan was not fully utilized by the MOL. The PCR adequately described the complexity of the implementation arrangements between different branches of government (judiciary and the executive branch) and across different layers of implementation (federal, provincial, and district offices). The AJP s initial design did not provide for PMUs at the provincial level; however, four provincial PMUs were set up subsequently. The PCR also noted a high degree of compliance with the loan conditions, with the exception of the setting up of the CPLCs; this condition was waived due to a lack of ownership. In all, there were 64 policy actions: 58 of these were fully complied with, 5 were substantially complied with, and 1 was waived. 18. The two program loans amounted to $330 million. The PCR noted delays in the disbursement of the second and third tranches (18 months for each) due to the difficulty experienced in complying with the program loan s policy conditions. Counterpart funds generated from loan proceeds were used to provide a $25-million endowment for the Access to Justice Development Fund. The PCR noted the low utilization of the TA loan ($3.3 million or 16.6%) in support of institutional development, despite the additional 3 years it was kept open. The PCR added that the release of all but the first tranche was delayed because of difficulties in completing policy actions, many of which needed to be implemented at both the federal and provincial levels. The final tranche was released on 28 August 2007, 18 months later than originally scheduled. The AJP closed after nearly 8 years, instead of the 3 years initially planned. The TA loan funded 85 technical investment proposals, of which 15 were complete and 70 were still being implemented. D. Preliminary Assessment of Sustainability 19. This validation agrees with the PCR s less likely sustainable rating of the outcomes. The Chief Justice of Pakistan reiterated the judiciary s strong commitment to the reform agenda; however, capacity constraints and poor infrastructure are likely to impede sustainability. In the police sector, the reforms are less likely to be sustained, due to conflicting views on reform and weak coordination among the stakeholders. E. Impact 20. The PCR rated the AJP s impact as significant. Although various changes were made in both the judiciary and the police sectors, the mandate for judicial training still remains unclear. The politicization of the police sector remains prevalent, and the attitude of the provincial governments toward the reforms is still lukewarm. For this reason, this validation rates the impact as moderate. III. OTHER PERFORMANCE ASSESSMENTS A. Performance of the Borrower and Executing Agency 21. The PCR notes that the performance of the borrower and the executing agency was partly satisfactory. This validation concurs. The MOL lacked experience in handling a complex program that had two major components (judicial and police reforms) and covered various government agencies over different layers of implementation (federal, provincial, and district). Provincial PMUs were neither provided for in the design nor established. It was only after experiencing sluggish performance in the implementation of the AJP that these units were created. Difficulties in coordination between the federal PMU and the provincial PMUs resulted

7 in delays. The MOL and implementing agencies were unable to develop an effective baseline monitoring system, or to collect and monitor the status of reforms. B. Performance of the Asian Development Bank 22. The PCR notes that ADB s performance was partly satisfactory. This validation concurs. In the initial 4 years of the program, the ADB resident mission was able to play a facilitative role in substantial policy dialogue. The PCR notes that ADB s responses to the government were generally timely, and approvals and disbursements were undertaken promptly. Later, however, the absence of international staff in the resident mission constrained ADB s ability to continue substantial policy dialogue. Consultants were hired under the various TA projects, but this did not make up for the lack of ADB staff. The PCR notes that administrative inefficiencies also hindered the work, among them the failure to properly file some consultants reports and to widely disseminate them. C. Other 23. The program helped mainstream gender reforms into the justice sector. MOL partnered with the Ministry of Women Development to implement the national Gender Reform Action Plan. The number of women judges in different courts nearly doubled between 2001 and 2007. Greater awareness of the rights of women and human rights was incorporated into police training programs and was supported by statements from senior Ministry of Interior officials. IV. OVERALL ASSESSMENT, LESSONS, AND RECOMMENDATIONS A. Overall Assessment and Ratings 24. The program was consistent with ADB s country strategy and the government s development goals. It was innovatively conceived and designed to have a broad impact on the poor, the vulnerable, and women. However, there were design limitations that precluded achievement of the full benefits. Foremost among these was that the AJP addressed two large, difficult, and structurally incompatible sectors, the judiciary and police. Understanding of the police sector was especially deficient, as is clear from the fact that it was only during implementation that the AJP learned that policing was a provincial function. PMUs at the provincial level were not envisaged in the beginning. Likewise, no monitoring and implementation arrangements were devised to track and report progress. During implementation, the TA loan, which was supposed to augment capacity on the ground, was not fully utilized. This, in turn, resulted in slower and more-drawn-out implementation. This limited capacity also compromised the AJP s efficiency and sustainability. For these reasons, this validation agrees with the PCR s overall rating, based on the integrated program framework, as partly successful. Overall Ratings Criteria PCR IED Review Reason for Disagreement/Comments Relevance Relevant Relevant Effectiveness in achieving outcome Efficiency in achieving outcome and outputs Preliminary assessment of sustainability Effective Less effective Poor performance in the area of police reforms, limited use of TA loan, and limited provision of evidence to support claims of effective performance in the PCR. Less efficient Less likely sustainable Less efficient Less likely sustainable

8 Criteria PCR IED Review Reason for Disagreement/Comments Borrower and executing Partly Partly agency satisfactory satisfactory Performance of ADB Partly satisfactory Partly satisfactory Impact Significant Moderate Significant impact cannot be justified, given the lack of supporting evidence in the PCR. Overall assessment Partly successful Partly successful Quality of PCR Satisfactory ADB = Asian Development Bank, IED = Independent Evaluation Department, PCR = program completion report. B. Identification of Lessons 25. This validation agrees with the lessons enumerated in the PCR. The lessons were that the AJP was too centrally designed and managed, supply-driven, and complex. The police reforms also lacked a well-grounded agenda, a clear mandate, and capacity on the part of ADB and the government to implement reforms. This portion of the PCR also noted the difference between the integrated outcomes of the DMF, on the one hand, and the separated outcomes of the policy matrix. C. Recommendations for Follow-Ups 26. This validation agrees with the PCR s recommendation that due to the complexity of dealing with the executive and the judicial branches of government at the same time, the separate management of future programs and programs involving both branches of government may be more appropriate. V. OTHER CONSIDERATIONS AND FOLLOW-UP A. Monitoring and Evaluation Design, Implementation, and Utilization 27. The PCR refers to the design in paras. 64 and 66. It appears that there was no systematic compilation of information to support the performance indicators in the program framework, thereby making it difficult to determine whether the results were achieved. B. Comments on Program Completion Report Quality 28. This validation rates this PCR satisfactory. The PCR has provided adequate information on the program s background and on some of its accomplishments in the judicial and police areas. However, there could have been a tighter coherence in reporting accomplishments in terms of the DMF and the program framework. C. Data Sources for Validation 29. The data sources included the PCR, the report and recommendation of the President, program files, and other related documents. D. Recommendation for Independent Evaluation Department Follow-Up 30. This validation recommends that a project performance evaluation report be undertaken as soon as is convenient, to fill in the data gaps in the achievement of the outcomes, as outlined in the program framework. It also recommends that reports on financial expenditures be updated.

REGIONAL DEPARTMENT S RESPONSE TO THE PROGRAM COMPLETION REPORT VALIDATION REPORT On 14 June 2010, the Independent Evaluation Department (IED) circulated the draft validation report for interdepartmental comments. IED received comments from the Central and West Asia Department (CWRD) on 30 June 2010. IED incorporated the relevant comments into the final draft and sent it back to CWRD on 12 May 2011. CWRD reviewed the final report, agreed with the changes made by IED, and had no further comments to offer.