New Empirical Frontiers in Measuring and Evaluating Governance: Illustrations and Issues for Discussion Profiles in Democracy and Governance Daniel Kaufmann and Francesca Recanatini The Carter Center, Oct. 30 th, 2001 www.worldbank.org/wbi/governance
What is Governance?: A working definition for public governance Governance is the process and institutions by which authority in a country is exercised: (1) the process by which governments are selected, held accountable, monitored, and replaced; (2) the capacity of governments to manage resources efficiently, and to formulate, implement, and enforce sound policies and regulations; and, (3) the respect for the institutions that govern economic and social interactions among them
The Dividend of Good Governance 90 80 70 Infant Mortality and Corruption 12,000 10,000 Per Capita Income and Regulatory Burden 60 8,000 50 40 6,000 30 4,000 20 10 2,000 0 0 Weak Average Good Weak Average Good x Development Dividend Control of Corruption x Development Dividend Regulatory Burden 100 Literacy and Rule of Law 10000 Per Capita Income and Voice and Accountability 9000 75 8000 7000 6000 50 5000 4000 25 3000 2000 1000 0 0 x Development Dividend Weak Average Good Rule of Law x Development Dividend Weak Average Strong Voice and Accountability Note: The bars depict the simple correlation between good governance and development outcomes. The line depicts the predicted value when taking into account the causality effects ( Development Dividend ) from improved governance to better development outcomes. For data and methodological details visit http://www.worldbank.org/wbi/governance.
Diagnostic Tools Key Questions Why is it necessary to measure governance and corruption? Which empirical tools we can use? What kind of surveys and diagnostics? Which methodological requirements? How can we use this empirical information?
Operationalizing Governance: Unbundling its Definition into Components that can be measured, analyzed, and worked on Each of the 3 main components of Governance Definition is unbundled into 2 subcomponents: Voice and Accountability Political Stability and lack of Violence Quality Regulatory Framework Government Effectiveness Control of Corruption Rule of Law
How can we measure each of the Governance components? Using existing databases and sources through aggregation and clustering Cross country aggregate indicators: for ex. Kray, Kaufmann and Zoido (1999) Gathering new detailed information through surveys and diagnostics In-depth analysis of the institutional vulnerabilities of a country
Main Features of the Aggregate Governance Indicators Coverage: 155 countries for 1997-98 (2001 forthcoming) Compiled using 14 separate sources and more than 300 indicators Techniques and coverage used by each source are different (for ex. polls, surveys, expert ratings, etc.) Clustered into 6 broad categories of governance
2 1.5 1 0.5 0-0.5-1 Control of Corruption: Aggregate Indicator (selected countries from 155 worldwide, for illustration, based on 1998 research data*) Corruption Level Margin of Error -1.5-2 POOR Tajikistan Myanmar (Burma) Azerbaijan Ukraine Kazakhstan Papua New Guin* Indonesia Pakistan Georgia Korea, North Vietnam Bangladesh China Philippines Thailand Mongolia Korea, South Mauritius Taiwan Belgium Chile Hong Kong Singapore Denmark GOOD Source: Governance Matters, PRWP 2196 by Kaufmann, Kraay and Zoido. http://www.worldbank.org/wbi/governance/
High Index of Rule of Law Application Application of Rule of Law Varies by Region (based on aggregation of surveys/polls 1997-99*) Low Index OECD East Asia Middle East South Asia Eastern Europe Latin Sub- America Saharan Africa Former Soviet Union Note: Regional Averages shown for discussion purposes, and hide large intra-regional variation in each case. Thin vertical line reflects estimated margin of error. Source: Kaufmann, Kraay and Zoido-Lobaton (1999). "Aggregating Governance Indicators" and "Governance Matters" PWRP 2195 and 2196. http://www.worldbank.org/wbi/governance
DATA MATRIX SYNTHESIS ATTRIBUTES RATING: DATA SOURCE 1. Comparative Index of Int'national Corruption 2. Opinion Surveys/ Citizen Poll 3. Enterprise Surveys 4. Detailed Governance-A-C Diagnostic (GAC) 5. Report Card/ /Rapid Monitoring 6. Qualitative Indepth Interviews 7. Oficial Data / Statistics 8. Monitoring Budget Use / Budget Tracking 9. Price Benchmarking /Price Comparisons 10. Judicial Investigations 11. Video Techniques / Sting" 12. Worldwide Aggregate Governance Indicators Conceptual Framework Quality of governance indicators MEDIUM Cross- Country Comparability MEDIUM/ HIGH GOVERNANCE DATA SOURCES AND ATTRIBUTES: A SYNTHESIS MATRIX Degree of Objectivity in Measurement Precision in Measurement Periodic Monitoring capacity within a country Quality of info. on agency or specific subject matter Breadth of scope within Governance Identifies Crossgeneric governance challenge Institutio- nal/multi- Agency Usefulness in country Usefulness for prioritizing reforms Cost Effectiveness Effectiveness for Judiciary Reforms LOW LOW MEDIUM NONE NONE LOW NONE NONE MEDIUM NONE MEDIUM LOW LOW LOW MEDIUM LOW LOW LOW LOW LOW LOW NONE MEDIUM /HIGH MEDIUM MEDIUM MEDIUM MEDIUM/ HIGH HIGH MEDIUM MEDIUM MEDIUM MEDIUM/ HIGH HIGH MEDIUM MEDIUM LOW/ MEDIUM HIGH LOW LOW LOW/ MEDIUM MEDIUM MEDIUM MEDIUM LOW LOW LOW/ MEDIUM MEDIUM/ HIGH LOW HIGH HIGH HIGH HIGH MEDIUM MEDIUM HIGH LOW MEDIUM HIGH MEDIUM/ HIGH HIGH MEDIUM/ HIGH LOW LOW HIGH HIGH HIGH LOW LOW MEDIUM LOW LOW HIGH MEDIUM LOW MEDIUM LOW LOW LOW LOW NONE LOW LOW HIGH MEDIUM MEDIUM LOW/ MEDIUM LOW MEDIUM MEDIUM MEDIUM LOW/ MEDIUM HIGH NONE MEDIUM MEDIUM/ LOW LOW MEDIUM LOW LOW LOW LOW LOW NONE NONE LOW LOW NONE NONE LOW/ MEDIUM NONE NONE HIGH LOW HIGH NONE NONE LOW HIGH NONE NONE MEDIUM HIGH NONE NONE NONE HIGH NONE NONE LOW HIGH HIGH HIGH MEDIUM MEDIUM/ HIGH LOW/ MEDIUM MEDIUM/ HIGH HIGH HIGH LOW/ MEDIUM Note: Ratings of attribute quality for each type of data source is preliminary. Desirable attributes are in columns. Color scheme follows a traffic lights approach from gray (no relevance)/dark red (poor) to dark green (very good). HIGH MEDIUM/ HIGH MEDIUM
Salient Features of the New Governance and Anti-Corruption Diagnostic Tools Multi-pronged pronged: : households, firms and public officials [ triangulation ], complemented by hard data. Experiencial (vs. (vs. opinions /generic) and closed questions Measuring Grand Corruption: State Capture and Kickbacks Towards Cardinal Measurement, away from subjectivity Conceptual framework: Focus on Incentive Structure Rigorous technical requirements for implementation Multidimensionality of Governance: Input for Action
Parliaments Misgoverned or Honest Institutions? 100.0 % who believe that Parliament is corrup 35/35 90.0 80.0 15/22 42/42 30/30 35/36 70.0 17/20 34/42 60.0 12/22 32/43 50.0 27/36 40.0 30.0 8/32 20.0 Cambodia Latvia Georgia Honduras Argentina Romania Paraguay Slovakia Peru Columbia Ecuador Rank of Parliament within country, by households Relatively good Relatively bad Source: WBI diagnostics and survey data; various countries; 1992-2001 Note: The chart shows percentage of respondents reporting that Parliament is dishonest institution. The number at the top of each bar reflects the ranking of Parliament relative to other, pre-defined government institutions.
Administrative and Judicial Corruption Very high 7 6 5 4 3 2 1 Ecuador Venezuela Mexico Argentina Colombia Peru Chile Singapore Administrative Corruption Corruption in the Judiciary
Corruption: Comparing Honduras, Peru, Colombia and Ecuador Corruption Index (KKZ, 1999, scaled) TI Corruption Index (scaled and inverted) % of firms reporting bribes are frequently used to obtain public contracts % of users asked to pay a bribe while trying to obtain a public service % of firms asked to pay a bribe while trying to obtain a public service % of public officials reporting corruption in public administration is frequent % of public officials reporting purchase of positions in their institutions % of public officials reporting public funds mismanagement is frequent Peru Colombia Honduras Ecuador 0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0
Romania: Dis-proportionally Negative Impact on the Poor % of Hh.Income in bribes 15% 10% 5% 50% 40% 30% 20% 10% % Disuaded from Medical Services 0% low income medium income high income 0% low income medium income high income
Corruption acts as a regressive tax, and small firms pay more in bribes % of gross monthly revenues paid in bribes 10 9 8 7 6 5 4 3 2 1 0 Ecuador Peru Honduras Small Medium Large
% of respondents reporting that purchasing positions is very common 60 50 40 30 20 10 0 Purchasing positions is more common within some agencies than others (selected government agencies, a Latin American country, 2001) EsSalud Colegios Ministerio del interior Hospitales Superiors Collegues at the same level Subordinates Universidades Publicas Instituto Nacional Penitenciario INPE Gobiernos locales
Extent of "State Capture"/ Undue Influence by the Elite to Shape Laws in Peru, Honduras, and Columbia (as reported by enterprises, 2001) To Parliamentarians to influence laws To Municipal authorities/councils To officials influencing ministerial/presidential decisions To Regulatory agencies To Judiciary Bribes to officials of Central Bank Peru Honduras Columbia 0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 % of firms responces
Use of alternative dispute resolution mechanisms 50 % of Alternative Mechanisms used by users 40 30 20 10 0 Through a private lawyer Through direct negotiation Through a High Level Official Through a Person in Police Use of Threats or Force Peru Ecuador Honduras NGO
Which Public Sector Characteristics do influence Determinants of Corruption? Internal Transparency Meritocracy Social Involvement and collective action Quality of norms and rules Accountability & citizen voice Rule application and supervision Salary Satisfaction Agency Missions Politicizing Others
Citizen Voice and Access to Public Services by the Poor (Bolivia illustration, each observation is a public agency) 100 Accessibility to the Poor 80 60 40 20 r = 0.54 Controlled Causal Link 0 10 20 30 40 50 60 70 80 90 Voice / External Accountability Based on Public Officials Survey. The sample of institutions includes 44 national, departmental, and municipal agencies which are a prior anticipated to be accessible to the poor
Citizen Voice Helps Control Bribery in LAC country 50 40 Bribery 30 20 10 Low Moderate_Low Moderate_High High Voice / External Accountability Simple Average Association Control Causal Link Margin of Error Based on 90 national, departmental, and municipal agencies covered in the Bolivia Public Officials Survey.
Transparency within Government Agencies Prevents Purchase of Public Positions 18 15 Job Purchase 12 9 6 3 Low Moderately Low Moderately High High Internal Transparency Simple Average Association Control Causal Link Margin of Error Based on 90 national, departmental, and municipal agencies covered in the Public Officials Survey.
Which Governance Characteristics Matter for Agency Performance? Results from Bolivia Public Officials Governance Determinant of Performance Simple Unconditional Relationship Conditional/Causal Relationship Ethical Values 0 0 Agency Autonomy ** 0 Enforcement of Rules ** 0 Quality of Rules * 0 Wage Satisfaction ** 0 Politicization ** ** Internal Transparency ** ** External Voice ** ** Corruption ** **
There are more than enough Challenges Strengthening Survey Data: 1) from general opinions/perceptions to results based on experiences; 2) quantify; 3) triangulate; 4) measure margin of error Understand thoroughly causes of corruption Confront Empirically: Capture/Grand Corruption Complement with extra hard data: which? Complement with qualitative information: which? (focus group, studies, etc) How to go from information to action?
Strategy for Good Government and Anticorruption Checks and Balances: Checks and Balances: Independent and effective judiciary Decentralization with accountability Accountability of Political Leadership: Disclosure of parliamentary votes Transparency in party financing Asset Declaration, Conflict of Interest Rules Civil Society Oversight: Freedom of information Public hearings of draft laws Monitoring by media/ngo s Good and Clean Government Competition & Entry : Competitive restructuring of monopolies Regulatory simplification Public Administration and Public Finance: Meritocratic civil service Transparent, monetized, adequate remuneration Accountability in expenditures (Treasury, Audit, Procurement)
Governance Improvement in Practice: Differentiated Strategies in Five Countries Institutional and Civil Service Reform: Transparent Privatization in Ecuador Salary Reform in Bolivia Disclosure of Assets in Bolivia and Georgia Economic Policy: Deregulation and Delicensing in Bolivia Revamping Customs in Ecuador and Latvia Tax Administration in Latvia and Ecuador Tax/Customs Code Reform in Albania Corruption Financial Controls: Procurement Reform in Bolivia, Ecuador and Georgia Legal-Judicial: Revamp of Supreme Court in Bolivia Competency test and firing of judges in Georgia, Albania and Latvia Mediation in Albania Public Oversight and Civil Society: Ombudsman in Albania and Bolivia Private Sector Coalition in Bolivia Diagnostic Surveys and NGO monitoring in Ecuador and others