Community Access To Justice And Conflict Resolution In Aceh And Maluku

Transcription

1 Communty Access To Justce And Conflct Resoluton In And -- Baselne Quanttatve Survey of the Medaton and Communty Legal Empowerment program n and, and the Conflct Resoluton and Medaton Tranng program n. DRAFT October,

2 Executve Summary The Medaton and Communty Legal Empowerment (MCLE) program s an access to justce program whch ams to strengthen and mprove farness of dspute resoluton processes and ncrease publc trust n and access to the formal legal system. As a part of the Indonesan Government s Support for Poor and Dsadvantaged Areas project, the MCLE wll operate n the provnces of and. Quanttatve research on access to justce ssues was conducted n those provnces as a part of a baselne nvestgaton for a mxed method mpact evaluaton of the program. Parts of ths qualtatve research are also beng used as a baselne for evaluatng a program of Conflct Resoluton and Medaton Tranng (CRT) for local leaders and project facltators n. MCLE and CRT Program Desgn The MCLE program ams to acheve the objectves above by establshng a network of communty legal ad and medaton posts and conductng awareness rasng and empowerment actvtes. Each partcpatng vllage wll have one man and one woman traned as volunteer paralegals, supported by a full-tme pad coordnator at the subdstrct level, and a part-tme pad communty lawyer at the dstrct level. Medaton tranng wll be provded for respected local fgures such as tradtonal (adat) leaders. The program wll tran around 384 paralegals n 24 subdstrcts over the two provnces. The program wll be mplemented by one NGO n each of the two provnces, contracted by the Mnstry for Dsadvantaged Areas. The two MCLE project provnces, and, fall wthn the ambt of the Support for Poor and Dsadvantaged Areas project because they are both remote, located at extreme ends of the archpelago, and both are recoverng from serous conflcts whch have set back development sgnfcantly. has been further dsadvantaged because of the severe loss of lfe and damage done to nfrastructure durng the Asan Tsunam of The CRT project s operatng only n, and ams to buld capacty of communtes to manage conflct and medate dsputes, partcularly n the context of the government s PNPM development program and the Helsnk post cvl war Memorandum of Understandng. Around 408 vllage leaders from 100 vllages n 20 subdstrcts wll be traned, along wth around 400 PNPM program facltators n 90 subdstrcts. The Evaluaton Ths baselne study s part of an evaluaton whch wll use a rgorous combned qualtatve and quanttatve approach to examne the mpacts of an access to justce program on human rghts and economc welfare. The MCLE evaluaton s based on sx key lnes of enqury (detaled n the methodology secton) desgned to determne mprovements for program benefcares n terms of economc welfare, enjoyment of rghts, safety and securty, decrease n corrupton, as well as engagement wth the state justce sector. From these, a seres of detaled questonares was developed. 2

3 The evaluaton nvolves samples from areas whch receve each of three treatments. Treatment One comprses vllage paralegals and subdstrct legal ad posts; Treatment Two has subdstrct legal ad posts only; and Control areas are outsde treatment subdstrcts (but wthn the same larger dstrct). Both treatments were employed n, however due to concerns about sample sze and budget, only Treatment One was sampled n. Wthn the above constrants, treatment and control locatons and samples were assgned based on a combnaton of randomsaton and probablty proportonal to sze samplng, explaned n Appendx 1. The quanttatve evaluaton comprses ths baselne survey, to be followed n around two years by a follow up survey conducted n the same locatons, and where possble nvolvng the orgnal households (a partal panel desgn). Ths wll allow longtudnal analyss of changes over tme n ndcator values wthn populaton groups (vllages) and n ndvdual households, whch wll help compensate for ntal varaton between treatment and control locatons. Three separate survey nstruments were admnstered. The Household Survey comprses a set of questons to gather standard demographc and socoeconomc (welfare) nformaton along wth ndcators of access to justce, gender equalty, perceptons of farness and corrupton n nformal dspute resoluton and formal justce sector nsttutons. In the ndvdual component of the Household Survey, half the respondents were male, and half female, allocated randomly among the chosen households. Vllage leaders both formal and nformal, and ncludng women, were asked to partcpate n a Key Informant Survey whch asked smlar questons, desgned to take advantage of leaders enhanced awareness of communty ssues, but also to determne ther pont of vew as dspute resoluton actors. Fnally, a Communty Survey nvolvng leaders and dspute resoluton actors was admnstered n a group ntervew settng, allowng for consensus reples to answers, poolng of knowledge and also wth the possble beneft of reducng self-nterested bas n answers by ndvdual vllage leaders. The fnal chapter of ths report analyses the degree of smlarty between varous baselne measures n treatment and control vllages. Overall the randomsed selecton of locatons has succeeded n mnmsng dfferences n observable dmensons. Where sgnfcant dfferences have been observed, they can be dealt wth durng the follow up evaluaton by usng double dfferences analyss. Legal Awareness The survey found a generally low level of legal awareness, consstent wth prevous studes elsewhere n Indonesa. Margnalzed groups have the lowest levels of legal awareness. When vllage leaders sought nformaton on behalf of members of ther communty, they dd so mostly from the formal justce sector n, but mostly from local government fgures n, perhaps reflectng bad experences wth polce and prosecutors n. NGOs ncludng legal and paralegal servces were also a source of nformaton n both (16%) and (9%), whch s a hgher fgure 3

4 than expected from prevous studes, and may reflect the actve role of cvl socety n post-conflct and post-dsaster stuatons. Women s Rghts Atttudes towards women s rghts depend on the type of rght n queston. In matters of land ownershp and nhertance, the majorty beleve that land should be owned jontly n matrmony, and that women are enttled to a share of ther deceased parent s estate. Ths stuaton s conducve to women exercsng ther economc rghts, especally for unmarred women and dvorcees. On the other hand, the majorty feel that a husband should have complete authorty over hs spouse wth only a small mnorty agreeng that the communty sometmes has responsblty to ntervene n the household matters of others. Legal Documents Legal documents are consdered a prerequste to provng or exercsng certan rghts such as holdng prvate property and accessng government servces. The degree to whch people possess these documents was surveyed, and fndngs dffered accordng to several factors ncludng gender, level of educaton, and remoteness from government centers. In the case of ID cards, almost everyone n holds a natonal card for hstorcal reasons relatng to the mposton of mltary rule, whle n the majorty do not. Men are more lkely than women to hold ID cards, urban dwellers more lkely than rural, and hghly educated people more lkely to possess offcal ID than people wth basc or no educaton. Smlar dfferences based on gender, remoteness and educaton were found for brth, marrage and dvorce documentaton. Offcal land certfcates can be crucal evdence durng ownershp dsputes, and whle the majorty n do possess documentaton for ther resdental and farmng land, the reverse s true n, where only a quarter of people hold wrtten proof of ownershp of ther agrcultural land. Dspute types Overall dspute occurrence was found to be hgher than otherwse beleved from prevous studes. Approxmately 13% of households n and 16% n reportng had been drectly nvolved n a dspute wthn the past two years. These dsputes were most commonly about ssues of economc lvelhood, partcularly ad dstrbuton and land. The next most commonly occurrng dspute type nvolved crmnalty such as theft and fghtng. These fndngs dffer from a prevous study n 2006 (Decentralzaton and Governance Survey) whch found crmnalty to be the most common dspute type. Crmnal dsputes were found to be more commonly referred eventually to the polce rather than settled durng local dspute resoluton processes. The rankng of dsputes whch respondents felt were of concern to the vllage were roughly smlar to rankng of dsputes actually experenced n. In, however, two ssues, namely land and fghtng, were rated as even greater concern than ther rankng n actual frequency of occurrence would suggest. 4

5 Dsputes are most commonly personal, and occur very close to home. Between 60 to 70 percent of all dsputes occurred wthn the famly or wthn the same vllage, and only around twenty percent nvolved external partes such as companes or people from a neghbourng vllage or from a hgher level of government. Dspute Resoluton Actors A large majorty (around 70%) of people attempted to resolve ther dsputes through drect negotaton wth the other party. In approxmately half these cases a settlement was acheved. In other cases, or f drect negotaton faled, local offcals were the next port of call for most communty members experencng a dspute (around half of all unsettled cases). Polce are the next most commonly approached dspute resoluton actor, albet less popular n. Despte hgh levels of trust n tradtonal and relgous leaders, they are less commonly called upon to resolve dsputes compared to famly, local government, and formal justce sector actors. These patterns emerged clearly when communty members were asked where they would seek assstance n a seres of hypothetcal dsputes. In cases where protagonsts were from wthn the same famly, people would turn frst to famly and frends for help, then to vllage or hamlet offcals. A sgnfcant number of people would also seek help from relgous and tradtonal leaders. Only f the frst two dspute resoluton actors from whom they sought assstance proved unable to resolve the dspute would the majorty of people turn to the formal justce system the courts n the case of an nhertance case, or the polce n the case of domestc volence. In hypothetcal assault, land dspute and corrupton cases, all nvolvng outsders, people would frst contact ther vllage leaders, then the formal justce system. The nference s that people would prefer to use local leaders as a referral mechansms rather than dealng drectly wth the formal justce system. Local dspute resoluton processes scored well n terms of perceved natural justce (opportunty to hear and be heard) and farness, wth only between a quarter and a thrd feelng the process was unfar. There s a correspondngly hgh level (~90%) of satsfacton wth the outcome of resolved dsputes. Conflct and Safety In, conflct between the communty and securty forces, vllage leaders or poltcal partes was not consdered to be serous. Ths was a greater concern n, where around a thrd of respondents dd consder t a serous problem n ther vllage. also had more serous concerns about the potental for horzontal conflct between vllages and ethnc or relgous groups. Only a small percent of respondents dentfed those types of conflcts as serous problems n. Furthermore, almost nnety percent of respondents n dd not feel that the tensons between former ndependence combatants (GAM) and other vllagers were a serous problem, wth the excepton of Besar dstrct, where 17% felt t was. 5

6 Whereas ndvduals dd not report personally experencng conflcts relatng to enterprses from outsde the vllage, focus group dscussons cted them as affectng around ten to twenty percent of vllages n and, wth half of these cases descrbed as havng serous mpacts. These conflcts, whch affected broad communty nterests, were reported n rural areas, suggestng the enterprses were predomnantly nvolved wth natural resources. These vllage level focus group dscussons renforced the fndngs from ndvdual households that conflct over development assstance were common and serous, affectng around a thrd of all vllages, and beng consdered serous (or worse) n about a thrd of those cases. A majorty of people n two dstrcts among the most conflct-affected n expressed some concern that ther vllage mght experence volence n the next year, hghlghtng the mportance of contnued efforts to strengthen peace and stablty n.. Trust, Farness and Corrupton n Publc Insttutons In general, communty members have a greater degree of trust n, and confdence that dspute resoluton processes and outcomes wll be far, when handled by local nsttutons compared to the formal justce system or provncal level government offces. Vllage level offcals are more trusted than those one step removed n the government herarchy, at sub-dstrct level, for example. Only a lttle over half of respondents n consder the state court system to have far processes and outcomes, and publc opnon of the courts s lttle better n. Tradtonal (adat) leaders and nsttutons are very well trusted overall, but especally by men n, of whom over nnety percent say they can trust adat leaders. Adat dspute resoluton s also perceved the farest n processes and outcomes, whch s nterestng gven that results cted above show that people rarely take ther problems to ths forum. Only a few percent of surveyed communty members perceve local level nsttutons such as tradtonal and vllage leaders to be corrupt. Formal nsttutons, partcularly the polce, the courts and the land ttles offce were more lkely to be perceved as engagng n petty corrupton. The polce n had the hghest negatve ratng of all nsttutons surveyed. Relgous courts were consdered the formal justce sector nsttuton least lkely to ask for unoffcal fees. Interestngly, publc atttudes towards petty corrupton are surprsngly permssve, half of respondents (and a thrd n ) feelng that t was reasonable for poorly pad publc offcals to ask for unoffcal fees. 6

7 7

8 Table of Contents Executve Summary... 2 MCLE and CRT Program Desgn... 2 The Evaluaton... 2 Legal Awareness... 3 Legal Documents... 4 Dspute types... 4 Dspute Resoluton Actors... 5 Conflct and Safety... 5 Trust, Farness and Corrupton n Publc Insttutons... 6 Table of Contents... 8 PART I - INTRODUCTION Introducton and : background Report outlne Project Descrptons Medaton and Communty Legal Empowerment Program Conflct Resoluton Tranng n MCLE and CRT Impact Evaluaton Evaluaton Desgn Treatment Assgnment Sample Sze Sources of Informaton & Research tools Methodology to solate the mpact of the program PART II - RESULTS AND DISCUSSION Impressons of the Law Legal Knowledge Impressons about Protecton of Rghts Possesson of legal documents Possesson of Natonal ID card (KTP) Possesson of Government Brth Certfcates for Chldren Possesson of offcal Marrage or Dvorce Certfcate Proof of property rghts Legal advce sought by vllage leaders Dsputes Dspute frequency and types Dspute frequency and types Dsputes of most concern to the vllage Dsputants Dspute resoluton trajectores Dspute Resoluton Actors The Dspute Resoluton Process Conflct and Safety The key securty concerns of communtes

9 4.1.1 Perceptons of justce n conflct outcomes Horzontal conflct : Conflct nvolvng former GAM Conflct wthn the vllage nvolvng securty forces, vllage leaders and poltcal partes Perceptons of personal safety and securty of possessons Potental for Vllage Unrest Vllage leaders conflct resoluton styles and confdence Communty Leaders opnons on conflct nvolvng government offcals, companes, and development efforts Dspute Resoluton Actors Recorded Preferences Preferences nvolvng crmnal assault case Preferences n an Inhertance case Preferences nvolvng Domestc volence case Land case Corrupton case Implcatons Insttutons: Trust, Farness and Corrupton Publc Trust n Formal and Non-State Insttutons Perceptons of Tradtonal (adat) Leaders or Insttutons Perceptons of local government offcals Perceptons of formal justce system actors Perceptons on Corrupton Results of Comparablty of Indcators Across Treatment and Control Groups PART III CONCLUSIONS Conclusons Appendx 1 - Treatment Assgnment and Sample Selecton Appendx 2 - Selecton Process for Key Informant Survey and Communty Survey Appendx 3 - Detaled Lst of Evaluaton Indcators

10 PART I - INTRODUCTION 1. Introducton Ths paper reports the methodology and fndngs of a baselne quanttatve study, the frst step towards evaluatng the mpacts of the Medaton and Communty Legal Empowerment (MCLE) program n the Indonesan provnces and. MCLE s a component of the Support for Poor and Dsadvantaged Areas (SPADA) project of the Indonesan Government. The same survey s beng used as a baselne for a program of Conflct Resoluton and Medaton Tranng (CRT) for local leaders and project facltators n. The SPADA program s ntended to help the government n addressng the problems of governance and poverty n Indonesa s poorest and conflct-affected provnces. The Government of Indonesa s plannng agency (Bappenas) and the Mnstry for Dsadvantaged Areas (KDPT) are jontly managng SPADA. The World Bank and the Indonesan Government are plannng to rgorously montor and evaluate the MCLE component of the SPADA project. The mpact evaluaton of MCLE wll feed nto polcy debates on justce reform and the welfare mpacts of provdng access to justce to poor communtes n the global context. It wll also help nform the mplementaton of the Government of Indonesa s new Natonal Strategy on Access to Justce, and the scalng up of ths approach wthn PNPM, a naton-wde communty empowerment program. The man goal of the MCLE program, whch s descrbed n more detal below, s to ncrease access to justce for margnalsed groups. The MCLE program does ths through establshng a network of vllage-level paralegals and sub dstrct-level communty legal ad posts, all supported by a communty lawyer n each dstrct. The actvtes of ths network are communty legal educaton; capacty buldng for nformal justce actors; legal ad and complant handlng; and awareness rasng of communty legal needs and challenges. CRT ams to buld the capacty of local leaders and facltators n post-conflct by provdng them wth a comprehensve tranng program amed at mprovng ther facltaton, medaton, arbtraton and negotaton capactes, and enhancng ther understandng of local conflct dynamcs. 1.1 and : background The two provnces n ths study have n common a tumultuous recent hstory n whch many ctzens lost ther lves and much physcal damage was done to nfrastructure. Both are relatvely under developed, although, a provnce of approxmately 700,000 resdents, s partcularly economcally dsadvantaged., wth 4 mllon resdents, suffered the dsrupton of the Asan Tsunam of 2004, a humantaran dsaster 10

11 of massve scale n whch around 226,000 Indonesans were klled and much nfrastructure destroyed. Precedng the tsunam, suffered a volent separatst conflct whch took the lves of around 15,000 people. The conflct began n 1976, contnung for thrty years wth varyng levels of ntensty, endng only n 2005 wth the sgnng of the Helsnk Memorandum of Understandng, a peace agreement between the Free Movement (GAM) and the Indonesan government. s now concludng a process of postconflct and post-tsunam rebuldng, whch has attracted an nflux of domestc and foregn ad, presentng many challenges n admnstraton and dstrbuton. As wth many post-conflct regons, there s concern that there s reluctance among former GAM combatants, although formally dsbanded, to relnqush power over local poltcs and resources. suffered a sectaran conflct whch began n 1999 and clamed several thousand lves before gradually settlng down by The horzontal conflct n was dfferent n many respects to that whch suffered, as well as beng shorter n duraton. One of the factors n the conflct n was arguably an abandonment of tradtonal dspute resoluton mechansms, and the falure of the formal law enforcement and justce system to deal wth dsputes, leadng to a culture of vglantsm, n whch people often took sdes along ethnc or relgous lnes. 1 After fghtng began n January 1999, Ambon was unoffcally dvded nto Chrstan and Muslm terrtores. A key mlestone n endng the conflct was the government-sponsored Malno II peace talks n 2002, after whch the volence gradually subsded. 1.2 Report outlne Secton 1.4 and Appendx 1 explan n detal the methodology used n the quanttatve aspect of the MCLE and CRT mpact evaluaton, of whch ths quanttatve baselne survey s a part. Secton 2 Impressons of the law attempts to gauge ordnary people s knowledge of Indonesan law; ther vews relatng to varous key rghts ncludng gender ssues; and the rate of possesson of key legal documents. Secton 3 examnes the types of dsputes actually experenced by communtes and the manner n whch those dsputes were resolved. Ths s compared to the dsputes types experenced by vllages as a whole. Secton 4 Conflct and Safety analyses responses to a seres of statements desgned to measure ordnary people s perceptons of justce n conflct resoluton, and prevalence or fear of volence and theft n ther communtes. 1 Brown,G. Wlson,C. Suprayoga,H. Overcomng Volent Conflct: Volume 4, Peace and Development Analyss n and North, 2005, UNDP, LIPI and BAPPENAS. 11

12 Secton 5 Dspute Resoluton Actors examnes a seres of hypothetcal dsputes n order that all the ndvdual respondents, the majorty of whom had not experenced an actual dspute, had an opportunty to reveal ther preferences n seekng legal assstance. Secton 6 Trust, Farness and Corrupton evaluates perceptons of justce-related nsttutons, both formal and nformal. Secton 7 Results of Comparablty of Indcators Across Treatment and Control Groups reports on the degree of smlarty between varous baselne measures n treatment and control vllages. 1.3 Project Descrptons Medaton and Communty Legal Empowerment Program The MCLE program was desgned based on a number of Government and World Bank analytcal studes and plot programs. The program s man am s to ncrease access to justce for rural communtes so that they can resolve conflcts and legal problems themselves. Ths can nclude access to both formal and nformal justce systems. MCLE has a number of specfc objectves: 1. Strengthen the capacty of nsttutons and ndvduals at communty level to resolve dsputes n an open, ndependent and just manner; 2. Assst and facltate the resoluton of dsputes nvolvng communty-wde nterests; 3. Increase publc trust n the formal legal system by enhancng communty legal awareness and access to the formal system. To acheve these ams and objectves, MCLE establshes a support network for provdng legal ad and empowerment actvtes. The man focus of actvtes s at communty level. MCLE establshes a network of Communty Legal Ad Posts (Posko) at the sub-dstrct-level. The posts wll functon as a centre for legal nformaton, medaton and complants handlng. The Posko wll be staffed by a full-tme facltator (one person per Posko) and paralegals n selected vllages (two per vllage one woman and one man). They wll work n partnershp wth local leaders and communty fgures, ncludng relgous leaders, the vllage head and adat nsttutons. They wll also work wth judges, prosecutors and polce. The Communty Lawyer at the dstrct level wll provde techncal legal support on a part-tme bass. The man actvtes are: ƒ Communty Legal Educaton ƒ Capacty Buldng for Informal Justce Actors ƒ Legal Ad and Complant Handlng Network ƒ Awareness Rasng of Communty Legal Needs and Challenges Bappenas and the Mnstry for Dsadvantaged Areas (KDPT) are jontly managng the program. KPDT has contracted two NGOs to operate as the Provncal Implementng 12

13 Agences for MCLE, wth overall responsblty for mplementng the MCLE component. The SPADA NMC Communty Legal Empowerment specalst wll provde techncal support to the Provncal Implementng Agences. The project s targeted at four levels (See Table 1 and Fgure 1). Frst, two provnces, and, have been chosen because of the level of conflct prevalent there. Wthn each provnce, a number of dstrcts or kabupaten (4 n, 2 n ) have been chosen based on an assessment of the greatest need for the program and demand from local stakeholders. The table below summarzes the program locatons n the two provnces. Table 1 Program treatments conducted at each admnstratve level Provnce Dstrct # Kecamatan # Vllages # Paralegals (Posko) Barat Besar Utara Pde Pde Jaya* Tengah Seram Bagan Tmur* Tenggara Kota Tual* Total: * These dstrcts have recently splt from the dstrct mmedately above them n the lst. For the purposes of ths evaluaton, they wll be treated as part of the orgnal dstrct. Below the dstrct (kabupaten) level, there are two levels of targetng where the program was randomly allocated. Frst, there s the selecton of sub-dstrcts (kecamatan) where the posko or communty legal post wll be located. In, 4 out of the 11 to 30 kecamatan per kabupaten are chosen (for a total of 16 kecamatan). In we chose 4 out of the 10 to 19 kecamatan per kabupaten (for a total of 8 kecamatan). Second, only a random selecton of vllages receves paralegals. In, ths s estmated to be about 8 of 30 vllages per kecamatan, for a total of 128 of 728 vllages. In, ths proporton s 8 of approxmately vllages per kecamatan, for a total of 64 of approxmately 112 vllages. Fgure 1 provdes a graphcal llustraton of program assgnment and mpact evaluaton desgn. 13

14 Fgure 1 - Program Assgnment Desgn for SPADA MCLE Project ( & ) Admn Level Kabupaten/ Dstrct (Preselected) Lawyer (6 kabupaten) Stage A: Kecamatan/ Posko No Posko Sub Dstrct Total: 24 Total: 88 (16 n 8 n ) (64 n and 24 n ) Stage B: T1 T2 C Desa/ Paralegal No Paralegal No paralegal Vllage Total: 192 Total: ~1,000 (128, 64 ) Stages of Randomzaton Stage A: Random selecton process for sub-dstrcts (kecamatan) recevng Poskos and for sub-dstrcts assgned as controls. Stage B: Random selecton process for vllages recevng paralegals. Evaluaton: T1 vs C: Effect of havng access to a posko and paralegal versus nothng T2 vs C: Effect of havng access to only a posko but no paralegal versus nothng Conflct Resoluton Tranng n The CRT program wll delver tranng to around 408 vllage leaders from 100 vllages n around 20 sub-dstrcts, as well as around 400 PNPM (the Government s natonal communty development program) facltators n 90 sub-dstrcts across. Project facltators receve a 5 day tranng course, followed by a 3 day advanced course three months later. Vllage leaders who may be vllage heads, women or youth leaders, or vllage councl representatves receve a smlar fve day tranng course. CRT ams to: Buld and strengthen networks of cvl socety, PNPM facltators, and vllage leaders n support of conflct resoluton; Enhance the capacty of PNPM facltators to deal wth conflct emergng from the communty development and partcpatory plannng processes, and to 14

15 enhance the capacty of facltators to deal wth the mpacts of post Helsnk MoU tensons on programs; Enhance the capacty of PNPM facltators to medate a lmted range of nonproject conflct, n partcular those related to other development and rentegraton programs; Enhance the capacty of vllage leaders to manage local conflcts through facltatng dalogue, medatng dsputes, and negotatng settlements. The CRT program was developed by the World Bank s Conflct and Development team, and the tranng s beng mplemented by Forum LSM. Vllages for ths treatment have been randomly selected wthn MCLE dstrcts through the same samplng process as MCLE but are separate from T1, T2 or C n Fgure 1 above. To avod problems of attrbuton, CRT treatment sub-dstrcts dffer from those where MCLE s beng mplemented. The World Bank s Conflct Team wll use ths thrd treatment along wth the same MCLE (C) control vllages to conduct a separate evaluaton. 1.4 MCLE and CRT Impact Evaluaton The Government of Indonesa and the World Bank are plannng to rgorously montor and evaluate the MCLE program. An mpact evaluaton s beng carred out and wll examne the welfare mpacts of provdng access to justce to poor communtes. Ths evaluaton s beleved to be the frst nstance of rgorous quanttatve and qualtatve evdence on the effect of the provson of communty level legal servces and educaton on a range of human rghts and economc outcomes. The mpact evaluaton wll examne the effects of MCLE on knowledge of rghts, exercse of rghts, dspute resoluton, reports and resoluton of corrupton, nvestment and household consumpton. Whle some of these effects are expected to take longer to manfest than others, there are no clear experences whch tell us what ths tmeframe s. Thus, to complement the practcal soluton of conductng multple follow up surveys we dentfed the followng key questons for the evaluaton to answer: 1. Does MCLE ncrease benefcares ablty to clam rghts, enforce rghts, and resolve dsputes through non-volent means? 2. Does MCLE reduce volent conflct and ncrease safety and securty? 3. Does MCLE mprove household welfare n the vllages that receve paralegals? 4. Does MCLE ncrease the reportng of corrupton and brbery and decrease these events n the long term? 5. Does MCLE mprove ndvduals capacty to engage wth the state and the state capacty to respond, partcularly n matters nvolvng the resoluton of dsputes? 6. Does MCLE mprove gender relatons at the household and communty level? 15

16 A lst of more detaled ndcators for the MCLE evaluaton s ncluded n Appendx 3. The survey also allows for a rgorous evaluaton of the CRT program. Questons to be answered by the CRT evaluaton are as follows: PNPM facltators: 1. How does CRT affect publc satsfacton wth and partcpaton n local development meetngs, overall satsfacton wth PNPM processes, and the outcomes of the processes? 2. How does CRT affect perceptons amongst wnners and losers n the competton for resources engendered by the PNPM process? Does the tranng result n hgher levels of satsfacton, perceptons of farness, and confdence n the ablty to have one s needs addressed through peaceful poltcal means? 3. How does CRT affect vllage-level ndcators of conflct and volence, n partcular development-related conflcts? 4. How does CRT affect the way facltators do ther work and ther satsfacton wth ther role n the conflct resoluton process? How do the tranng programs affect the role facltators play n the resoluton of other conflcts n the communty? Vllage leaders: 1. At the vllage level, how does conflct resoluton tranng for vllage leaders affect levels of conflct and volence? 2. How does t affect vllage s levels of socal captal, satsfacton wth vllage government, partcpaton n vllage actvtes, and communty perceptons of local dspute resoluton mechansms, justce and securty? 3. How does t affect the ways n whch these leaders deal wth conflcts and communty confdence and satsfacton wth ther partcpaton n the conflct resoluton process? 4. At the household level, how does conflct resoluton tranng for vllage leaders affect the way households deal wth conflct? 5. How does t affect household-level perceptons of the way conflct and volence affect and are dealt wth n the communty? 6. How does t affect household-level perceptons of socal captal, satsfacton wth vllage government, partcpaton n vllage actvtes, and communty perceptons of local dspute resoluton mechansms, justce and securty? Evaluaton Desgn The program evaluaton s desgned to measure the mpact of legal empowerment actvtes on householders knowledge and understandng of law, dspute settlement preferences, and awareness and voce aganst corrupton. At the core of the evaluaton there are two measurements: A baselne survey conducted before the program begns, 16

17 and a follow up survey conducted after the program has been operatng for a perod, perhaps around two years. The evaluaton methodology s schematcally summarzed as follows: Fgure 2 - Evaluaton schematc Baselne: Follow up: around 2011 Treatments Control T1: Posko and paralegal No Posko no paralegal T2:Posko only (no paralegal) Posko: 24 kecamatan Paralegals:192 vllages 88 kecamatan Households: : 396 for T1, 390 for T2 : 396 : 420 for T1 : 450 The baselne analyss allows us to determne f the randomzaton has resulted n the choce of treatment and control groups whch are comparable at the begnnng of the program. If they are comparable, an end-of-project analyss s suffcent and no further statstcal adjustments are necessary. However, f the treatment and control groups are sgnfcantly dfferent, the follow up survey analyss mght provde a based result due to lack of nformaton on the magntude and drecton of change that has occurred n the treatment and the control groups 2 durng mplementaton perod. In that case, a dfference n dfference approach mght be useful for a rgorous evaluaton and, for that matter, a baselne analyss s essental Treatment Assgnment In, four sub-dstrcts were randomly assgned to the treatment n Barat, Besar, Utara and Pde. Smlarly, n, four sub-dstrcts were randomly assgned to the treatment n both Tengah and Tenggara. The table below summares the probablty of an ndvdual sub-dstrct beng chosen n a gven dstrct. Table 2 - Treatment assgnment Provnce Dstrct Sub-Dstrct Selected Sub-Dstrct Total Percentage Barat % Besar % Utara % Pde % Tengah % Tenggara % Total: See Valadez and Bamberger 1994, page

18 Followng the selecton of the sub-dstrct, eght vllages were randomly selected wthn each selected sub-dstrct to receve the paralegal treatment. Therefore, 32 vllages should be assgned to receve the paralegal treatment n each dstrct, though some subdstrcts do not have a suffcent total number of vllages (see Appendx 1) 3. A note on dstrct border changes: The samplng calculatons were conducted based on borders of dstrcts n and as they were at the tme the MCLE mpact evaluaton was desgned. At that tme, the program locatons covered four dstrcts n and two dstrcts n. Snce then, a seres of pemekaraan (dstrct splttng) has meant that the followng dstrcts have changed: Pde () Pde and Pde Jaya; Tengah Tengah, Seram Bagan Barat and Seram Bagan Tmur; Tenggara Tenggara, Kepulauan Aru and Kota Tual Where ths report refers to dstrcts t does so based on the orgnal borders. Reference to Tengah n ths report, for example, s actually reference to three dstrcts as they currently stand. The sample desgn allows for analyss between geographc areas based on the old borders of dstrcts but s not representatve across the new borders. As prevously noted, the survey was conducted n four dstrcts n and two dstrcts n. For ease of reference throughout the report reference s made to and. Where ths s the case, the nformaton s only relevant to the dstrcts where the survey was conducted and can not be assumed to be relevant to other dstrcts n those provnces or the provnces as a whole Sample Sze 4 Our own calculatons ndcate a sample sze of households per treatment arm (wth two treatment arms n and one treatment arm n ) would be approprate gven the assumptons on expected outcome and ntra-household correlaton. Due to cost consderatons and n consultaton wth the survey frm, t was decded to opt for 10 households per cluster n (45 total clusters) and 6 households per cluster n (66 total clusters). The MCLE program desgn has two varatons. One set of treatment vllages, dentfed as T1 n Fgure 1, receve Posko at the kecamatan level AND paralegals at the vllage 3 In three sub-dstrcts n, Leupung, Geumpang and Grong Grong, and two sub-dstrcts n, Tayando Tam and Pulau Dullah Selatan, there are fewer vllages than the eght necessary for the program. In these cases, all avalable vllages wll be treated. As to the remanng seven vllages n needed for a complete assgnment of the 128 total vllages, and the fve needed for for a total of 64, they wll be chosen n such a way to mnmze mplementaton costs wthout contamnatng the control vllages. As they are not randomly selected, they wll not be ncluded n the evaluaton. 4 A number of vllages were excluded from the potental pool of treatment and control sample vllages by survey frm as they are not accessble by commercal transportaton. Excluded from the sample by the survey frm are the followng sub-dstrcts: all of Aru Selatan, all of Aru Tengah (except Benjna, Gardakau, Marrmar, Namara, Papakula, Selbata-Bata and Sellau), all of Pulau-Pulau Aru (except Gala Dubu, Gorar, Jabulenga, Karanggul, Swalma, Tungu, Tunguwatu, Ujr and Wangel), and all of PP Kur. These vllages were not excluded from the assgnment of the program because t s government funded and vllages cannot be excluded from elgblty based on locaton. 18

19 level (192 vllages). The other set of treatment vllages (T2) are vllages n kecamatan where there are Posko but the vllage does not receve paralegals (600 vllages n and 48 vllages n ). The control group s drawn from vllages n the same kabupaten but dfferent kecamatan. These vllages wll have access to nether paralegals nor Posko. The dstrbuton of sample households across treatment and control groups are presented below. The method used for treatment and control sample selecton s provded n Appendx 1. Table 3 - Dstrbuton of households across treatment and control T1: Posko and Provnce paralegals T2: Posko only Control Note: only has treatment 1: Posko and paralegals. Comparng T1 or T2 to the control group wll allow us to measure the mpact of both versons of the program relatve to nothng. Comparng T1 to the control group would provde the combned effect of both the Posko and the paralegal. Comparng T1 vs. T2 allows us to measure the effects of havng a Posko only relatve to havng both poskos and paralegals. Ths addtonal layer of comparson s useful because provdng poskos only s clearly much cheaper. Evaluatng the effect of poskos plus paralegals versus the control (T1 vs C) s the prorty for ths evaluaton, wth the comparson of posko and paralegal versus Posko only s conducted only n the cases where we have a suffcent sample of vllages n the kecamatan that were selected. Note that n the 4 selected kabupaten n there are a total of 2700 vllages makng ths analyss feasble n, but not n where there are far fewer vllages n selected kecamatan Sources of Informaton & Research tools The quanttatve tools and data collecton efforts nvolve sx tools. Frstly baselne surveys of households, key nformants and communtes were conducted between November 2008 and January 2009 at the locatons summarsed n Table 4 - Samplng Locatons. These are to be followed later by a program facltator survey, newspaper conflct montorng, and MIS data. Table 4 - Samplng Locatons Provnce Dstrcts Kecamatan Vllages Vllage Blocks HH Surveys KI Surveys C Surveys 5 Note: delays due to bad weather condtons resulted n two vllages n Tengah beng excluded from the analyss of the baselne data. The vllages wll be sampled before the program begns and the mssng data should be avalable for ncluson n the endlne comparson, 19

20 , TOTAL , Household Survey The baselne survey was admnstered to a random sample of households n selected vllages/communtes. It contans a standard set of demographc and socoeconomc nformaton (household composton, consumpton expendture and housng condton) consstent wth the natonal household survey (Susenas). In addton, the questonnare also contans questons specfcally pertanng to outcome ndcators asked of the head of household and/or hs/her spouse (volence, crme, dspute resoluton experences, partcpaton n vllage lfe, perceptons of safety and dspute resoluton, and the lke). Respondents for the ndvdual portons of the household survey questonnare (Sectons F-J) were randomly stratfed by gender to ensure that half of the ntervews n each vllage were women and half men. Respondents for Secton D of the household survey (consumpton) were those best able to answer. All of other sectons of the household survey were taken usng the household head. The follow-up surveys wll be desgned to measure changes n outcomes. A "partal panel" desgn wll be consdered, where the mdterm/endlne households are partally a panel of households ntervewed n the baselne survey and partally a new cross-secton to allow for mgraton/populaton shfts n the 2 year nterval. In addton, pror to the mdterm/end lne survey, potentally trackng baselne households that moved out of the vllage/communty wll be consdered/revewed, snce households who moved who mght have dfferental outcomes (for example, f they moved due to a dspute that could not or was not resolved). 2. Key Informant Survey The key nformant surveys are admnstered to vllage authortes (ncludng nformal leaders see Appendx 3 for detals on selecton crtera). The questonnare contans demographc and socoeconomc nformaton, as well as questons pertanng to outcome ndcators at the ndvdual and vllage-level (legal knowledge and opnons, perceptons of safety and securty, corrupton, servce delvery, and clam-makng). 3. Communty Survey Baselne communty surveys are admnstered to vllage authortes and nformal leaders (see Appendx 2 for selecton crtera) usng a group ntervew method. The questonnare contans questons about vllage characterstcs such as geographcal accessblty, nformaton accessblty, access to state nsttutons, etc., and also the presence of other government and NGO programs, the frequency and type of both ndvdual and communty dsputes n the vllage, and communty clam-makng. 4. Program Facltator Survey A baselne survey of paralegals and posko staff are admnstered to all program staff workng at the vllage-level (paralegals) and kecamatan level (fasko). Ths survey 20

21 attempts to provde measures of attrbutes of these ndvduals that may prove mportant for project success (e.g. educaton levels, experence, socal connectons, legal knowledge). 5. Newspaper Conflct Montorng As part of ts montorng and evaluaton, the SPADA program wll collect data on conflct and crme levels and mpacts as reported by local newspapers n all nne SPADA provnces. Ths data source wll be partcularly relevant for examnng the research questons relatng to volent conflct levels, securty and safety, and ncdence of (reported) corrupton. 6. MIS Data The mpact evaluaton wll also utlze the montorng data on the program functonng. Ths wll nclude data on posko actvtes, paralegal actvtes, and case handlng data. Ths wll also possbly nclude a small satsfacton survey of posko and/or paralegal users Methodology to solate the mpact of the program To solate the mpact of the program, we wll evaluate the changes n treated group after treatment (I 1 I 0 ) relatve to that of control group throughout the same perod of tme (C 1 C 0 ). The change n the magntude and drecton allows for the dentfcaton of nfluental factors external to the program. It also permts the quantfcaton of changes n the evaluaton area caused by these factors occurred n the project and the control groups. 6 The changes n control group are calculated by subtractng the ntal condton of the control group (C 0 ) from the fnal condton (C 1 ): Changes caused by external factors = C 1 C 0. Smlarly, besdes the program, there may be some external factors n the nterventon group (I) as well. As a result, the mpact of the program transfers can be estmated by subtractng the effect of changes caused by external factors from the changes n treated group after treatment By rearrangng we get, Impact of the program transfers = (I 1 I 0 ) (C 1 C 0 ) Impact of the program transfer = (I 1 -C 1 ) (I 0 -C 0 ). 6 Valadez, J. and M. Bamberger Montorng and evaluatng socal programs n developng countres. A handbook for polcymakers, managers, and researchers. Washngton D.C.: The Internatonal Bank for Reconstructon and Development. 21

22 However, despte precautons, t s dffcult to prevent the contamnaton due to dfferent local hstory. The random assgnment permts us to determne n advance the probablty of obtanng an apparently sgnfcant effect for the program when n realty t s due to ntal nequaltes, bases, or pure chance. To overcome these problems, the surveys are desgned to collect nformaton necessary to detect the problem of dfferent local hstory as much as possble so that the statstcal analyss can control for any dfferences usng multvarate analyss, f necessary. The purpose of the random assgnment of sub dstrcts or vllages to nterventon and control s to ensure that the two groups are smlar along both observables and unobservables. Our survey data allows us to verfy whether the selecton was effectve along observable dmensons by comparng households across comparson and treatment groups. Even f dfferences between groups exst, the evaluaton can stll measure the mpact because we observe these dfferences n the baselne and can control for them n the analyss by usng double dfferences. Therefore, perfect equalty s not a necessary condton for evaluaton; t only makes the evaluaton less complcated. 7 Drawng upon ths argument, our baselne analyss provdes nformaton on (I 0 -C 0 ). We used 2 dfferent methodologes to calculate the dfferences. Frst, we used normalzed mean dfferences between the treatment and control groups. Second, we used Welch s t- test wth assumpton of unequal sample sze and unequal varance. The frst method to assess overlap s to report the summary statstcs for all ndcators usng normalzed dfference n ndcators weghted means by treatment status: X t X c normalze dff = 2 2 St + Sc The advantage of ths method s that t can take care of dfferent sample szes for a comparatve study. The second method uses the Welch s t-test of the followng form: t value= X t X c 2 St S + N N t 2 c c Essentally, the t-statstc s equal to the normalzed dfference multpled by the square root of the sample sze. As such, the t-statstc partly reflects the sample sze. Gven the same standard devatons between the two groups n terms of average ndcator values, a larger t-statstc just ndcates a larger sample sze, and therefore n fact an easer problem n terms of fndng credble estmators for average treatment effects. However, a larger t-statstc for the dfference between ndcators by treatment group does not ndcate that the problem of fndng credble estmates of the treatment effect s more dffcult. In general, a dfference n average means bgger than 0.20 standard devatons 7 Internatonal Food Polcy Research Insttute (2001): Evaluaton Desgn For the Plot Phase of the Ncaraguan RED DE PROTECCIÓN SOCIAL, Report (translated from Spansh) 22

23 s substantal. 8 We use both for a robust value of dfferences whch wll permt us to decde the comparablty of the treatment and comparson groups. Any sgnfcant dfferences across groups strengthen the need for dfference n dfference methodology for evaluaton usng longtudnal data. There are several possbltes for the analyss of longtudnal data collected at dfferent ponts n tme. Two mportant ones we wll use are: a. Longtudnal analyss of populaton groups. The basc unt of analyss s the unt of assgnment, n ths case, the vllages for paralegals and kecamatan or sub dstrcts for poskos. For each vllage, we can estmate the change regstered between the baselne and the follow-up measures. Or, b. Longtudnal analyss of ndvduals or households. For each ndvdual (or household), we can estmate the change regstered between the baselne and the follow-up measures. The results of the normalzed mean dfference and t-tests are presented n Secton 7 and Appendx 4. 8 Wooldrdge, J. M and G.W. Imbens (2009): Recent Developments n the Econometrcs of Program Evaluaton, Journal of Economc Lterature 47, 5 86, March

24 PART II - RESULTS AND DISCUSSION 2. Impressons of the Law Ths secton analyses communtes general mpressons about the law. Three aspects are looked at. Frst, questons were asked to gauge communtes knowledge of Indonesan law across a range of ssues. Second, communtes were asked to provde ther vews relatng to how ther rghts should be protected under the law. Ths analyss ncludes a focus on how gender ssues are perceved at the vllage level. Thrd, nformaton was collected on the extent of legal documentaton communtes possess and how these legal documents are obtaned. Consstent wth prevous research on these ssues, the man fndngs confrm that legal awareness vares sgnfcantly dependng on the soco-economc backgrounds of respondents. The less well educated and those lvng n rural areas are lkely to have lower levels of legal awareness. Sgnfcant gender dfferentals exst as well. Legal documentaton and, n partcular, possesson of natonal ID cards, vares sgnfcantly not only across provnces but at the dstrct level. The low level of ID card possesson n Tengah needs to be assessed n more detal to dentfy both the reasons why these rates are low and the mpact, f any, of low rates of possesson. 2.1 Legal Knowledge The level of knowledge about aspects of Indonesan law s low across the dstrcts surveyed. Ths s consstent wth prevous surveys conducted on communtes awareness of law n Indonesa. 9 Communtes were asked ten questons on Indonesan law. The questons covered ssues drectly affectng the lves of vllagers ncludng property rghts, famly law, gender equalty, labour and corrupton. For each queston, respondents were provded wth a rght and wrong answer and asked to dentfy whch response was consstent wth the law. There s consderable varaton between and the dstrcts covered n relatng to knowledge of the law. As outlned n Fgure 3 below, n, 87.1% of respondents could only answer 1 to 6 out of the ten questons correctly (compared to 62.5% n ). Communty members n the dstrcts surveyed n are more lkely to have greater knowledge of the law wth 24.5% of respondents answerng 7 or more questons correctly, whereas n only 12.5% were able to answer ths number of questons accurately. The excepton here s that there s a consderable porton of nese respondents, 13%, who were not able to answer any of the questons correctly. Ths can probably be attrbuted to the fact that the questons were asked n 9 Asa Foundaton, Survey Report on Ctzen s Perceptons of the Indonesan Justce Sector, Jakarta, 2001, pp 25-29; World Bank/AC Nelson,.Baselne Study toward Revtalzaton of Legal Ad and Women s Legal Empowerment, Jakarta,