So what difference does it make? Assessing the impact of participation, transparency and accountability John Gaventa World Bank Institute Seminar November 22, 2010
Taking a Citizen Led Approach: 10 Years of DFID funded research on Citizenship, Participation and Accountability (www.drc-citizenship.org). First phase, much attention was on dynamics of state-society relationship Second phase, more on how citizens mobilise and empower themselves, often outside of the state In this presentation draw from this work, but also highlight two recent projects and their implications for voice and accountability Mapping outcomes of citizen engagement The impact of transparency and accountability
Active citizens build effective states - not (only) the other way around Much has been learned about citizens view the state, and about the state-society relationship Citizens can help to build democratic institutions, legitimacy, responsiveness, capability, accountability societal opportunities create possibilities of political reform But we need more focus on the society side: how to build active, empowered citizens how active citizens mobilise to change development policies, build responsive states, and do things for themselves
What difference does citizen engagement make? The plea for evidence The idea that good governance cannot be achieved without the active involvement of citizens and civil society actors has gained growing consensus in recent years. Many donors and NGOs now support "participatory governance", "social accountability" or "demand for good governance" programmes aimed at promoting the active involvement of citizens/csos in public decisionmaking and holding government accountable [...] I'm currently involved in a research project to gather evidence of the results and/or impact of such initiatives. E-mail to author from World Bank consultant, 2009. Our number one challenge is to demonstrate what difference citizen engagement makes. - Representative of large donor agency 2008.
What difference does citizen engagement make? Results from a meta-synthesis of 100 case studies in 20 countries Drawing from widely accepted approaches of systematic review, qualitative case study analysis and synthesis Coding of 800 outcomes Evidence points to largely positive contributions With risks of negative outcomes
Distribution of positive and negative outcomes 90 80 70 60 Percentage 50 40 30 Positive Negative 20 10 0 Construction of citizenship Practices of citizen participation Responsive and accountable states Inclusive and cohesive societies Total
Outcome 1 Better Citizens Positive Increased civic and political knowledge Greater sense of empowerment and agency Negative Increased knowledge dependencies Disempowerment and reduced sense of agency
Outcome 2 More Effective Participation Positive Increased capacities for collective action New forms of participation Deepening of networks and solidarities Negative New capacities used for negative purposes Tokenistic or captured forms of participation Lack of accountability and representation in networks
Outcome 3 More Responsive and Accountable States Positive Greater access to state services and resources Greater realisation of rights Enhanced state responsiveness and accountability Negative Denial of state services and resources Social, economic and political reprisals Violent or coercive state response
Outcome 4 More inclusive and cohesive societies Positive Inclusion of new actors and issues in public spaces Greater cohesion across social groups Negative Reinforcement of social hierarchies and social exclusion Increased horizontal conflict and violence
Types of outcomes and types of engagement Citizens engage in multiple ways Local associations Social movements and campaigns Formal governance spaces Multiple forms of engagement Local associations and movements are particularly important for positive outcomes Multiple forms of engagement are particularly important for realising responsive and accountable states Beyond Putnam not just the density but the nature of the quality and quantity of the association that counts
Distribution of positive and negative outcomes across type of citizen engagement Outcomes sorted by type of citizen engagement (n=828) Outcome type Local associations (n=324) Social movements and campaigns (n=233) Formal participatory governance spaces (n=153) Multiple (n=118) Positive 90% 71% 55% 68% Negative 10% 29% 45% 32% Total 100% 100% 100% 100%
Distribution of positive outcomes across types of citizen engagement Types of citizen engagement (n=100) Positive outcomes sorted by outcome categories Local association s (n=29) Social movements and campaigns (n=29) Formal participator y governance spaces (n=19) Multiple (n=23) Construction of 36% 35% 33% 29% citizenship Practices of citizen 26% 24% 30% 22% participation Responsive and 29% 33% 25% 44% accountable states Inclusive and 9% 8% 12% 5% cohesive societies Total 100% 100% 100% 100%
The relationships of outcomes to context Grouping of countries across regime (Polity IV, Freedom House, Economist Intelligence Unit ) Positive outcomes are not linearly associated with level of democratisation highest proportion of positive outcomes are found in most and least democratic countries Associations are particularly strong for least democratic settings
Distribution of positive and negative outcomes across country types 100 90 80 70 Percentage 60 50 40 Positive Negative 30 20 10 0 Tier 1 Tier 2 Tier 3 Total
Types of citizen engagement Positive Outcomes in Tier Three countries (n=273) Construction of citizenship (n=96) Practices of citizen participatio n (n=66) Responsiv e and accountabl e states (n=83) Inclusive and cohesive societies (n=28) Local associations 89% 92% 83% 78% Social movements and campaigns 0% 2% 2% 0% Formal participatory governance spaces 6% 0% 4% 11% Multiple 5% 6% 11% 11%
Implications 1. Citizen engagement makes a difference, but not always. We need to understand more the factors that lead to positive vs negative change 2. These gains do not emerge automatically: pay more attention must be paid to building citizenship e.g. of empowerment, as a component of other action 3. Only through empowered citizens will accountability relationships likely occur.
Key Findings 4. Citizen engagement makes a difference across regimes, not just in more democratic settings. 5. The role of associations in fragile settings is particularly important for building cultures of citizenship 6. Citizen engagement is often met by reprisals: it is critical to protect the democratic space for engagement if developmental outcomes are to be achieved.
Part II: What is the impact of citizen-led accountability initiatives? Over a decade of rapid growth and spread of transparency and accountability work in development and aid circles and development academia Apparent promise of T&A as the cures for many evils As yet, little clarity about what is being achieved, what works, how it works, and how best to fulfil that promise
... Particular focus: Citizen-led, demand-side and social accountability activities and their connection to state actors, institutions and processes Connections between T, A and participation Methodological challenges of impact assessment in the T&A field effectiveness Vs impact Service delivery, budget processes, FoI, natural resource governance, aid
... Particular focus: Citizen-led, demand-side and social accountability activities and their connection to state actors, institutions and processes Connections between T, A and participation Methodological challenges of impact assessment in the T&A field effectiveness Vs impact Service delivery, budget processes, FoI, natural resource governance, aid
Some evidence of impact, but highly uneven. The positive story - in some conditions TAIs demonstrably contribute to: Greater state responsiveness Lower corruption Building spaces for engagement and empowering local voices Better budget utilization and delivery of services But not always: Evidence is uneven and scattered Initiatives are new and impacts unknown Much focus on effectiveness rather than impact Positive evidence in one case not corroborated by studies in another How do we enhance demonstrable impact?
Challenge 1: Aims, claims, assumptions and expectations (or, against what are we assessing impact and effectiveness?) Aims vary: Developmental outcomes, democratic outcomes, voice empowerment outcomes Assumptions vary: eg on links between T, A and participation; about citizens ; hierarchies of objectives; how explicit/implicit; etc The need for sharper theories of change
Challenge 2: Methodological issues: How do we know what we know? We found some: Quantitative surveys Analysis of aggregated survey data, multivariate analysis Experimental approaches (RCTs) Qualitative case studies and case study analysis Stakeholder interviews Indices and rankings We found a lack of: Comparative studies Ex-post long-term evaluations Appropriate use of baselines Rigorous participatory approaches Complexity-aware approaches eg Most Significant Change; Outcome Mapping; narrative-based Methodological mixes
Challenge 3: Factors that make a difference Little evidence that supports generalisations of the kind initiative x produces outcome y... A more useful question to ask: Which factors (enabling and disabling) shape the possibility that TAIs will achieve their stated goals in a particular context?
Challenge 3 (cont.): Factors State (supply side) Citizen voice (demand side) Level of democratisation Level of political will Broader enabling legal frameworks, incentives and sanctions Capabilities of citizens and civil society Interaction of TAIs with other mobilisation and collective action Embeddedness of TAIs in broader policy processes Linking Mechanisms
Challenge 3 (cont.): Factors Thinking beyond the dichotomies (state-society; voice response; supply-demand) New thinking in governance would urge us to pay more attention to: Multiple actors, accountability coalitions, networked approaches to governance Changing norms and cultures of accountability in state, private sector and civil society simultaneously Looking across levels and scales: Linking the local, national, regional, international Bringing politics back in: Power The black box of political will and political economy Links to parties, elections and political regimes
Key Lessons The evidence base (+/-) is weak - but that doesn t mean that TAIs are not significant. The challenge is to deepen the evidence and the methods for developing it. On the state of the evidence: Develop new approaches to assessment, with complexity perspective, that combine methods and approaches Explore further user-centred and participatory approaches Support comparative in-depth work across contexts and TAIs, multi-case and other synthetic analysis Strengthen capacities of researchers and practitioners to develop and build on innovative approaches Build into new TAIs ToCs, baselines, comparators, etc
Key lessons (cont.) On factors for greater impact: Deepen understandings of synergies between T, A, participation and voice Move beyond dichotomies to build new knowledge on cross-cutting accountability coalitions Apply to T&A field the cutting-edge thinking on governance, especially networked governance; interaction of global national local; and private sector Explore whether initiatives can travel across context, method and issue
For links to these studies www. drc-citizenship.org www.ids.ac.uk Gaventa and Barrett, So what difference does it make? Mapping the outcomes of Citizen Engagement, IDS Working paper 247 Review of Impact and Effectiveness of Transparency and Accountability Initiatives, Rosemary McGee & John Gaventa with contributions from Greg Barrett, Richard Calland, Ruth Carlitz, Anuradha Joshi and Andrés Mejía Acosta, IDS, October 2010