Bridging Research, Policy and Practice

Overview Bridging Research, Policy and Practice John Young: ODI, London j.young@odi.org.uk Sudarno Sumarto: SMERU, Indonesia ssumarto@smeru.or.id Addressing Poverty: Pro-Poor Growth and Financial Inclusion in Asia Pacific Introduction to ODI & RAPID Definitions Policy processes & the role of research Policy-makers perspective Some theory A systematic approach Some tools An example Some conclusions Sources of further information. ODI, UK Development Think Tank 60 researchers Research / Advice / Public Debate Rural / Humanitarian / Poverty & Aid / Economics / Policy Processes DFID, Parliament, WB, EC Civil Society RAPID Group Promoting the use of research-based evidence in development policy Research / Advice / Public Affairs & Capacity-building Programmes: Research for Policy Progressive Policymakers Parliamentarians Southern Think Tanks For more information see: www.odi.org.uk for further information see: www.odi.org.uk/rapid SMERU, Indonesia Independent research and policy studies on the impact of government programs and policies, poverty, health, education, gender, labour, fuel costs labour market conditions etc decentralization and impact on services etc. Publications and public affairs. NGO unit: provides research-based evidence to and facilitates NGO interaction capacity building: research methodology. Civil Society Programme More use of researchbased evidence for policy and practice through: A Network of Think Tanks. Capacity-development for the network and other CSOs. Research and learning from practical experience. Global and national action-research projects. for further information see: www.odi.org.uk/cspp 1

End of slide show, click to exit Group work: Stories of evidence and Policy Change Stories of evidence and policy Refresh your memory about your story Find someone you don t know and tell each other your stories Don t write anything down! 2½ minutes each! Use the story templates to interview and write each others story down Make sure your images and messages are clear. 5 minutes each! Tell your partner s story to everyone else at your table 2½ minutes each! Discuss among yourselves and identify 5 concrete actions emerging from these stories. How might you apply these? 10 minutes! The theory and practice of bridging research and policy Definitions Research: any systematic effort to increase the stock of knowledge Policy: a purposive course of action followed by an actor or set of actors Evidence: the available information supporting or otherwise a belief or proposition Evidence-based Policy: public policy informed by rigorously established evidence. Policy Processes Identify the problem Commission research Analyse the results Choose the best option Establish the policy Implement the policy Evaluation Donors Civil Society Policy Processes Private Sector Agenda Setting Monitoring and Evaluation Policy Formulation Decision Making Policy Implementation Cabinet Parliament Ministries 2

Chronic Poverty in Uganda Kate Bird et al, Fracture Points in Social Policies for Chronic Poverty Reduction, ODI WP242, 2004 (http://www.odi.org.uk/publications/working_papers/wp242.pdf) in reality The whole life of policy is a chaos of purposes and accidents. It is not at all a matter of the rational implementation of the so-called decisions through selected strategies 1 Most policy research on African agriculture is irrelevant to agricultural and overall economic policy in Africa 2 Research is more often regarded as the opposite of action rather than a response to ignorance 3 1 Clay & Schaffer (1984), Room for Manoeuvre; An Exploration of Public Policy in Agricultural and Rural Development, Heineman Educational Books, London 2 Omamo (2003), Policy Research on African Agriculture: Trends, Gaps, and Challenges, International Service for National Agricultural Research (ISNAR) Research Report No 21 3 Surr (2003), DFID Research Review But Policy makers are practically incapable of using research-based evidence because of the 5 Ss Speed Superficiality Spin Secrecy Scientific Ignorance Vincent Cable Lib. Democrat MP & Shadow Minister of Finance More at: www.odi.org.uk/rapid/meetings/evidence Factors influencing policy making Experience & Expertise Pragmatics & Contingencies Judgement Lobbyists & Pressure Groups Habits & Tradition Evidence Values and Policy Context Resources Source: Phil Davies Impact to Insight Meeting, ODI, 2005 Different Notions of Evidence Researchers Evidence Scientific (Context free) Proven empirically Theoretically driven As long as it takes Caveats and qualifications Policy Makers Evidence Colloquial (Contextual) Anything that seems reasonable Policy relevant Timely Clear Message Source: Phil Davies Impact to Insight Meeting, ODI, 2005 Existing theory X 1. Linear model 2. Percolation model, Weiss 3. Tipping point model, Gladwell 4. Context, evidence, links framework, ODI 5. Policy narratives, Roe 6. Systems model (NSI) 7. External forces, Lindquist 8. Room for manoeuvre, Clay & Schaffer 9. Street level bureaucrats, Lipsky 10. Policy as social experiments, Rondinelli 11. Policy Streams & Windows, Kingdon 12. Disjointed incrementalism, Lindquist 13. The tipping point, Gladwell 14. Crisis model, Kuhn 15. Framework of possible thought, Chomsky 16. Variables for Credibility, Beach 17. The source is as important as content, Gladwell 18. Linear model of communication, Shannon 19. Interactive model, 20. Simple and surprising stories, Communication Theory 21. Provide solutions, Marketing I 22. Find the right packaging, Marketing II 23. Elicit a response, Kottler 24. Translation of technology, Volkow 25. Epistemic communities 26. Policy communities 27. Advocacy coalitions etc, Pross 28. Negotiation through networks, Sebattier 29. Shadow networks, Klickert 30. Chains of accountability, Fine 31. Communication for social change, Rockefeller 32. Wheels and webs, Chapman & Fisher 3

Existing theory a short list Policy narratives, Roe Systems of Innovation Model, (NSI) Room for manoeuvre, Clay & Schaffer Street level bureaucrats, Lipsky Policy as social experiments, Rondene Policy streams and policy windows, Kingdon Disjointed Incrementalism, Lindblom Social Epidemics, Gladwell The RAPID Framework An Analytical Framework External Influences Socio-economic and cultural influences, donor policies etc The links between policy and research communities networks, relationships, power, competing discourses, trust, knowledge etc. The political context political and economic structures and processes, culture, institutional pressures, incremental vs radical change etc. The evidence credibility, the degree it challenges received wisdom, research approaches and methodology, simplicity of the message, how it is packaged etc Case Studies Detailed: Sustainable Livelihoods Poverty Reductions Strategy Processes Ethical Principles in Humanitarian Aid Animal Health Care in Kenya Dairy Policy in Kenya Plant Genetic Resources Summary GDN x 50 CSPP x 20 Good news case studies x 5 Mental health in the UK 1970s 1980s 1990s Professionalisation of Public International Services. Research Structural Adjustment collapse of services. Paravet projects emerge. ITDG projects. collaborative research. Privatisation. ITDG Paravet network. and change of DVS. Rapid spread in North. The Hubl Study Dr Kajume KVB letter (January 1998). 2000s Animal Healthcare in Kenya Multistakeholder WSs new policies. Still not approved / passed! A Practical Framework External Influences Campaigning, Lobbying Politics and Policymaking political context Policy analysis, & research What you need to know The external environment: Who are the key actors? What is their agenda? How do they influence the political context? The evidence: Is it there? Is it relevant? Is it practically useful? Are the concepts new? Does it need re-packaging? Scientific information exchange & validation Media, Advocacy, Networking links Research, learning & thinking evidence The political context: Is there political interest in change? Is there room for manoeuvre? How do they perceive the problem? Links: Who are the key actors? Are there existing networks? How best to transfer the information? The media? Campaigns? 4

What you need to do Policy entrepreneurs What need to know What need to do How to do it Political Context: Who are the policymakers? Is there demand for ideas? What is the policy process? Get to know the policymakers. Identify friends and foes. Prepare for policy opportunities. Look out for policy windows. Work with them seek commissions Strategic opportunism prepare for known events + resources for others Storytellers Networkers Evidence What is the current theory? What are the narratives? How divergent is it? Establish credibility Provide practical solutions Establish legitimacy. Present clear options Use familiar narratives. Build a reputation Action-research Pilot projects to generate legitimacy Good communication Engineers Fixers Links Who are the stakeholders? What networks exist? Who are the connectors, mavens and salesmen? Get to know the others Work through existing networks. Build coalitions. Build new policy networks. Build partnerships. Identify key networkers, mavens and salesmen. Use informal contacts Conclusions To influence policy you need: Clear intent A thorough understanding of the context A strategic approach The right incentives / culture The right skills in the team To engage, engage, engage End of slide show, click to exit How SMERU influences policy and practice 5

BRIDGING RESEARCH POLICY AND PRACTICE: SMERU S EXPERIENCE A Case Study of Indonesia s Unconditional Cash Transfer SUDARNO SUMARTO The SMERU Research Institute Jakarta, Indonesia Background Since the New Order era, domestic retail fuel price in Indonesia has been subsidized. The fuel subsidy is regressive as consumption of each energy category increases with income. The subsidy was not pro-poor because the poor benefited less from it, including the kerosene subsidy. Very high world price of oil in 2005 has made the subsidy balloon out of proportion. CSPP Partner s Workshop 2006 Canberra, Australia 7 December 2006 Background To ease the pressure on the state budget from the ballooning subsidy and to improve spending allocation, GOI has increased the fuel price twice in 2005: 1 March, by an average of 30% compensation program: free education & healthcare, rice for the poor, rural infrastructure. 1 October, by an average of 120% compensation program, among others, unconditional cash transfer (UCT) to poor households. Brief Description of the UCT Program The largest cash transfer program in the world, targeting more than 15 million households starting in October 2005. Transfer of Rp 300,000 per beneficiary household per quarter for 1 year. Eligibility was determined by a household census conducted by BPS using district-specific proxy means testing based on 14 indicators of poverty. Delivery of the UCT is conducted through the post office by way of direct payment. An additional 2.5 million households were added as eligible recipients after the first tranche. SMERU's Involvement in Evaluating UCT SMERU carried out simulations on the likely impact of fuel price increase on poverty (based on Susenas 2004 data): Baseline: poverty head count (P0) = 16.66% After oil price increase: P0 = 22.05% Compensation through cash transfer: 100% correct targeting: P0 = 17.87% 80% correct targeting: P0 = 18.73% 50% correct targeting: P0 = 20.05% SMERU's Involvement in Evaluating UCT SMERU conducted a rapid appraisal in Jakarta at the end of October 2005. The results were then disseminated to policymakers through consultations and presentations of research findings. Bappenas commissioned SMERU to conduct a larger evaluation, funded by the World Bank. The larger evaluation was conducted in 5 districts across Indonesia. 6

Bridging Research, Policy and Practice, Thursday 7th December 2006, University House, ANU, Canberra. SMERU s Policy Inputs Convert the UCT to conditional cash transfers (CCT) on education, health, and nutrition Improve targeting by reformulating criteria, strengthening local cadre, and empowering locally-specific poverty assessments Develop cash for work or guaranteed employment scheme to improve rural infrastructure Strengthen the supply sides of social services (health, nutrition, and education facilities) Results from SMERU's Inputs The government will pilot test CCT in 2007 The government intensified its effort to increase the budget allocation for education and health What Worked? Stakeholders trusted SMERU s credibility because of its evidence-based recommendations SMERU s approach with gov t and top level officials Effective linkages and networking with stakeholders Maintenance of quality of research Impartiality Successful presentation of findings through discussions Constraints Encountered Political leverage Lack of tools to understand the political context of policy change Lack of lobbying skills & opportunities Lack of resources (human and financial) and time to monitor and follow-up the policy decision-making process SMERU being associated with donors The SMERU Research Institute www.smeru.or.id 7

End of slide show, click to exit Tools for bridging research and policy Introducing some tools Some simple tools for researchers RAPID Framework Outcome Mapping Problem Situation Analysis (Tree Analysis) Policy Process Mapping Stakeholder Analysis Force field analysis SWOT analysis Some examples Some more complex tools (for donors) Some tools for policymakers A Practical Framework Policy entrepreneurs External Influences political context Campaigning, Lobbying Politics and Policymaking Policy analysis, & research Storytellers Networkers Scientific information exchange & validation Media, Advocacy, Networking links Research, learning & thinking evidence Engineers Fixers Practical Tools RAPID Framework Overarching Tools - The RAPID Framework - Using the Framework - The Entrepreneurship Questionnaire Communication Tools - Communications Strategy - SWOT analysis - Message Design - Making use of the media Policy Influence Tools - Influence Mapping & Power Mapping - Lobbying and Advocacy - Campaigning: A Simple Guide - Competency self-assessment Context Assessment Tools - Stakeholder Analysis - Forcefield Analysis - Writeshops - Policy Mapping - Political Context Mapping Research Tools - Case Studies - Episode Studies - Surveys - Bibliometric Analysis - Focus Group Discussion 8

Outcome Mapping OUTCOME MAPPING: Building Learning and Reflection into Development Programs Sarah Earl, Fred Carden, and Terry Smutylo http://www.idrc.ca/en/ev-9330-201-1-do_topic.html Problem Tree Analysis 1. Discuss and agree the problem or issue to be analysed. 2. Identify the causes of the focal problem (roots) and then the consequences (branches) NB: The discussion is the most important thing: as issues are debated factors are arranged and re-arranged, often forming sub-dividing roots and branches Policy Process Mapping General Context issues domestic and international. Specific Policy Issues (i.e. the policy cycle) Stakeholder analysis Arena: government, parliament, civil society, judiciary, private sector. Level: local, national, international What is their Interest and Influence? Process matrix + political matrix Political and administrative feasibility assessment [Sources: M. Grindle / J. Court ] Policy Process Mapping Formulation Implementation Politicians Cabinet Government Bureaucrats Civil Society International Stakeholder Analysis Forcefield Analysis Why: To understand who gains or lose from a policy or project. To help Build Consensus. Steps: 1. Identify Stakeholders 2. Analysis Workshop 3. Develop Strategies High Power Low Low Keep Satisfied Monitor (minimum effort) Interest Engage Closely Keep Informed High Identify what you want to achieve Identify forces for and against change Identify which are most important Develop strategies to reinforce those for and overcome those against 9

SWOT Analysis What type of policy influencing skills and capacities do we have? In what areas have our staff used them more effectively? Who are our strongest allies? When have they worked with us? Are there any windows of opportunity? What can affect our ability to influence policy? Strengths Opportunities Weaknesses Skills and abilities Funding lines Commitment to positions Contacts and Partners Existing activities Threats Other orgs relevant to the issue Resources: financial, technical, human Political and policy space Other groups or forces Communications strategy Identify the audience(s) Identify the message(s) Promotion Evaluate impact and change as necessary Clear Strategy Interactive Multiple formats Who? How? What? Writing Policy Papers Providing a solution to a policy problem The policy community The policy process Structural elements of a paper Problem description Policy options Conclusion Key issues: Problem oriented, targeted, multidisciplinary, applied, clear, jargonfree. [Source: Young and Quinn, 2002] Groundwater in India to maximise impact of DFID forest/ ground water research project in India Researchers, policy makers and activists Used framework to analyse factors in water sector in India Developed strategy for final phase: Less research More communication Developing champions in regional and national government Local, Regional & National advocacy campaign SMEPOL Project Egypt Policy Process Mapping RAPID Framework Stakeholder Analysis Force-Field Analysis SWOT Action Planning Political Context Tools Civil Society Index (CIVICUS) Country Policy & Institutional Assessment (World Bank) Democracy and Governance Assessment (USAID) Drivers of Change (DFID) Governance Questionnaire (GTZ) Governance Matters (World Bank Institute) Power Analysis (Sida) World Governance Assessment 10

Demand-side Tools Increasing the pull for evidence Require the publication of the evidence base Require spending bids to provide evidence base Submit government analysis to external expert scrutiny Provide open access to information Facilitating better evidence use Encourage better collaboration across analytical services Co-locate policy makers and internal analysts Integrate analytical staff at all stages Link R&D strategies to departmental business plans Cast external researchers more as partners than as contractors Second more university staff into government Train staff in evidence use Source: Abstracted from PIU 2000, Bullock et al (2001) UK Government Tools Overview and Checklist 1. Impact Assessment and Appraisal: guidance checklist for policy makers. Strategy and Policy Evaluation 2. Strategy Survival Guide 3. Magenta Book: Guidance notes on Policy Evaluation 4. Green Book: Appraisal and evaluation in Central Government 5. Regulatory Impact Assessment (RIA) Ensuring Key Perspectives are Included 6. Incorporating regional perspectives into policy making toolkit (Subnational) 7. International Comparisons in Policy Making Toolkit 8. Gender Impact Assessment: a framework for gender mainstreaming 9. Managing risks to the public: Appraisal Guidance Testing Policy Ideas 10. Policy Pilots Public-orientated Tools 11. Concern Assessment Tool 12. Community Engagement How to Guide 13. Connecting with Users and Citizens Getting Better Advice and Evidence 14. Expert Advisory Bodies for Policymakers 15. Improving Standards of Qualitative Research End of slide show, click to exit Group work: Trying out some of the tools Trying the tools Working on one of the stories, try out one (or more) of the tools: RAPID Framework (Page 12) Stakeholder analysis (Page 24) Force-field analysis (page 20) Problem Tree (Page 22) SWOT (Page 28) Building effective organisations The (changing) role of CSOs Is huge: Worth $12bn globally, reach 20% of world s poor, provide 40% health & education services in Ghana, Zimbabwe & Kenya. Is changing: service provision policy engagement. Can be very effective: Globally, eg Jubilee 2000; locally eg animal health in Kenya, Is uncomfortable: with governments: lack of trust with donors: emphasis on GBS & policy with academics/policy advisers: weak evidence 11

Some examples AFREPREN: a network to promote propoor energy policies. CIPPEC: works on Education, Fiscal Policy, Health, Transparency and Justice an Argentina. Unnayan Onneshan: works on pro-poor agricultural and trade policies in Bangladesh. International Budget Project: works to promote budget transparency and accountability SMERU: provides reseach-based evidence to NGOs and other actora and promotes public debate How CSOs influence Policy How CSOs influence Policy Obstacles to CSO Engagement Obstacles for Research Key problems and solutions External Difficult Political Contexts Internal Weak understanding of political contexts Weak engagement Inadequate use of evidence Weak communication Isolation Capacity constraints Campaigns Boomerangs Policy Pilots Rigorous context assessments Better strategies for engagement at all parts of the policy cycle Collecting the right evidence for each situation (qualitative vs quantitative etc) Better communication: publications, events, faceto-face Collaboration with other CSOs, donors and government agencies: Networks Systemic capacity-building: of organisations and networks within their contexts 12

Organisational Capacity Knowledge, Attitudes & Practice Policy entrepreneurs Training & mentoring etc Organisational development Finance, admin & personnel systems Strategic (action & business) planning Fundraising & reporting Struyk, 2002, Local Governance Institute, Open Society Network, Budapest Building an organisational profile Communications, Public Affairs and the Media Learning & KM Learning before, during & after Peer Assist Reflective Inquiry After-Action Review Collaboration Tools E-discussions Shared workspaces Information management tools Intelligent search engines Incentives Policy Processes in DFID Looking at internal policy processes what works in DFID. Small, informal workshop with 7 staff. Participatory pair-wise ranking of factors influencing the success of 8 policy processes. Worked quite well. In DFID - agendas and processes rather than documents are key End of slide show, click to exit How SMERU does it 13

The SMERU Research Institute SUDARNO SUMARTO The SMERU Research Institute Jakarta, Indonesia What is SMERU? SMERU is an independent research institute that undertakes research and policy studies on socioeconomic, poverty, decentralization, and vulnerability issues considered most urgent and relevant for the welfare of the people of Indonesia SMERU adopts both quantitative and qualitative research methods to carry out these studies CSPP Partner s Workshop 2006 Canberra, Australia 7 December 2006 What is SMERU? SMERU engages in policy dialogues and provides inputs SMERU enhances the capacity of NGOs through training and discussions Organizes workshops, seminars, and discussions as part of the sharing and networking process Regularly publishes papers, reports, and newsletters and maintains a website to disseminate research findings How and Why SMERU Was Set Up? In October 1998, SMERU (previously known as the Social Monitoring & Early Response Unit) was established with a mandate to carry out independent, reliable, real-time monitoring of the social impact of the crisis unfolding in Indonesia It was funded by AusAID, ASEM, and USAID, and with logistical and administrative support from the World Bank. SMERU's mandate from its stakeholders came to an end by December 2000 The core group of staff and researchers continued SMERU's existence as an independent institution for research and policy studies under the name of The SMERU Research Institute Initial Sources of SMERU s Core Funding (Year 2001) AusAID The Ford Foundation DFID (ended in 2005) Other Sources of Funding Commissioned research projects and training How Does SMERU Operate? Planning Based on strategic planning and Vision 2010 Administrative and financial matters: in consultation with its management and staff, and occasionally with the Board of Governors Research matters: in consultation with designated research teams Reporting Project Consultative Committee Meetings and BOG meetings; quarterly and biannual reports 14

Bridging Research, Policy and Practice, Thursday 7th December 2006, University House, ANU, Canberra. Maintenance of Academic Credibility Peer review/refereed journals Linkage with universities, research institutes, development organizations Participation in local and international seminars, conferences, and workshops through presentations of papers and attendance as participants Maintenance of Independence SMERU does not belong to any entity No kickback remains impartial, transparent and accountable Third party agreement SMERU has the right to co-own the data and publish the research findings Research outcome is not influenced by political or religious and other entities, and is not being controlled by any entity Not driven by donors agenda Engagement with Policy Processes Interacting, organizing seminars and workshops, disseminating SMERU s publications, research findings and recommendations, sending memorandum/policy briefs to relevant high-ranking government officials, donors, and other stakeholders. Establishing networks and collaboration/coalition with CSOs, and lobbying with MPs Engagement with Policy Processes Disseminating information/research findings towards forming better-informed CSOs Providing free access to all SMERU s publications, reports, working papers, and NGO Database through the SMERU Website: www.smeru.or.id Linking SMERU s website to others CHALLENGES.. Developing trust and credibility through better research quality Understanding and dealing with other influencing contextual factors in bridging research to policy Striving to be financially sustainable by maintaining the research quality to win the trust of stakeholders and donors Balancing consultancy work and SMERU s own research agenda Widening networks and linkages Facilitating linkages and cooperation among CSOs, enhancing capacity and promoting public accountability The SMERU Research Institute www.smeru.or.id 15

End of slide show, click to exit Conclusions and sources of more information Conclusions It is possible to improve the impact of research on policy if you: really, really want to do it understand the (esp. political) context use a strategic approach establish the right incentives / culture develop the right skills in the team engage, engage, engage There are some good tools There are a growing number of organisations who can help Civil Society Programme Further Information ODI www.odi.org.uk RAPID - www.odi.org.uk/rapid CSPP www.odi.org.uk/cspp or contact: John Young j.young@odi.org.uk SMERU: www.smeru.or.id or contact: Sudarno Sumarto sumarto@smeru.or.id 16