Public participation and democratic legitimacy: A framework for evaluation Dr. Bastian Rottinghaus, Dr. Tobias Escher, Katharina Gerl, M.A. ECPR General Conference, Oslo, September 7th - 9th 2017 Featured Panel: Smart Cities: Politics, Policy and Governance
Motivation public participation: processes that are purposefully instituted in order to involve the public or particular stakeholders in the process of administrative decision-making in democracies widespread use of participation processes (and growing), in particular online But many open questions: 1. What is effective public participation? / When is it successful? 2. Is public participation effective / successful? 3. Which factors contribute to effective public participation?
Research Gap scientific evidence for the (substantial, instrumental and/or normative) effectiveness of public participation is still lacking lack of standardized evaluation procedures, widely accepted criteria of success validated success factors (Aichholzer et al. 2016; Geissel 2012; Kersting 2008; Macintosh & Whyte 2008; Pratchett et al. 2009) comparative research overall fundamental theoretical deficits (Dietz & Stern 2008; Macintosh et al. 2009)
Research Project Research gap potentially closed by: 1. deducing from theory of democracy & inducing from evaluation literature => catalogue of analytical criteria to assess the effectiveness of (Online-) Public Participation => Analytical framework 2. operationalizing these criteria 3. deployment of framework in evaluation research to determine the hypothesized effects quantitatively => Pilot project, initiating online public participation events in three German municipalities (Bonn, Cologne, Moers) => currently underway
Object of Research Invited Spaces rather than Invented spaces : Formal Top-Down- Initiation Informal Top-Down- Initiation Formal Bottom-Up- Initiation Informal Bottom-Up- Initiation
Induction Deduction Analytical Framework: Epistemological Basis Theory of Democracy Analytical Framework Evaluation Literature
Causal Model
Process-Criteria Core Principle of Democracy Political Equality: Equal rights to participate (Bühlmann et al. 2008, Dahl 1989, Diamond & Morlino 2004) Inclusiveness Corresponding Criteria in Evaluation Literature Inclusiveness (e.g. Kubicek, Lippa and Koop, 2011; Geißel, 2008; Franklin, Krane and Ebdon, 2013; Martini and Fritzsche, 2015; Smith, 2009; Vetter, Geyer and Eith, 2015) Representativeness (e.g. Rowe and Frewer, 2000) Reach (e.g. Kubicek, Lippa and Koop, 2011
Process-Criteria Core Principles of Democracy Political Equality: Equal rights to participate (Bühlmann et al. 2008, Diamond & Morlino 2004), free discourse (Habermas 1981) Fairness Corresponding Criteria in Evaluation Literature Fairness (Renn, Webler and Wiedemann, 1995; Barnes, 1999; Engagement Global, 2012; Goldschmidt, 2014) Degree of deliberation (Lippa et al., 2008) Efficacy (Mkude, Pérez-Espés and Wimmer, 2014) Influence on internal debate (Geißel 2008)
Process-Criteria Core Principles of Democracy Control, Informational openness, Enlightened understanding (Bühlmann et al. 2008, Dahl 1989, Diamond & Morlino 2004) Transparency Corresponding Criteria in Evaluation Literature Transparency (Goldschmidt, 2014; Lippa et al., 2008; Rowe and Frewer, 2000, 2004; Franklin, Krane and Ebdon, 2013; Martini and Fritzsche, 2015; Smith, 2009; Vetter, Geyer and Eith, 2015) Feedback (Lippa et al. 2008) Independence (Rowe & Frewer 2000) Provision of broad information (Martini & Fritzsche 2015)
Process-Criteria Core Principle of Democracy Freedom (Bühlmann et al. 2008, Diamond & Morlino 2004) Efficiency Corresponding Criteria in Evaluation Literature Efficiency (Joss 1995; Smith 2009; Kubicek et al. 2011; Nabatchi 2012; Goldschmidt 2014; Mkude et al. 2014) Speed of Decision-making (Geißel 2008) Structured decision making, Costeffectiveness (Rowe & Frewer 2000) Program Organization (Nabatchi 2012) Organizational flawlessness (Vetter et al. 2015)
Result-Criteria (Output) Goal Attainment & Substantive Quality Core Principle of Democracy Corresponding Criteria in Evaluation Literature Output-Legitimacy (Scharpf 1999) Problem-solving-capacity of the procedure (Geißel 2008) Quantity and quality of contributions made by participants (Kubicek et al 2011) Adequate closure of the participation event/ quality of results (Goldschmidt 2014) Participant Satisfaction (Nabatchi 2012) Quality of substantial Output (Newig et al (2013) Perceived quality of Output (Range & Faas 2016) Transferability (Smith 2009)
Result-Criteria (Outcome) Formal Implementation & Substantive Quality Core Principle of Democracy Output-Legitimacy (Scharpf 1999) Corresponding Criteria in Evaluation Literature Influence on results (Kubicek et al. 2011) Transformation and formal implementation (Goldschmidt 2014) Influence on formal implementation (Geißel 2008; Rowe, Marsh, and Frewer 2004; Lippa et al. 2008) Output-related effects of ingle participation events on policy (Vetter et al. 2015)
Result-Criteria (Impact I) Changes in Civic Skills und Civic Culture Core Principle of Democracy Corresponding Criteria in Evaluation Literature Output-Legitimacy (Scharpf 1999) Competence development / societal effects (Goldschmidt 2014) From singular participation events dissociated effects on: (1) processes and parties in politics and administration, (2) the citizenry (Vetter et al. 2015) Promotion of democratic virtues(kubicek et al. 2011) Increase of political knowledge, Raise of identification with democracy (attitudes) and improvement of civic skills (Geißel 2008) Increasing trust in institutions (Beierle (1999)
Result Criteria (Impact II) Core Principle of Democracy Changes in Target Indicators Corresponding Criteria in Evaluation Literature Output-Legitimacy (Scharpf 1999) Improving the substantive quality of decisions(beierle 1999) Actual impact (Guston 1999) General Outcomes (Benefits for Individuals/ Benefits for Community/ Benefits for the Agency/ Benefits for Policy or Public Action) (Nabatchi 2012)
Summary of Criteria Evaluation Criteria Relevant Stakeholders Level of Measurement Process Throughput-Legitimacy: o Transparency o Inclusiveness o Fairness o Efficiency Participants Organizers Perceptions by relevant Stakeholders Objective Indicators Output Output-Legitimacy (Process): o (Degree of) Goal Attainment o Substantial Quality Participants Organizers Political Decision Makers Perceptions by relevant Stakeholders Objective Indicators Outcome Output-Legitimacy (formal implementation) o (Degree of) formal implementation o Substantial Quality Participants Organizers Political Decision Makers Population affected by measures Perceptions by relevant Stakeholders Objective Indicators Impact Pre-Post-Change (singular participation event)/ Variations between contexts of participation events: o Civic Skills o o Civic Culture Target Indicators concerning the subject of participation Participants Organizers Political Decision Makers Population affected by measures Attributes of relevant Stakeholders Context Indicators
Contact Dr. Tobias Escher escher@diid.hhu.de Dr. Bastian Rottinghaus rottinghaus@diid.hhu.de Katharina Gerl, M.A. gerl@diid.hhu.de Düsseldorf Institute for Internet and Demokratie Heinrich-Heine-Universität Düsseldorf Universitätsstraße 1 Gebäude 25.12, Ebene 01, Raum 32 40225 Düsseldorf E-Mail: diid@hhu.de Web: www.diid.hhu.de Telefon: +49 211 81-10429 Twitter: @diid_hhu
References Aichholzer, G., Kubicek, H., & Torres, L. (2016). Evaluating e-participation. Frameworks, Practice, Evidence. (G. Aichholzer, H. Kubicek, & L. Torres, Eds.). Springer International Publishing. https://doi.org/10.1007/978-3-319-25403-6 Barnes, M. (1999). Building Deliberative Democracy: An Evaluation of Two Citizens Juries. Institute for Public Policy Research. Beierle, T. C. (2005). Using Social Goals to Evaluate Public Participation in Environmental Decisions. Review of Policy Research, 16(3 4), 75 103. https://doi.org/10.1111/j.1541-1338.1999.tb00879.x Bühlmann, Marc; Merkel, Wolfgang; Wessels, Bernhard; Müller, Lisa (2008): The quality of democracy: democracy barometer for established democracies. In: National Centre of Competence in Research (NCCR). Challenges to Democracy in the 21st Century (Working Paper No. 10a) Dahl, Robert A. (1989): Democracy and its Critics. New Haven/London: Yale University Press Diamond, L. and Morlino, L. (2004): The Quality of Democracy - An Overview. In: Journal of Democracy 15 (4): 20-31 Dietz, T., & Stern, P. (2008). Public participation in environmental assessment and decision making. National Research Council. Retrieved from http://books.google.com/books?hl=en&lr=&id=6os69zzngl8c&oi=fnd&pg=pa1&dq=public+participation+in+environmental+assessment+ and+decision+making&ots=a78ctahrtk&sig=9tcfzo6i85ine8hp_ud2jdcntfs Engagement Global. (2012). Bericht zum Workshop Erfolg oder Misserfolg? ( Wie ) ist eine Evaluation von Bürgerhaushalten möglich? Bonn. Retrieved from http://www.buergerhaushalt.org/article/bericht-zum-workshop-erfolg-oder-misserfolg-wie-ist-eine-evaluation-vonb?rgerhaushalten Franklin, A., Krane, D., & Ebdon, C. (2013). Multilevel governance processes - citizens & local budgeting: Comparing brazil, china, & the United States. Public Administration Faculty Publications, 18(1), 121 144. Retrieved from http://www.scopus.com/inward/record.url?eid=2-s2.0-84884382139&partnerid=tzotx3y1 Geißel, B. (2008). Wozu Demokratisierung der Demokratie? Kriterien zur Bewertung partizipativer Arrangements. In A. Vetter (Ed.), Erfolgsbedingungen lokaler Bürgerbeteiligung (pp. 29 48). Wiesbaden: VS Verlag fu r Sozialwissenschaften. https://doi.org/10.1007/978-3- 531-09026-9_2
References Geißel, B. (2008). Zur Evaluation demokratischer Innovationen die lokale Ebene. In H. Heinelt & A. Vetter (Eds.), Lokale Politikforschung heute (pp. 227 248). Wiesbaden. Retrieved from http://link.springer.com/chapter/10.1007/978-3-531-91111-3_9 Geissel, B. (2012). Democratic innovations - Theoretical and empirical challenges of evaluation. In B. Geißel & K. Newton (Eds.), Evaluating democratic innovations: curing the democratic malaise? (pp. 209 214). Abingdon: Routledge. Goldschmidt, R. (2014). Kriterien zur Evaluation von Dialog- und Beteiligungsverfahren. Wiesbaden: Springer Fachmedien Wiesbaden. https://doi.org/10.1007/978-3-658-06120-3 Guston, D. H. (1999). Stabilizing the Boundary between US Politics and Science: Social Studies of Science, 29(1), 87 111. https://doi.org/10.1177/030631299029001004 Habermas, Jürgen (1981): Theorie des kommunikativen Handelns. (Bd.1: Handlungsrationalität und gesellschaftliche Rationalisierung, Bd. 2: Zur Kritik der funktionalistischen Vernunft), Frankfurt am Main Kersting, N. (2008). Evaluation dialogischer Beteiligungsinstrumente. In N. Kersting (Ed.), Politische Beteiligung (pp. 270 292). Wiesbaden: VS Verlag fu r Sozialwissenschaften. https://doi.org/10.1007/978-3-531-91071-0_17 Kubicek, H., Lippa, B., & Koop, A. (2011). Erfolgreich beteiligt? Nutzen und Erfolgsfaktoren internetgestuetzter Buergerbeteiligung - Eine empirische Analyse von 12 Fallbeispielen. Guetersloh: Verlag Bertelsmann Stiftung. Lippa, B., Aichholzer, G., Allhuter, D., Freschi, A. C., Macintosh, A., Moss, G., & Westholm, H. (2008). eparticipation Evaluation and Impact (D13.3 Demo-net booklet). Evaluation. Bremen/Leeds. Macintosh, A., & Whyte, A. (2008). Towards an evaluation framework for eparticipation. Transforming Government: People, Process & Policy, 2(1), 16 30. Macintosh, A., Coleman, S., & Schneeberger, A. (2009). eparticipation: The Research Gaps. Electronic Participation, 1 11. Retrieved from http://www.springerlink.com/index/h8176658567u1546.pdf Martini, M., & Fritzsche, S. (2015). Kompendium Bürgerbeteiligung. Rechtliche Rahmenbedingungen kommunaler Beteiligungsangebote im Internet. München: Innovationsstiftung bayerische Kommune. Retrieved from http://www.bayinnovationsstiftung.de/fileadmin/docs/obb/online-buergerbeteiligung.pdf
References Mkude, C. G., Pérez-Espés, C., & Wimmer, M. A. (2014). Participatory budgeting: A framework to analyze the value-add of citizen participation. In Proceedings of the Annual Hawaii International Conference on System Sciences (pp. 2054 2062). IEEE Computer Society. Nabatchi, T. (2012). A Manager s Guide to Evaluating Citizen Participation. Washington. Retrieved from http://www.businessofgovernment.org/report/manager s-guide-evaluating-citizen-participation Newig, J., Adzersen, A., Challies, E., Fritsch, O., & Jager, N. (2013). Comparative analysis of public environmental decision-making processes a variable-based analytical scheme. leuphana.de. Lüneburg. Retrieved from https://www.leuphana.de/fileadmin/user_upload/forschungseinrichtungen/infu/files/infu-reihe/37_13.pdf Pratchett, L., Durose, C., Lowndes, V., Smith, G., Stoker, G., & Wales, C. (2009). Empowering communities to influence local decision making. A systematic review of evidence. London. Range, J., & Faas, T. (2016). Politische Kultur, Bürgerbeteiligung und wissenschaftliche Evaluation. In M. Glaab (Ed.), Politik mit Bürgern - Politik für Bürger (pp. 111 131). Wiesbaden: Springer Fachmedien Wiesbaden. https://doi.org/10.1007/978-3-658-12984-2_7 Renn, O., Webler, T., & Wiedemann, P. M. (1995). Fairness And Competence In Citizen Participation: Evaluating Models for Environmental Discourse. Springer. Retrieved from http://www.amazon.com/fairness-competence-citizen-participation-environmental/dp/0792335171 Rowe, G., & Frewer, L. J. (2000). Public Participation Methods: A Framework for Evaluation. Science, Technology & Human Values, 25(1), 3 29. https://doi.org/10.1177/016224390002500101 Smith, G. (2009). Democratic Innovations: Designing Institutions for Citizen Participation (Theories of Institutional Design). Cambridge University Press. Retrieved from http://www.amazon.com/democratic-innovations-institutions-participation-institutional/dp/0521730708 Vetter, A., Geyer, S., & Eith, U. (2015). Die wahrgenommenen Wirkungen von Bürgerbeteiligung. In Baden-Württemberg Stiftung (Ed.), Demokratie-Monitoring Baden-Württemberg 2013/2014 (pp. 223 342). Wiesbaden: Springer Fachmedien Wiesbaden. https://doi.org/10.1007/978-3-658-09420-1_6