This article was downloaded by: [ABO Akademis Bibliotek] On: 21 May 2012, At: 23:45 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK Journal of Information Technology & Politics Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/witp20 Online Deliberation and Its Outcome Evidence from the Virtual Polity Experiment Kim Strandberg a & Kimmo Grönlund a a Social Science Research Institute, Department of Politics and Administration, Åbo Akademi University Available online: 08 Nov 2011 To cite this article: Kim Strandberg & Kimmo Grönlund (2012): Online Deliberation and Its Outcome Evidence from the Virtual Polity Experiment, Journal of Information Technology & Politics, 9:2, 167-184 To link to this article: http://dx.doi.org/10.1080/19331681.2011.637709 PLEASE SCROLL DOWN FOR ARTICLE Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-conditions This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.
Journal of Information Technology & Politics, 9:167 184, 2012 Copyright Taylor & Francis Group, LLC ISSN: 1933-1681 print/1933-169x online DOI: 10.1080/19331681.2011.637709 Online Deliberation and Its Outcome Evidence from the Virtual Polity Experiment Kim Strandberg Kimmo Grönlund ABSTRACT. In this article, findings are presented from a pilot experiment carried out online via a platform designed specifically for citizen deliberation. The purpose of the experiment was to assess how online deliberation works in practice and also to test several hypotheses concerning the effects of taking part in an online deliberation has on participants. Additionally, an examination is made to ascertain whether a variation in the deliberative procedure affects the outcome. The findings show that carrying out an online deliberation has its problems especially technical ones. Moreover, the effects of deliberation are somewhat modest. Knowledge gains and opinion changes were indeed traced, but regarding other potential effects, only a few significant changes were found. The article also comes up with some recommendations regarding the organization of online discussions. KEYWORDS. Citizens, deliberation, Finland, online politics, pilot experiment For decades, traditional forms of political participation have been in decline in mature representative democracies (Dalton, 2007; Inglehart & Welzel, 2005; Kippen & Jenkins, 2004). For instance, membership figures of political parties as well as electoral turnout have generally declined in mature democracies over the past decades (Dalton, 2000; Mair & van Biezen, 2001; Scarrow, 2000). Coinciding with these trends, scholars have highlighted the need for an inclusion of citizens in democratic decisionmaking (Dahl, 1989; Wright & Street, 2007). Basically, it is perceived that a more inclusive democratic system, where citizens are engaged in political discussion and decision-making, would increase the legitimacy of the democratic system (e.g., Barber, 1984; Dahl, 1989). Moreover, a deliberative turn has been evident in democratic theory, stressing the need for public discussion between free and equal Kim Strandberg holds a Ph.D. in political science and is senior researcher at the Social Science Research Institute at the Department of Politics and Administration at the Åbo Akademi University. His primary areas of research are political communication, citizen deliberation, and political uses of the Internet. Kimmo Grönlund holds a Ph.D. in political science and is the director of the Social Science Research Institute at Åbo Akademi University. His major research interests include political behavior in general and electoral behavior in particular, the role of social and institutional trust in democracy, as well as deliberative democracy and experiments in citizen deliberation. The article is one of several describing the Virtual Polity project, which is part of the work carried out by the Democracy: A Citizen Perspective centre of excellence at the Abo Akademi University. The data from our experiment will be made available in the JITP metaverse, and are also available on request from the authors. Address correspondence to: Kim Strandberg, Department of Social Sciences, Åbo Akademi University, Strandgatan 2, Vasa 65100, Finland (E-mail: kistrand@abo.fi). 167
168 JOURNAL OF INFORMATION TECHNOLOGY & POLITICS citizens. According to this view, the essence of democracy itself is now widely taken to be deliberation, as opposed to voting and interest aggregation (Dryzek, 2000, p. 1). Still, there have been concerns regarding the impracticality of carrying out deliberation in large scale societies (Coleman & Goetze, 2001; Wright & Street, 2007). Recently, these practical problems have been addressed to some extent with so-called deliberative mini-publics, including citizen juries, consensus conferences, and deliberative polls 1 (Dryzek & Goodin, 2006). Nonetheless, while these events go a long way in organizing and facilitating true deliberative discussions, they are nonetheless rather costand labor-intensive (Luskin, Fishkin, & Iyengar, 2006). For years, scholars have put their hopes in information and communication technology in engaging citizens in politics (e.g., Barber, 1984; Budge, 1996; Dahl, 1989), and with regard to tackling the impracticalities of carrying out deliberative discussions (e.g. Luskin et al., 2006; Price & Capella, 2002). Regarding the latter, it is conceived that large-scale citizen deliberations are more easily carried out online than off-line, since people can participate regardless of physical location (Coleman & Goetze, 2001; Strandberg, 2008; Wright & Street, 2007). Wright and Street (2007) assert that It is suggested that the technical characteristics of the Internet [... ] have led to the creation of a virtual world that establishes the conditions for deliberative democracy (p. 851). However, several scholars have raised concerns regarding the impact of the Internet for public deliberation. There have been concerns about polarization (Sunstein, 2001, 2006; Wilhelm, 1999, pp. 172 183) in online citizen discussions, as well as proof of such discussion often failing to meet qualitative standards of deliberative conversations (Jankowski & Van Os, 2004; Papacharissi, 2002; Strandberg, 2005, 2008). The present article examines both the feasibility of conducting online deliberation and the potential effects of online deliberation in a controlled experiment. It has been suggested that the Internet could indeed be a suitable venue for deliberative discussions if only the forums themselves were designed with such discussions in mind (Wright & Street, 2007; cf. Luskin et al., 2006). This study presents findings from a deliberative pilot experiment concerning nuclear energy and energy policy. The pilot was organized in Finland in the spring of 2008. To our knowledge, the experiment was the first full-scale online deliberative experiment using live video and audio conducted in Europe. 2 This Virtual Polity 3 was specifically built for the purpose of achieving the suitable conditions for citizen deliberation (cf. Hamlett & Cobb, 2006). First, the article seeks to examine how online deliberation works in practice. Second, the effect that online deliberation has on the participating citizens is examined in light of five hypotheses drawn from the literature. In the following section, a theoretical backdrop is outlined for the study. Thereafter, detailed information of the experiment at hand is presented. THEORETICAL BACKGROUND Citizen Deliberation Deliberative democracy acknowledges the importance of argumentation and fact-based discussion between free and equal citizens or their representatives (Elster, 1998). This is perceived as being more likely to bring about good policies than a mere aggregation of individual preferences (e.g., Gutmann & Thompson, 1996; Habermas, 1981, 1996). The process, wherein decisions are met through discussions in which views need to be justified and the arguments of others are to be taken into consideration, is perceived to change individual preferences and values towards a more collective understanding (Barber, 1984; Gutmann & Thompson, 1996). Typically, deliberative theorists emphasize the opinion-formation process more than the decision-making process. Several attributes have been associated with an ideal deliberative process. Inclusion, plurality of views, equality of discussion, reciprocity, reasoned justification, reflection, and sincerity are some of the central assumptions that should be followed in a deliberative discussion (cf., Kies, 2010). Deliberative discussions are normally designed so that the ideals of deliberation can be realized.
Strandberg and Grönlund 169 In practice, organizers create rules that facilitate the attributes such as equality and respect (Ryfe, 2005). The goal of reaching consensus through judging arguments by their merits is central, especially to Habermas (1996). It is believed that participants in a public deliberation arrive at a decision not by determining what preferences have the greatest numerical support, but by determining which proposals the collective agrees are supported by the best reasons (Young, 2000, p. 22). Theorists, on the other hand, are not overly naïve and do acknowledge the need for voting and majority decisions in political decision-making in the real world (Dryzek, 2000; Habermas, 1996). Grönlund et al. (2010) also assess that deliberation brings about other so-called sideeffects, besides changes of preferences and values. These effects are seen as an increase in political knowledge, political efficacy, interpersonal trust, and trust for institutions, as well as readiness for political and collective action. Studies reporting on deliberative polls (Fishkin, Luskin, & Jowell, 2000; Luskin, Fishkin, & Jowell, 2002) have established that participation in deliberation increases knowledge on the issue at hand. Several studies have shown that participation in deliberation may increase civic skills (Fung, 2003) as well as the feeling of internal political efficacy (Gastil, 2000; Smith & Wales, 2000). This may in turn increase the likelihood of participating in political or other collective action. It is also reasonable to expect that trust in political institutions and the closely related external political efficacy may receive a positive boost from participation in deliberations: Political trust may increase through deliberation as people learn to understand the processes of democratic decisionmaking, which can make them more trusting toward institutions and actors of representative democracy (Grönlund, Setälä, & Herne, 2010, p. 99). Participation in deliberation is also potentially beneficial for interpersonal trust, partly due to the need for taking other people into consideration when deliberating (Barber, 1984). Interpersonal trust is also boosted through the deliberative setting in which, ideally, people with different social backgrounds are forced to listen to each other, and deceitful behavior often leads to a loss of credibility and trustworthiness in the eyes of other deliberators (Dryzek & List, 2003). Obviously, deliberative theory is not without its critics. Some scholars have, for instance, questioned whether people actually want to take part in decision-making (Hibbing & Theiss- Morse, 2002). There are also doubts as to whether people are comfortable with situations in which disagreement frequently takes place (Mutz, 2006). Additionally, if deliberation takes place among like-minded people, group polarization can occur. This means that the group s average opinion moves towards a more extreme value as a result of deliberation. Sunstein (2009) presents evidence, according to which polarization is a human phenomenon that seems to occur regardless of profession or nationality. He demonstrates that it affects judges and lay citizens alike. If the group mean is liberal to start with, the group moves towards a more liberal position after deliberation. Consequently, a conservative group moves toward an even more conservative position in the end. The tension between people s reluctance to act within groups where disagreement occurs and the danger of polarization of like-minded groups remains to be solved within deliberative theory. Even though the real-life solutions of deliberative democracy still encounter challenges and opposing arguments, deliberation has been found to affect citizens in several ways. It has also been examined whether the impact of deliberation varies according to different decisionmaking procedures. In a recent comparison of two distinct procedures, an experimental setting with two treatments concerning the decisionmaking process in the deliberation session was created (Grönlund et al., 2010). One treatment used voting through secret ballot at the end of deliberation, whereas participants in the other treatment were to agree on a written common statement concerning the important aspects of the issue at hand. The main conclusion is that the decision-making rule does not have a clear overall impact on the outcome of citizen deliberation. The formulation of a common statement procedure was, however, more deliberative in some aspects, since it led to a more effective
170 JOURNAL OF INFORMATION TECHNOLOGY & POLITICS learning process and increased the participants belief in other people s preparedness for collective action. ICTs and Deliberation As we briefly mentioned in the introduction to his article, scholars have regarded the Internet as a suitable channel for realizing deliberation (e.g., Coleman & Goetze, 2001; Luskin et al., 2006; Price & Capella, 2002). In fact, even one of the strongest critics of deliberative theory, Sunstein (2006), places great hope in the Internet in correcting the potential pathologies of deliberations. Naturally, an obvious benefit of using the Internet for conveying and organizing deliberative discussions concerns practical and logistic advantages (Luskin et al., 2006). Additionally, partly due to such practical advantages, many scholars have regarded the possibilities of utilizing the Internet for bringing about broader changes to democracy through involving citizens (e.g., Barber, 1984; Budge, 1996; Dahl, 1989; Rheingold, 1993). Hauben and Hauben s (1997) statement summarizes the essence of the early debate:... the democratic ideas of some great political thinkers are becoming practical (p. 319). However, with time, concerns have been raised whether the Internet and other information and communication technologies (ICTs) per se actually guarantee that it is used in ways that may serve to improve democracy. After all, ICTs are not necessarily democratic as such but can be beneficial for democracy if used in appropriate ways. A debate concerning whether online politics will ever reach out beyond the already active citizens has been evident within the research field (e.g., Davis, 1999; Hill & Hughes, 1998; Norris, 2001). Scholars have pointed out that citizens need to have a strong interest in discussing politics in order to participate in online deliberation. As a consequence, online discussions are at risk of becoming isolated discussions lacking any real connection to policies and representativeness of the public at large (Wright & Street, 2007). Similar to off-line deliberation (e.g., Sunstein, 2006), other scholars have also demonstrated that online discussions often do not meet the qualitative ideals of deliberative discussions (e.g., Jankowski & Van Selm, 2000; Wilhelm, 1999). Concerns regarding the abundance of hasty opinions and severely conflict-laden discussions often found online have been raised (e.g., Papacharissi, 2002; Strandberg, 2005, 2008). Even so, it is our opinion that online deliberation is not to be regarded as being futile as such. If online citizens discussions are left to their own devices, there is a significantly higher risk of failing to achieve democratic discussions than if these discussions are carefully designed to be deliberative (Kies, 2010,; Wright & Street, 2007). This view is also supported by Hamlett and Cobb (2006), who state that... polarization and other deliberative pathologies might be held at bay by manipulating key facets of the deliberative environment (p. 632). A growing number of research and policy activities have by now been carried out within the field of participatory electronic democracy (e.g., Luskin et al., 2006; Muhlberger, 2005; Price & Capella, 2002; Shane, 2008). In general, these studies have combined face-to-face meetings with online elements, and some of them have also addressed deliberative democracy directly. As a common denominator, the studies have also organized online discussions and designed them stringently in order to avoid the abovementioned problems with impromptu online citizen discussions. Reporting on a year-long online citizens discussion project, Price and Capella (2002) found several interesting effects that were quite similar to those reported from off-line deliberative studies (cf., Fung, 2003; Fishkin et al., 2000; Gastil, 2000; Grönlund et al., 2010; Luskin et al., 2002; Smith & Wales, 2000). First, opinions changed as a result of taking part in the discussions. Second, the participants learning increased, and their understanding of the arguments of others improved significantly. Finally, even when controlling for initial interest and political activity, participation in the discussions fostered engagement in terms of the propensity to vote and participate in community activities. Luskin et al. (2006) report findings from two deliberative polls concerning U.S. foreign policy one of them carried out off-line and the other online. Although their findings show
Strandberg and Grönlund 171 more effects for off-line deliberation, some findings regarding the online deliberative poll merit attention here. Regarding learning, they demonstrate that the participants of the deliberative poll became significantly more informed compared to the control group on five of nine knowledge items. Additionally, they found several significant changes in attitudes and opinions regarding the subject of the deliberative poll. The authors conclude their reports by stating that online deliberation holds great promise and has roughly comparable effects to those found in the off-line deliberative poll. Muhlberger (2005) also reports some findings from the so-called Virtual Agora online deliberation project. His findings regarding opinion change differ somewhat from other studies, in as much as he states that mean opinion changes were due to information and not deliberation. This was tested using an information-only group in the experiment. Deliberation did however reduce the variation of opinions within the groups. In other words, the groups became more coherent. Also, the community-mindedness of the deliberators appeared to increase (Muhlberger, 2005, p. 11). Our brief summary of studies that have addressed online deliberation arguably shows that there is a potential for online deliberation, given the right conditions. Obviously, research on online deliberation is relatively new and growing and mostly based in the U.S. Still, the findings have mostly been in line with findings from off-line deliberations. In the next section we present the areas of enquiry of our Virtual Polity online deliberative experiment, after which the design and methods of our experiment are presented. HYPOTHESES Generally, since research on online deliberation is still rather scarce, one of our goals was to further examine how such endeavors work in practice. Above, we also discussed several potential effects both regarding knowledge gains and opinion changes as well as positive side effects of deliberation. Bringing these two aspects together, then, our ambition is to examine the feasibility of conducting online deliberations and the effects it may have on those who participate. Five hypotheses have been formulated. We anticipate that participation in online participation produces the following effects: H1 s opinions H2 Increases knowledge of the issue at hand H3 Increases internal and external political efficacy H4 Increases social and political trust H5 Increases propensity for political action Concerning all of the listed hypotheses, besides looking at the experiment population as a whole, we also look into potential differences in effects due to a variation in the deliberative procedure mentioned earlier; i.e., the use of a secret ballot vote or common statement regarding the issue of deliberation at the end of deliberation (Grönlund et al., 2010). Before looking into our findings, though, we present the design of the Virtual Polity experiment in the following section, in which some aspects of the feasibility of online deliberation are also touched upon. DESIGN AND METHODS The Virtual Polity experiment, which focused on nuclear energy and energy policy, 4 was carried out in the spring of 2008 as a replication of the first Finnish face-to-face deliberative experiment held in the autumn of 2006. The Virtual Polity was a pilot experiment seeking to test how online deliberation works and to assess its impact on the participating citizens. By using a strict experimental design and aiming at rigidly replicating a face-to-face experiment, including the use of live Webcam streaming and audio, the Virtual Polity pilot was the first online deliberative experiment of its kind in Europe and hence broadened the research beyond the U.S. context. Speaking in terms of general design, the experiment applied a pre-test/post-test control group experimental design. The experiment used random group allocation into two treatment groups common statement or secret ballot vote
172 JOURNAL OF INFORMATION TECHNOLOGY & POLITICS at the end of deliberation as well as a control group. As mentioned earlier, the examination of the effects of such a variation in the deliberative procedure had been carried out in an off-line deliberative experiment earlier (e.g., Grönlund et al., 2010). Albeit the findings regarding differences in effects between the variations in the deliberative procedure were modest in the Citizen Deliberation experiment, we still regard it important to further examine the potential effects of the two treatments in an online setting. The common statement groups ended their deliberation sessions by writing down a statement, based on the discussions in the group, containing the most important topics that all the participants could agree on being the most important regarding nuclear power. In the groups with secret ballot voting, no such statements were written. Instead, the deliberation session ended with a simple yes/no vote whether the participants were for or against the expansion of nuclear power in Finland. In the Virtual Polity pilot experiment, a random sample of 6,000 Finnish adult citizens was invited to participate. The sample was generated randomly by the Finnish population registry, which provided the citizens postal addresses. We thereafter sent invitations to participate in the experiment to all 6,000 people. The participants were promised free technical equipment and a voucher worth EUR 30 for their participation (see Iyengar, Luskin, & Fishkin, 2003 for a similar procedure). Participants who already owned the necessary equipment were promised a voucher worth EUR 50. Disappointingly, despite the fact that these rewards were promised for taking part, and even though a reminder to participate was also sent out, only as few as 147 citizens volunteered for the experiment. Probability-based recruitment to online panels has struggled with similar problems in the American context (e.g., Baker et al., 2010). Since our target number of groups was 12 with 12 persons in each, our census of volunteers was not adequate, which made perfect random allocation to treatments somewhat challenging. It was also clearly problematic that only 79 participants completed the actual deliberations. Some volunteers simply did not show up, and some suddenly dropped off during the experiment. There were eventually only 40 participants in the six groups ending their sessions with a secret ballot vote and 39 participants in the six common statement groups. Clearly, we had overestimated people s interest in taking part in a Web-based deliberation experiment. We examined whether there were any systematic differences between the vote and common statement treatments concerning the dropouts. None of the aspects we looked at i.e., the distributions of gender, age, education, interest in energy politics, and political affiliation displayed any statistically significant differences between participants dropping off in each of the two treatments prior to deliberation (see Appendix A for detailed tables). Besides the experiment population, an additional 96 individuals were allocated to a control group. On the whole, considering the severe failure of the recruitment process for the experiment, the mere 147 participants included in the experiment were still acceptably representative of the general Finnish population (see Appendix B for detailed tables). Still, it should be noted that the experiment s population had a clear overrepresentation of males, and a slight overrepresentation of citizens belonging to younger age groups and having a high level of political interest. An additional brief comparison (not shown in detail here due to reasons of length) of participants dropping off and those completing the experiment shows that the participating citizens were higher educated than those dropping off. This may call for caution in drawing broader generalizations from our findings, since our de facto sample of citizens is slightly skewed towards higher educated citizens potentially more accustomed to using computers and technical equipment. We used surveys in order to trace how information and deliberation affected the participants. Initially, a baseline survey was answered electronically by both the volunteers and the control group. Thereafter, there were several surveys answered electronically at different stages of the experiment in order to assess both the effects of deliberation and also to measure knowledge gains. After deliberation, a replication of the baseline survey was answered by the participants. The same survey was also followed
Strandberg and Grönlund 173 TABLE 1. The Phases of the Virtual Polity Experiment March 2008 April 2008 June 2008 Baseline survey for treatment groups and control group (T1) Survey measuring knowledge (T2) Reading information material about nuclear power Watching recordings of expert panel discussions A short poll of 7 questions (T3) Discussions in small groups 2 4 hours Decision making in the small groups 1 hour Final survey including measurement of knowledge (T4) Follow up survey for treatment groups and control group (T5) up approximately two months after the experiment in order to see the long-term effects of deliberation. At this point, the control group also answered the survey. The different surveys of the Virtual Polity experiment are summarized in Table 1. The surveys abbreviated as T1 and T5 were thus answered by both treatment groups and the control group, while surveys T2, T3, and T4 were only answered by citizens taking part in the deliberation. The surveys focused on participants opinions concerning energy issues, their feelings of internal and external efficacy, their political knowledge, and levels of interpersonal and general trust, as well as their propensity for political activity. The actual deliberation events were carried out on weekends in April and May 2008 via the Virtual Polity Web site. Each group had one session in total. Prior to the session, participants read neutral information material on nuclear power and watched videos of a panel discussion between experts representing a broad range of opinions on nuclear power and its expansion. 5 Contrary to most online deliberations thus far (e.g., Luskin et al., 2006; Price & Capella, 2002), the actual discussions were carried out with the use of videoconferencing technology; both live audio and Webcam streaming were thus used in the discussions. All participants took part from their home computers through logging in via the Virtual Polity Website (cf., Grönlund et al., 2009; Shane, 2008). Trained facilitators, who were assigned with the task of ensuring that all participants expressed their views and that the flow of discussion was good, were used in all discussion groups. In essence, the Virtual Polity experiment fulfilled many of the strategic design measures deemed beneficial for deliberation events by scholars (Hamlett & Cobb, 2006; Sunstein, 2003). Turning to the feasibility of carrying out online deliberations, nonetheless, we have to acknowledge that our experiment did not get off to a good start. The actual deliberation sessions included 12 groups, consisting of only four to 11 participants. The number of participants was not intended to vary, but partly due to participants not showing up at all, and mostly due to rather severe problems with our host servers, several groups ended up rescheduling with some participants not showing up the second time. Many participants also had problems with their own Internet connections, and several of them did not set up their audio and video equipment properly and ended up leaving the deliberation session. Nonetheless, the participants were divided into six groups, ending the session with a secret ballot vote and six groups ending with a common statement. The flow of discussion was set so that only one participant could talk at any given time. A text-based chat option was also available for use, in case the participants experienced problems with audio or video streaming. This text-chat was also used by the facilitator for posting technical information not directly related to the actual discussion. The discussion room also contained a whiteboard window, in which the facilitator listed the topics suggested by the participants in order to create an agenda for discussion. It was also used for the writing of the common statements. The online discussions lasted for a couple of hours, varying mainly according to group size. The facilitator initially
174 JOURNAL OF INFORMATION TECHNOLOGY & POLITICS held a short introduction and helped the participants to get the video and audio streaming working. Otherwise, as mentioned, the facilitator had mainly an observational role in the discussions. The participants filled out a postdeliberation survey (T4) immediately after the sessions had ended. FINDINGS The findings are presented in five sections one for each of our hypotheses. We return to the issue of how online deliberations work in practice in the conclusions. First, we present findings regarding the development of opinions, followed by changes regarding knowledge, political efficacy, trust, and finally concerning the participants propensity for political activity. For each of these aspects, in addition to displaying the development for the entire experimental group, we also compare the two experimental treatments (vote and statement) in tables. Each table also includes a column displaying whether the pre-test to post-test change values for the control group were significant or not for the corresponding items, in order to ensure that observed experimental effects can be established. 6 Opinions on Energy Policy Our questionnaire contained 19 items measuring the participants opinions on energy policy. Hypothesis H1 predicted that the participants opinions should change due to taking part in the online deliberation; Table 2 shows the development of opinions for the entire experiment population, and also specifically for the two treatment groups (vote and statement) compared to the development for the control group. Looking at all participants as a whole, Table 2 shows that there were six significant changes of opinions found upon examination. In that sense, H1 is supported to some extent, albeit we note that there were no changes considering the majority of items measuring opinions. The participants became more critical toward the use of nuclear power, peat, and coal in the deliberations. On the other hand, their support for an increased use of wind power increased, as did their belief in other people s propensity to save electricity, if needed. There were no significant changes in opinions for those participants who ended their sessions with a secret ballot vote. In the statement treatment, on the other hand, there were six significant changes in the participants opinions. Evidently, the variation in the deliberative procedure had some impact on opinion change (cf., Grönlund et al., 2010). This finding is also supported by the five noted significant differences between the two treatment groups as shown in the table (independent samples t-test). None of the aforementioned effects occurred in the control group, indicating that there were indeed opinion shifts occurring as a result of taking part in the deliberation. The post-hoc analysis of variance (ANOVA) test also supports this finding. As a group aggregate, we note that the shift in opinions, according to our findings, is very similar to those which Barabas (2004) found in off-line deliberations. Our analysis shown here does not, in contrast to Barabas study, contain detailed information regarding which type of citizens shifted opinions the most. However, a brief post-hoc analysis (Pearson s correlation not shown here) indicates that people strongly disagreeing with a statement before deliberation did not alter their initial opinion in the experiment. Participants with neutral opinions, and to some extent, agreeing somewhat with a statement appear to have altered their initial opinion comparably more. Detailed analyses are, naturally, called for in future studies of online deliberations, since the data in this experiment are inadequate for the rigid testing of how initial opinion strength affects the outcome of deliberation. Gains in Political Knowledge Increased knowledge of the issue at hand is an often reported effect of deliberation (e.g., Fishkin et al., 2000; Luskin et al., 2002). This notion appears reasonable, since by taking part in a deliberative process, participants thoroughly learn about, discuss, and debate the issue of the deliberation. Hence, we also formulated a hypothesis H2 predicting the same outcome of taking part in the Virtual Polity experiment.
Strandberg and Grönlund 175 TABLE 2. Development of Opinions on Energy Policy in the Two Treatments and the Control Group All participants (N = 79) Vote treatment (N = 40) Statement treatment (N = 39) Control group (N = 96) Finland should build a sixth nuclear plant. The use of coal should increase in Finnish energy production. The use of peat should increase in Finnish energy production. The use of wind energy should increase in Finnish energy production. The encouragement of saving on electricity should be central in Finnish energy policy. There is serious harm associated with the mining of uranium. There are considerable risks with storing nuclear waste. Finns would save electricity themselves if there was a power shortage and citizens would be encouraged to do so. I myself would save electricity in the case of a shortage. 2.71 2.51 0.20 2.70 2.67 0.03 2.72 2.33 0.39 n.s. 1.56 1.33 0.23 2.57 2.35 0.22 1.55 1.22 0.23 n.s. 2.60 2.31 0.29 2.68 2.39 0.19 2.62 2.21 0.41 n.s. 3.44 3.59 0.15 3.47 3.62 0.15 3.52 3.52 0.00 3.41 3.72 0.31 n.s. 2.80 2.96 0.16 2.82 2.74 0.08 2.78 3.17 a 0.39 n.s. 3.01 2.97 0.04 3.10 2.87 0.23 2.92 3.08 a 0.16 n.s. 3.14 3.32 0.18 3.12 3.22 0.10 3.15 3.41 0.26 n.s. 3.59 3.65 0.06 3.67 3.57 0.10 3.51 3.72 0.21 n.s. p <.05, p <.01. Note: Items marked with a show a significant development within the treatment group (paired samples t-test). Bolded items differ significantly between the vote and statement groups (independent samples t-test). Items marked with a differ significantly between groups in the post-hoc ANOVA test of between groups mean differences. The table only shows the items for which there were significant changes in either group. Figures are group averages on a scale 1 5, where 1 stands for disagreeing totally with the statement and 5 for agreeing totally. However, some have noted that the information material used as a part of the deliberative design might have a more profound effect on knowledge gain than the actual discussions (e.g., Muhlberger, 2005). In order to account for this possibility, we construct our analysis of knowledge gains so that those items whose answers could be found in the pre-discussion information material are separated from other knowledge items. The effects of the deliberation itself are examined through separating knowledge concerning energy policy items from those concerning politics in general (Table 3). 7 Judging from the findings presented in Table 3, it appears that taking part in the deliberative experiment increased the participants knowledge but only concerning the items found in the information material that the participants read prior to the small-n group discussions. Hypothesis H2 is thus not readily supported. Further, only the participants in the common statement group increased their knowledge concerning these items. A tentative explanation for this, bearing in mind that there were no differences in the demographic composition of participants between the two treatments, could
176 JOURNAL OF INFORMATION TECHNOLOGY & POLITICS TABLE 3. Knowledge Gains for the Entire Experiment and the Two Treatments All participants (N = 79) Vote treatment (N = 40) Pre-test Statement treatment (N = 39) Items found in the information 2.96 3.35 0.39 3.02 3.27 0.25 2.90 3.44 0.54 material (0 6) Other energy policy items (0 4) 2.22 2.26 0.04 2.35 2.55 0.20 2.10 1.97 0.13 Genera political knowledge items (0 5) 3.59 3.48 0.11 3.56 3.43 0.13 3.61 3.54 0.07 p <.05 Note: Items marked with a show a significant development within the treatment group (paired samples t-test). Bolded items differ significantly between the vote and statement groups (independent samples t-test). be that the discussion process of writing the common statements proved to be more deliberative and information-rich than in the secret ballot groups. There were no significant knowledge changes for either group concerning the other items on energy policy. Political Efficacy As we noted earlier, increased political efficacy is one of the perceived side-effects of deliberation. It is proposed that deliberators may increase their civic skills and feeling of internal political efficacy after deliberation (Gastil, 2000; Smith & Wales, 2000). Our third hypothesis, H3, also makes the prediction that taking part in an online deliberation will yield the same outcome. This is of course a rather logical expectation, since deliberation is a form of democratic practice through which participants become familiar with political reasoning and decision-making. Of course, the counterargument would be that deliberation might decrease feelings of political efficacy, since the complexity of politics becomes more evident to the deliberating citizens. In this section, we look at the impact of deliberation on both internal and external political efficacy; Table 4 shows the development for all participants and the two treatment groups as well as for the control group. Evidently, deliberation did not affect the participants feelings of political efficacy in the total experiment population or in either treatment group. Hypothesis H3 is not supported. Only one significant change toward a feeling of more external efficacy was found for the vote group concerning the functioning of Finnish democracy but none for the statement group or the entire experimental population. There were also no significant differences between the changes in the vote and statement group to be found upon direct comparison (independent samples t-test). In looking at the pre-test to post-test changes in the control group, only one significant change regarding internal efficacy took place. This was regarding an item where no experimental effects were found. Interpersonal Trust and Trust for Institutions Interpersonal trust can be perceived to receive a boost from taking part in deliberative discussions where people of different social and political backgrounds engage in respectful, other-regarding conversations (Dryzek & List, 2003). It is, however, rather uncertain how the virtual environment lacking real life faceto-face interaction affects interpersonal trust (see Barabas, 2004). Nonetheless, we predicted in hypothesis H4 that taking part in the online deliberation would bring about boosts in the participants interpersonal trust and trust for institutions. Table 5 presents detailed findings regarding interpersonal trust and trust for institutions for the participants and the two treatments as well as for the control group. There was only one significant effect regarding interpersonal trust; after deliberation the participants increasingly felt
Strandberg and Grönlund 177 TABLE 4. Political Efficacy in the Two Treatments and the Control Group: Before and After Deliberation All participants (N = 79) Vote treatment (N = 40) Statement treatment (N = 39) Control group (N = 96) Internal efficacy Sometimes politics seems so complicated that an ordinary citizen cannot really understand what is going on. I know more about politics and government than most of my fellow citizens. External efficacy People can exert influence through voting. An ordinary citizen cannot influence politics. On the whole, democracy works well in Finland. 2.99 3.06 0.07 3.02 2.97 0.05 2.95 3.16 0.21 n.s. 2.56 2.65 0.09 2.53 2.62 0.09 2.60 2.67 0.07 0.25 3.42 3.46 0.04 3.40 3.32 0.08 3.44 4.59 0.15 n.s. 1.95 2.11 0.16 1.95 2.15 0.20 1.95 2.08 0.13 n.s. 2.99 3.13 0.14 2.95 3.17 0.22 3.03 3.08 0.05 n.s. p <.05. Note: Items marked with a show a significant development within the treatment group (paired samples t-test). Bolded items differ significantly between the vote and statement groups (independent samples t-test). that people are generally kind and good. Upon detailed examination, this effect was found for the statement treatment but not for the vote treatment. No other significant changes were found in either treatment. It should also be noted that the direct comparison of the treatments revealed no significant differences (independent samples t-test). Generally speaking, hence, the impact of our online deliberation on interpersonal trust and trust for institutions was small. Hypothesis H4 is only supported to a limited extent. The control group showed no significant change on the trust items. Propensity for Political Activity We end the presentation of the findings by examining what kind of impact online deliberation had on the participants readiness for political and collective action. In our review of deliberative theory, we presented an argument that deliberation may increase readiness for political action, as it often increases the participants civic skills, knowledge, and interpersonal trust, as well as feeling of internal political efficacy (Grönlund et al., 2010). Hypothesis H5 also predicted that online deliberation would result in similar effects. As shown earlier, however, there were only modest effects on internal efficacy, trust, and political knowledge found for the present experiment. We will examine whether taking part in online deliberation nonetheless had any effect on the participants propensity for political and collective action. We now look further at the development for the entire experiment and the two treatment groups in Table 6. Generally, the effects of deliberation on the participants readiness for political action were minor. The support for hypothesis H5 was hence very small. There were five weak but significant developments for all participants, of which only three remain as experimental effects when controlling for the control group s development. Surprisingly, contrary to theoretical expectations, the participants had actually become slightly less prone to engage in political action. There was also one significant effect for each of the treatment groups, but both of these were also found for the control group and are therefore regarded as more-or-less spurious. The two treatments did not differ significantly from each other.
178 JOURNAL OF INFORMATION TECHNOLOGY & POLITICS TABLE 5. Development of Interpersonal Trust and Trust for Institutions in the Two Treatment Groups and the Control Group All participants (N = 79) Vote treatment (N = 40) Statement treatment (N = 39) Control group (N = 96) Social trust and others regarding attitudes Most people are basically good 3.13 3.27 0.14 3.17 3.22 0.05 3.08 3.31 0.23 n.s. and kind. It is safe to assume that all 2.08 2.04 0.02 2.07 2.10 0.03 2.08 1.97 0.11 n.s. people have a vicious streak and it will come out when they are given a chance. Honesty is the best policy in all 3.47 3.46 0.01 3.37 3.37 0.00 3.56 3.54 0.02 n.s. cases. There is no excuse for lying to 2.96 2.99 0.03 2.82 2.92 0.10 3.10 3.05 0.05 n.s. someone else. Anyone who completely trusts 2.59 2.45 0.04 2.55 2.52 0.03 2.63 2.58 0.05 n.s. anyone else is asking for trouble. Generally speaking. people 1.95 2.04 0.09 1.90 2.00 0.10 2.00 2.08 0.08 n.s. don t work hard unless they reforcedtodoso. Generally speaking. most 3.27 3.26 0.01 3.20 3.15 0.05 3.34 3.37 0.03 n.s. people can be trusted. Trust for institutions The parliament 2.77 2.79 0.02 2.72 2.74 0.02 2.82 2.85 0.03 n.s. The judicial system 3.08 3.13 0.05 2.90 3.02 0.12 3.26 3.24 0.02 n.s. The police 3.27 3.22 0.05 3.17 3.12 0.05 3.36 3.31 0.05 n.s. Politicians 2.15 2.16 0.01 2.15 2.08 0.07 2.14 2.26 0.12 n.s. p <.01. Note: Items marked with a show a significant development within the treatment group (paired samples t-test). Bolded items differ significantly between the vote and statement groups (independent samples t-test). Figures are group averages on a scale 1 5 where 1 stands for disagreeing totally with the statement and 5 for agreeing totally. CONCLUSIONS The present article has looked at the feasibility of carrying out deliberation online as well as examined how participants are affected by taking part in an online deliberative experiment. Our empirical focus was drawn from the literature, and included both the common effects found in earlier studies as well as positive side-effects. Based on this, we formulated five hypotheses. Additionally, we examined potential variations in the effects of deliberation depending on whether the deliberation session ended in a common statement or with a secret ballot. Additionally, our goal was to use the pilot Virtual Polity experiment as a means for assessing how deliberation works online. The main findings of our study are summarized, and their interpretations discussed in this concluding section. Concerning the feasibility of online deliberation, we note that our experiment, which in a sense failed to serve as a controlled experiment, has highlighted several challenges most of which were technical in nature. Arguably, though, the limited interest shown by Finnish citizens to take part in our experiment should be stressed. Whether this is a reflection of digital divides (e.g., Norris, 2001), technophobia, or lack of interest in the issue remains unclear. Nonetheless, this poses a clear challenge for future online endeavors and has also been highlighted in a recent research synthesis (Baker et al., 2010). The most severe technical problem
Strandberg and Grönlund 179 TABLE 6. Propensity for Political Action Before and After Deliberation in the Two Treatment Groups and the Control Group All participants (N = 79) Vote treatment (N = 40) Statement treatment (N = 39) Control group (N = 96) Vote in elections 1.00 1.00 0.00 1.00 1.00 0.00 1.00 1.00 0.00 n.s. Write to a newspapers public.89.78 0.11.90.80 0.10.87.77 0.10 0.14 section Contact politicians regarding.96.89 0.07.92.87 0.05 1.00.90 0.10 0.16 an important matter Sign a petition.91.91 0.00.90.87 0.03.92.95 0.03 n.s. Take part in a boycott or.77.66 0.09.75.60 0.15.79.72 0.07 n.s. similar consumers strike Take part in a peaceful.71.58 0.13.65.50 0.15.77.67 0.10 n.s. demonstration Demonstrate civil.22.18 0.04.25.20 0.05.18.15 0.03 n.s. disobedience by taking part in non-violent illegal activity Take part in a demonstration.14.08 0.06.12.15 0.07.15.10 0.05 n.s. where violence has occurred previously Use force in order to achieve one s goals.05.03 0.02.05.02 0.03.05.03 0.02 n.s. p <.05, p <.001. Note: Items marked with a show a significant development within the treatment group (paired samples t-test). Bolded items differ significantly between the vote and statement groups (independent samples t-test). Figures are group averages on a scale 0 1 where 0 stands for I would not take part in the activity and 1 stands for I might take part in the activity. had to do with our own server. Some of our deliberation groups experienced server crashes, even though the server should have been able to handle the number of users in question. After a server restart, many participants just quit and remained unreachable. Naturally, a face-to-face setting is safer from such unforeseen events, making online deliberations more risky to carry out. A second problematic circumstance challenge was presented by the problems our participants had setting up their Webcams and headsets. However, it can be assumed that as time goes by, people will be more accustomed to such technical equipment, and this issue should not be apparent if further online deliberations are also conducted using live images and sound. Finally, we noted that our online deliberation was somewhat hindered by the limited bandwidth in some participants connections. Again, though, this ought to be less of a problem when broadband speeds develop over time. Barabas (2004) stresses the need for scholars to examine whether, and under which conditions, the public benefits from alternatives to face-to-face deliberation. Consequently, an important conclusion of this study is that technical issues such as those experienced in the present experiment could undermine citizens trust and future willingness to take part in online deliberations regardless of what benefits taking part might have. So the conditions established by a virtual setting, albeit having many theoretical benefits for carrying out public deliberations, could backfire due to practical issues. What about the effects of online deliberation on the citizens actually taking part? The findings from the Virtual Polity online deliberative experiment echo earlier results. The hypothesis H1 regarding changes in the participants opinions about energy policy was the only hypothesis receiving support in our analysis (cf., Barabas, 2004). We also found that there were several differences in effects on opinions due to the variation in deliberative procedure common statement or secret ballot vote tested in our experiment. In all of those instances, the