A matter of interpretation? Experimental account of effects of. persuasive words on voters evaluation of election pledges.

Transcription

1 A matter of interpretation? Experimental account of effects of persuasive words on voters evaluation of election pledges. Elina Lindgren, Department of Political science, University of Gothenburg Abstract Election campaigns are permeated with persuasive value-laden words, and it is well known that this can help parties muster support for policy pledges. Prior studies show that persuasive value-laden words trigger emotions, and that, if the words match voters ideological preferences, they are more likely to perceive candidates as representatives of their interest. Despite the large number of studies on this topic, research is not yet conclusive about what exact mechanisms that are at play when citizens attitudes change. This paper contributes to the understanding of mechanisms by accounting for substantial and ideological (left-right) interpretation of policies that are described by persuasive words. An online survey experiment on 2000 Swedish citizens reveal that persuasive words influence peoples substantial interpretation of proposed policies, and that persuasive words, interpretation of policies in left-right terms and individuals left-right orientation interact in influencing attitudes to election pledges. Paper to be presented at the General Conference of the European Consortium for Political Research, Montreal, August Section: Political Communication at Cross-Roads. Panel: The More the Better? Information Abundance and its Consequences. Saturday, August 29,

2 Uttering a word is like striking a note on the keyboard of the imagination. (Ludwig Wittgenstein in Philosophical investigations, 1953). Emotive and value-laden words have been described as powerful tools for persuaders (Nelson and Garst, 2005: 490). Such words draw attention (Marcus et al, 2000; Lodge and Taber, 2005), alter political attitudes (De Steno et al, 2004) and increase support for candidates and polices (Mutz et al, 1996; Lodge et al, 1995), and therefore, political parties aiming for government often describe policy pledges with words that define them in terms of ethical and social values (Feldman & Steenbergen, 2001; Feldman and Zaller, 1992; Clifford & Jerit, 2013; Ball-Rokeach & Loges, 2013). But what is it that makes these kinds of words so powerful in shaping attitudes? It has previously been concluded that citizens use their core values and emotions to decide where they stand on political issues (McClosky & Zaller, 1984; Rokeach, 1973; Fleming & Petty, 2000; Popkin, 1991). Values are people s normative assessments of life (Kuklinkski, 2001: 364), and shared values therefore play an important role for evaluation of political objects (Nelson & Garst, 2005; Schemer et al, 2012). We shield ourselves from information that challenges our beliefs (Domke et al, 1998; Taber & Lodge, 2006; Zaller, 1992; Huang and Price, 2001; Lord et al, 1979) and emphasize information that reinforces our prior attitudes (Clifford, Jerit et al, 2015; Shen & Edwards, 2005; Druckman, 2001; Barker, 2005), and policies and candidates gain support when perceived as advocates of peoples favored values (Nelson & Garst, 2005; Schemer et al, 2012; Keum et al, 2005). Persuasive value-laden words seem to serve as informational cues to evaluate whether this is the case (Barker, 2005; Entman, 1993). Despite a large number of studies on this topic research is as yet not conclusive about what exact mechanisms that are at play when citizens are cued, and it is neither clear whether 2

3 or to what extent persuasive words influence interpretation of the factual content of policies. By applying theories from literature on philosophy of language and linguistic semantics to an election pledge context, this paper gain a theoretical framework for studying the more substantial effects of persuasive words on evaluation of election pledges. More specifically, I acknowledge that certain words open up for individual and positive interpretation of policy pledges, and explores what effects substantial and ideological interpretation of policies in turn have on peoples attitudes to the same. Further, while prior studies have emphasized persuasive words with ideological connotations, this paper makes a difference between ideological (henceforth Left- and rightladen) persuasive words, and words that denote universal and ideology-neutral values (henceforth Universal persuasive words). We know that left- and right-laden words have positive influence on attitudes among individuals with similar ideological orientation (Clifford, Jerit et al, 2015; Schemer et al, 2012; Barker, 2005), while small or even negative effects on people of different predispositions (Domke et al, 1998; Nelson & Garst, 2005; Taber & Lodge, 2006). But what happens if parties use universal persuasive words that are neutral to ideology? Can this also impact voters attitudes to election pledges? While studying the effects of left- and right-laden words, this paper also includes universal persuasive words in the analyses. I hypothesize that, while left- and right-laden words attract people with similar ideological preferences but distance those of different predispositions, universal persuasive words can be used to attract people from both sides of the political left-right divide. Theoretical argument is that universal persuasive words activate interpretation of policies that fit individual ideological preferences; left voters will ascribe proposed policies left-laden values, while right voters will ascribe them right-laden qualities. An online survey experiment on 2000 Swedish adults reveal that different persuasive words, when used to describe a policy pledge concerning higher demands in junior 3

4 high school, generate different substantial interpretations of the proposed policies both when Ss describe policies in own words and when they describe them using pre-defined categories. The study also reveals significant interaction between effects of persuasive words, interpretation of policies in left-right terms and individual left-right orientation, on attitudes to the policy pledge. Results are discussed in concluding chapters. BACKGROUND So what is it that makes persuasive words so powerful in shaping attitudes? In short, what seems to be critical when voters evaluate political objects is whether there is a perceived fit between the object, the advocates and the voters own value preferences. Persuasive words provides easy accessible informational cues both for identifying whether an advocate have same value preferences (Barker, 2005; Domke et al, 1998; Entman, 1993), and whether a political object should be considered good or bad (e.g Clifford & Jerit, 2013; Kidwell et al, 2013). In a world were we are constantly surrounded by multiple political messages from various stakeholders who wants to either reinforce or challenge our prior beliefs (Gamson, 1992), even highly politically informed voters need such informational shortcuts to navigate through and evaluate the political information we all receive (Price & Tewksbury, 1997; Schwarz et al, 2003). Since persuasive words stands out in messages they become an easy way to trigger emotions and value orientation (Kinder & Sanders 1996; Brewer, 2003, 2008), and hence also the wished for reactions to appeals (Ball-Rokeach & Loges, 1996). So indubitably there seem to be a relationship between voters value preferences, persuasive words and willingness to support a policy. But how do people actually process information formulated using persuasive words? A possibility is that people simply accept the value cues they are given through the words. Several studies have depicted citizens as limited capacity information processors who submissively rely on elite cues for political decision- 4

5 making (Nelson & Kinder, 1996; Entman, 1993; Iyengar, 1991; Zaller, 1992). Hence, the relationship may reflect that people uses persuasive words as mere leads to determine more general predispositions of advocates and whether or not one should support a political object or not. But it is also possible that words have more profound effects, and that peoples substantial and ideological interpretations of the content of political objects per se, are influenced. Prior studies have shown that media framing of issues can shape mass opinion and peoples thoughts about issues when the frames define what the issues are really about. For example, scholars have found that media framing of political issues influence peoples interpretation of the issues in that they express thoughts consistent with the frames (Shen & Edwards, 2005; Tewksbury et al, 2000; Wettstein, 2012; Shen, 2004), that ethical framing of political issues in news articles generates more ethical related thoughts when people discuss the issues (Brewer & Gross, 2005), and that ethical framing in news articles in conjunction with ethical related thoughts affect peoples decision-making regarding the issues at question (Domke & Shah, 1995; Shah et al, 1996; Domke et al, 1998). Is this the case also when voters evaluate political information that is presented by political parties in election campaigns? And can persuasive words change peoples substantial interpretation of the factual content of proposed policies? In order to test this, we need more theoretical accounts of mechanisms involved when people evaluate policy pledges that are described by persuasive words. THEORETICAL FRAMEWORK AND HYPOTHESES The theoretical account is drawn from literature on philosophy of language and linguistic semantics, and in particular from theories of persuasive words, the concept used to describe words that help persuaders convince people of a cause, object or issue (e.g. Larson, 2012; Stevenson, 1944; Walton; 2001). The literature defines in great detail the characteristics of 5

6 words that make them persuasive, and the theories are therefore well suited as framework for empirical investigation of mechanisms involved in the process when words change attitudes. Following section define characteristics of persuasive words described in the literature, and present arguments for why these should open up for free and positive interpretation of associated objects. It proceeds with a discussion of why effects should differ depending on individuals left-right orientation, and ideological connotations of the persuasive words used to describe an object. Following characteristics define the notion of persuasive words; 1) Ethical and value-laden connotations, 2) Strong emotive meanings, and 3) Vague and ambiguous descriptive meaning. 1) Ethical and value-laden connotations. That persuasive words are ethical and valueladen means that they identify norms of individuals, groups or cultures (Bench-Capon, 2003; McGee, 1980). They appeal to peoples value preferences, activate ethical reasoning, and identify what should be considered morally good or bad (Stevenson, 1944; Hare, 1952; Schacter et al, 2010). If for example in reporting on a military conflict, our side is described as freedom fighters and the other side as terrorists, a domestic or international war, in general viewed as morally bad, can become morally acceptable and even considered a necessary good (Walton, 2006: 220). 2) Strong emotive meaning. A second characteristic of persuasive words is that they trigger emotions. War and terrorism for example creates fear, injustice creates anger, love creates happiness, and peace and security generate a sense of calmness and safety (see e.g. Walton & Macogny, 2014; Hallden, 1965). For persuasive words, the emotive meaning is stronger and more persistent than the factual one, and therefore people like or dislike them independent of how they are exactly defined (Walton, 2001; Stevenson, 1944; Junichi, 1985). If in addition, persuasive words are preceded with enhancing words such as true, great and real (Stevenson, 1944:213; Walton, 2001:119), and are repeated in same message 6

7 (Stevenson, 1944: 241), they become augmented which increase the emotive reactions to appeals. 3) Vague and ambiguous descriptive meaning. With descriptive meaning means the factual content of a word. In general, persuasive words do not have a fixed, but vague and ambiguous descriptive meaning. They are not specific to a situation, can be applied to many issues (Stevenson, 1944; Walton, 2001), and lack a clear-cut reference such as the will of the people (Orwell, 1946; Rigotti, 2005; Walton & Macogno, 2014). As such, they can be used referring to (often quite considerably) different things (Schiappa, 2003). With above characteristics in mind, I now move on explaining why persuasive words can be used to influence and persuade people of objects they are associated with. Even if persuasive words trigger emotions and value considerations, it is not obvious that they should impact objects they are associated with. The main argument made in literature on philosophy of language and linguistic semantics for why this should be the case is that persuasive words put spin on arguments, and make it look as if statements of fact are being made (Walton, 2005; Zarefsky, 1998; Pawlowski & Tadeusz, 1980). If conclusions about an object are drawn based on persuasive words that looks like simple fact, people will perceive the words as inevitably linked to the object. And if the object is perceived as linked to the persuasive word, it will also likely receive the same response as the word (Stevenson, 1944; Walton, 2001, 2005; Johnson and Blair, 1983). The first characteristic involved in the process when words change attitudes to an object is the vague and ambiguous rules on how to define the precise meaning of persuasive words. If objects are inevitably linked with vaguely defined persuasive words, also the precise meanings of the objects become blurry (Stromer-Galley & Schiappa 1998; Schiappa, 2003). People will therefore use their subjective understanding of the persuasive words used to describe the object when interpreting what this is really about (Macagno & Walton, 2014; 7

8 Walton, 2005). Applied to an election pledge context, this means that individuals exposed to policies described by persuasive words will also ascribe factual qualities to the policies they consider matching the persuasive word. In the present study this will be seen in that Subjects make substantial interpretation of policies that matches connotations of the persuasive words used to describe them; H1 Persuasive words generate substantial interpretation of policy pledges that matches the connotations of the persuasive words. Moving on to why effects of different persuasive words should differ depending on individuals left-right orientation. As described earlier, arguments from values is grounded on shared premises classifying what should be considered desirable or objectionable. Not all values are universally shared or appreciated; instead values often differ depending on individual characteristics and contexts. An important difference when it comes to values and perceptions of society is ideology, thus a difference is made between persuasive words that are ideological (left- or right-laden), and persuasive words that are universal and ideologyneutral (see Shen, 2004; Clifford et al, 2015; Stevenson, 1944: 11-19; Stevenson, 1963: 1-10; Walton, 2005: ; Turiel, 1996: ). In this paper, ideological left- and right-laden, and universal persuasive words are expected to differ in effects, both on individuals interpretation of policy pledges in ideological (left-right) terms, and whether or not they make an object appealing to an individual. First, objects that are described by ideological left- (right)-laden persuasive words will be perceived as having the same left- (right) qualities, and the perception will be same for individuals of different left-right predispositions. If an object is associated with equality for example, it will be interpreted as having left qualities both by individuals of left- and right predispositions: H2 Left- (right)-laden persuasive words generate same interpretation of policies in left (right) terms irrespective of individuals left-right orientation. In contrast, objects that are associated with universal persuasive words will be perceived as being in 8

9 accordance with individual predispositions. Objects associated with democracy for example, a universally cherished value in western societies (McFaul, 2004), will be perceived as having left qualities by people of left predispositions, whereas it will be perceived as right-laden among people oriented to the political right. Therefore: H3 Universal persuasive words generate interpretation of policies in accordance with individuals left-right orientation; individuals of left predispositions will interpret policies as left-laden whereas individuals of right predispositions will interpret them as right-laden. Finally, if a person appreciates a persuasive word she will more likely perceive an object described by the word both as congruent with her ideological preferences, and as positive and morally good. Atkinson, Bench-Capon and McBurney (2006: 166) describe the logic when conclusions are being drawn based on persuasive words as follows: In Current circumstances C, we should perform Action A, which will realize Goal G, that will promote Value V (=persuasive word). This can be made more concrete: Currently, parental leave is used by mothers (C). There should be a daddy quota (A), so that dads start participate in parental leave (G), which will improve gender equality (V) (=persuasive word). Since gender equality is linked to the policy daddy quota, individuals who view gender equality as an important value will also find a daddy quota (more) appealing. 1 If an individual on the other hand rejects the value (i.e. (s)he do not appreciate the value of gender equality), (s)he will not perceive the associated object (here daddy quota) as congruent with her value preferences, neither as positive nor good (McGee, 1980: 7-8). Expectations drawn from this are that, when aim is to target specific groups of people with ideological left- (right) orientation, left- (right)-laden persuasive words should be most effective. But when aim is to appeal to a large and diverse audience, universal persuasive 1 Effects are reinforced when persuasive words are used in deterministic ways, i.e. if an object is described as necessary in order to achieve the values the persuasive words denotes (Stevenson, 1944:310-18). 9

10 words that are cherished by people of different predispositions should be more favorable (Stevenson, 1944; Walton, 2005; Bench-Capon, 2003). Together, these expectations make the fourth hypothesis: H4 Effects of persuasive words, individual left-right orientation and interpretation of policies in left-right terms, interact in changing peoples attitudes to policy pledges. An overview of hypotheses is presented in Appendix A. DATA AND METHODS Hypotheses are tested in a survey experiment on a diverse sample of 2000 Swedes in the Citizen Panel 2, an online opt-in university-based survey conducted at the Laboratory of Opinion Research (LORE) at the University of Gothenburg. Description of treatment and operationalization of persuasive words Treatment is a fictive election pledge proposing mandatory summer school and longer school days in Junior high school. Policies are chosen taking into consideration that the pledge should be as little connected as possible to the left or the right of the political spectrum. The experiment has a between-subjects design, and respondents are randomly assigned to one of four different versions of the pledge. Policies are the same while persuasive words used to describe the policies vary between universal, left-laden and right-laden. A fourth treatment serves as baseline, and describes policies in neutral terms. Empirical operationalizations of persuasive words are inspired by results from two pilot studies. The first study involved content analyses of election manifestoes of the Swedish parliamentary parties during election campaigns 2002, 2006, 2010 and The second consisted of a small-scale survey on 80 Swedish citizens who were asked to evaluate a list of 2 The Citizen panels consist of a pool of approximately 53,000 self-recruited Swedish citizens. The experiment was run in December 2014 (Martinsson, J., Andreasson, M., Markstedt, E., Riedel, K., Samanni, M., 2014, Technical Report Citizen Panel , Gothenburg: University of Gothenburg, LORE). 10

11 persuasive value-laden words common in Swedish political discourse 3. Persuasive words that were used by parties and appreciated by individuals from both left- and right side of the ideological spectrum inspired to the first treatment condition including universal persuasive words (henceforth Universal treatment or U). Words used only by left (right) parties and appreciated only by left (right) oriented individuals inspired to the conditions including left- (right)-laden persuasive words (henceforth Left- (Right)-laden treatment or L (R)). Universal words that were finally included in the universal treatment are wise, well-functioning, development, quality and knowledge-orientation. Left-laden words included in the leftladen treatment are progressive, fair, equality, social orientation, equal, and rightladen words included in the right-laden treatment are effective, competitive, first-class, ambition and result focus. To enhance the persuasive words, some are repeated in the same treatment (e.g. knowledge-orientation in the universal treatment), and I use words that associate to determinism, for example, knowledge-orientation / result-focused / socially oriented investments are necessary if we are to create a well-functioning / just / competitive school". Exact formulations of the treatments are found in appendix B. Dependent variables Substantial interpretation of policy pledge. The first hypothesis H1 concerns substantial interpretation of policy pledges. To test whether substantial interpretation of the school pledge 3 Subjects were recruited at the University of Gothenburg and the Central train station in Gothenburg. Ss were asked to evaluate a list of words common in the Swedish political debate. First, they were asked to place themselves on a political left-right scale ranging from 1= far to the left and 5= far to the right. Secondly, they were asked to place the words on a like-dislike scale ranging from 1= strongly dislike to 5= strongly like. Finally, they were asked to place the words on a political leftright scale ranging from 1= far to the left to 5= far to the right. Words that in bivariate regression analyses gained a mean over 4 on the like-dislike scale, and where the answers did not differ significantly between left and right oriented Ss, were identified as universal persuasive words. Words with mean scores over four only among Ss with left (right) predispositions were identified as ideological left (right)-laden persuasive words. 11

12 vary due to persuasive words, Subjects (Ss) were asked to describe their interpretation of the intentions with the proposed policies. The sample was divided in two, where one sample was asked to describe their interpretation of the policies using an open-ended question: Please describe with your own words what you perceive that the politicians want to achieve with the proposed policies about the school (N=930), and the other was asked a closed-ended question including 12 pre-defined alternative interpretations based on results from a pilot experiment 4 (N=993). The closed ended-question read: For each statement, please indicate on a scale from 0-10 how well you think the statement is consistent with your understanding of what the politicians want to achieve with the proposed policies (0= do not agree at all to 10= completely agree ). The pre-defined interpretations were; 1) Improving quality of education, 2) Better opportunities for development, 3) Increase equality in education, 4) Increase students knowledge, 5) Create a competitive education, 6) Extra help for students with special difficulties, 7) Improving student results, 8) Fair school, 9) Harder demands, 10) More efficient school, 11) Equal opportunities for all students, 12) Students should spend more time in school. Interpretation of policy pledge in left-right terms. Hypotheses H2 and H3 concerns interpretation of policies in ideological left-right terms. To test whether interpretation of policies in left-right terms vary due to persuasive words, Ss were asked to place the proposed policies on an 11-point left-right scale (0= far to the left to 10= far to the right ). Attitude to policy pledge. The last hypothesis H4 concerns attitudes to policy pledge. To test whether attitudes to the policy pledge vary due to persuasive words, Ss were asked: 4 The pilot was conducted at the central train station in Gothenburg, 2013 (N=320). The Ss, randomly assigned to one of four treatment versions (U, L, R, baseline), were asked to interpret a similar pledge about longer school days in junior high school using the question: Please describe with your own words what you perceive that the politicians want to achieve with the proposed policies about the school: Open answer. Each answer was coded into different categories based on larger dimensions distinguished from the answers. 12

13 What do you think about the proposed school policies? (11-point scale, 0= strongly dislike to 10= strongly like ). Moderating variable To test whether effects of different persuasive words are moderated by individual left-right predispositions, Ss were asked the following question: Political parties are sometimes placed on a political left-right scale. Were would you place yourself on such a scale? (1=far to the left, 2=somewhat to the left, 3=to the center, 4=somewhat to the right, 5=far to the right). Randomization checks. Randomization checks were employed on some demographic variables 5 to test whether randomization of treatments worked. Checks were conducted both for each of the two samples separately, and on the sample in total. Results are reported in appendix C. RESULTS H1 I begin by investigating H1, that persuasive words should generate substantial interpretation of policy pledges that matches the connotations of the persuasive words. I start with analyses of the open-ended question, and proceed with the closed-ended question based on pre-defined interpretations. Results from the analyses are then compared. Starting with the open question, logistic analyses are conducted separately for 14 interpretation categories found in participants open answers (see appendix D for coding). The 5 Variables checked for are gender, age, level of education, left-right predispositions, number of children in household, political interest and political trust. 13

14 analyses reveal differences between Ss exposed to treatments including persuasive words when compared to Ss in the baseline group, and the different interpretations seem to comply largely with the persuasive words used in respective treatment. Results from the open question are reported in table 1. TABLE 1: Effects of persuasive words on substantial interpretation of policies (open answer). Results presented in odds ratio 6, std errors in parentheses. N=930 TREATMENT (ref: Baseline) INTERPRETATION Universal Left Right (ref: Interpret. Not mentioned ) 1: Better education (omitted) (.60) (.22) 2: Individual opportunities.83 (.41).89 (.44) 1.23 (.56) 3a: Increase equality.86 (.34) 3.69*** (1.19).70 (.30) 3b: Increase conformism.56 (.41) 2.72(*) (1.46) 1.2 (.74) 4: Improve knowledge 2.44*** (.62).80 (.24).87 (.26) 5: Competitive education.61 (.23).37* (.17) 1.24 (.41) 6a: Ext. ass. for students in need.76 (.25) 1.36 (.40).58 (.20) 6b: Ext. ass. for all students 1.03 (.46) 1.12 (.50).89 (.42) 7: Improve results 1.03 (.32).67 (.23) 1.92* (.55) 8: Harder demands.99 (.24).46** (.14).59(*) (.16) 9: More efficient school 1.88 (2.31) 1.01 (1.43) 7.16(*) (7.68) 10: More time in school.90 (.24).61(*) (.18).53* (.16) 11: Better school.82 (.43).75 (.41).87 (.46) 12: Eligibility to high school 1.24 (.36) 1.25 (.37).95 (.29) 13: Other 1.43 (.37) 1.09 (.30) 1.11 (.31) 14: Do not know/ do not want to answer 1.53 (.88).81 (.55) (omitted) NOTES. Method: Logistic regression. Significance levels: ( * ) p<= 0.1, * p<= 0.05, ** p<= 0.01, *** p<= In some cases, variables are omitted due to collinearity. 6 Critical values for strength of association: Strong (OR>3), moderate (OR= ), weak (OR= ). 14

15 Looking at table 1, we first see that the odds of Ss interpreting the policies to be about Improving knowledge are significantly higher (2.44 times) when they are exposed to universal persuasive words compared to when they are exposed to the baseline version of the treatment (sig. 99.9% level). The universal treatment included the persuasive word knowledge orientation, hence the differences in interpretation are in line with expectations; universal persuasive words generate substantial interpretations of policies that match connotations of the words. Moving on to the left-laden treatment including the left-laden persuasive words equality and fairness, we see that Ss are more likely to interpret the policies as increasing equality and/or conformism in school, compared to Ss in the baseline group. The odds of interpreting the policy as to Increase equality are 3.69 times higher for Ss exposed to the left-laden treatment than for Ss in the baseline group (sig. 99.9% level), and the odds of Ss interpreting the policies as to Increase conformism are 2.72 times higher than for Ss in the baseline group (sig. 90% level). Hence, also left-laden persuasive words seem to generate interpretation of policies that match connotations of the words. Likewise previous treatments, the right-laden treatment proves to increase Ss likability of interpreting policies in ways that comply with right-laden persuasive words used to describe the policies; result-focus and effectiveness. The odds of interpreting the policies as to Improving results are for these Ss 1.92 times higher than for Ss in the baseline group (sig. 95% level), and odds of interpreting the policies as to create a More efficient school are 7.16 times higher for Ss exposed to the right-laden treatment than for Ss in the baseline group (sig. 90% level). Moreover, the odds of interpreting policies as being about Students spending more time in school are lower both for Ss exposed to left-laden words (sig. 90% level) and for Ss exposed to right-laden words (sig. 95% level), than for Ss in baseline group. This 15

16 interpretation complies with the factual content of the school policy extending the school day, and the effects are therefore anticipated. People exposed to policies that are not described by persuasive words should be more likely to interpret the policies based on factual qualities of the policy, since they are not manipulated by words that can influence their perceptions. In conclusion, analyses of the open question support H1; there are significant effects of persuasive words on Ss substantial interpretation of policies, and the results hold for universal, left- and right-laden persuasive words. Moving on to the closed-ended question. Twelve one-way ANOVA analyses are employed on Ss interpretation of the policies when using pre-defined alternative categories. The analyses reveal significant differences between the Ss exposed to each treatment involving persuasive words compared to Ss exposed to the baseline version, for 9 of the 12 pre-defined interpretations. Most results are in accordance with expectations in that interpretations comply with persuasive words used in respective treatment (7 out of 9 significant results are in accordance with expectations). Results from the closed-ended question are reported in table 2 (two-way ANOVA analyses), and 3 (pairwise comparisons for statistical significance). 16

17 TABLE 2: Effects of persuasive words on substantial interpretation of policies (pre-defined interpretations). Results presented in means, std errors in parentheses. N = 993 TREATMENT INTERPRETATION U L R Baseline Total F-test Sig. 1: Quality (3.08) (3.03) (3.08) (3.32) (3.14) 2: Indiv. opportunities 6.63 (2.70) 6.54 (2.63) 6.19 (2.90) 6.22 (3.03) 6.40 (2.82) :Increasing equality 5.82 (3.12) 6.66 (2.54) 5.46 (3.05) 5.49 (3.09) 5.86 (2.99) : Improve knowledge 6.84 (2.76) 6.08 (2.70) 6.78 (2.64) 6.02 (2.90) 6.44 (2.77) : Competitive edu (2.72) 5.76 (2.96) 6.95 (2.58) 5.84 (3.14) 6.24 (2.89) : Help stud. ext. needs 7.68 (2.22) 7.98 (2.04) 7.40 (2.55) 7.63 (2.37) 7.67 (2.31) : Improve results 7.89 (2.08) 7.51 (2.28) 7.80 (2.25) 7.50 (2.39) 7.68 (2.25) : Fair school 6.01 (2.87) 6.71 (2.67) 5.56 (3.03) 5.81 (3.04) 6.02 (2.93) : Higher demands 6.70 (2.71) 6.24 (3.01) 7.31 (2.29) 6.85 (2.68) 6.78 (2.70) : Efficient school 6.01 (2.95) 5.54 (2.96) 6.36 (2.78) 5.64 (3.07) 5.90 (2.95) : Equal opportunities 6.48 (2.94) 7.13 (2.56) 5.88 (3.08) 6.12 (3.00) 6.40 (2.93) : More time in school 7.14 (2.39) 7.32 (2.58) 7.46 (2.43) 7.21 (2.48) 7.28 (2.47) NOTES. Method: One-way ANOVA. Dependent variables consist of scales ranging from 0=do not agree at all, to 10=completely agree. 17

18 TABLE 3: Effects of persuasive words on substantial interpretation of policies (pre-defined interpretations): Pairwise comparison tests of statistical significance. TREATMENT (ref: baseline) INTERPRETATION Universal Left Right 1: Quality : Individual opportunities : Increasing equality : Improve knowledge : Competitive education : Help students with extra needs : Improved results : Fair school : Higher demands : Efficient school : Equal opportunities : More time in school NOTES. Tukey hsd test of statistical differences: pairwise comparisons between baseline treatment, and the universal, left-and right-laden treatment, respectively. In table 2 we first see that Ss exposed to universal persuasive words agree more with interpretation Improving knowledge (mean=6.84) than do Ss in baseline group (mean=6.02), (p.000). Since the universal treatment included the persuasive word knowledge orientation results are in line with expectations. Next, we see that Ss exposed to the left-laden treatment agree more with three alternative interpretations when compared to Ss in the baseline group, which comply with left-laden persuasive words used in this treatment: equality and fair. First, the Ss agree more with Increasing equality (mean=6.66) compared to Ss in the baseline group (mean=5.49), (p.000). Secondly, Ss agree more with interpretation Fair school (mean=6.71) compared to Ss in the baseline group (mean=5.81), (p.005), and third they agree more with Equal opportunities (mean=7.13) compared to Ss in the baseline group (mean=6.12), (p.001). Finally, when looking at Ss exposed to the right-laden treatment, we see that also they agree more on three interpretations compared to the baseline group, which comply fairly well 18

19 with persuasive words used in this treatment: competitive, ambition, result focus and effective. First, the Ss agree more with interpretation Competitive education (mean=6.95) compared to Ss in the baseline group (mean=5.84), (p.000). Secondly, they agree more with interpretation Efficient school (mean=6.36) compared to Ss in the baseline group (mean=5.64), (p.038), and third they agree more (although not significantly) with Harder demands (mean=7.31) compared to Ss in the baseline group (mean=6.85), (p.243). In conclusion, also the analyses of the closed-ended question support H1; left-laden, right-laden and universal persuasive words generate substantial interpretation of policies that matches persuasive words used in respective treatment. When comparing the results from the open- and closed-ended question, we find consistent support for H1. When compared to Ss in the baseline group, Ss exposed to the universal treatment are more likely to interpret the policies as to Improve knowledge, Ss exposed to the left-laden treatment are more likely to interpret the policies as Increasing equality, and Ss exposed to the right-laden treatment are more likely to interpret the policies as to create a More efficient school. Hence, all three types of persuasive words, left-laden, right-laden and universal seem to generate substantial interpretation of the proposed policies that matches the connotations of the same. H2 and H3 To test hypothesis H2 and H3, whether persuasive words have the same or different effects on interpretation of policies in left-right terms due to individual left-right predispositions, I look for interactions between effects of persuasive words and S s left-right predispositions. First, a two-way ANOVA 7 show significant main effects of persuasive words on S s interpretation of the policies in left-right terms, F 3, 1886 = 55.76, p.000, and as such, the main results of the treatments are in accordance with expectations. The baseline group places the 7 Results of the two-way ANOVA are reported in detail in appendix F, table F.1. 19

20 policies in the center of the 11-point left-right scale (mean=5.34). This is expected, since the factual qualities of the policies were designed to be neutral to a political left-right dimension. Also the group exposed to universal words places the policies in the center (mean=5.33). Since the analyses do not make a difference here between S s of different ideological predispositions, also these results are in line with expectations. In further support of expectations, Ss exposed to the left-laden treatment place the policies furthest to the left (mean=4.15), whereas S s exposed to the right-laden treatment place them furthest to the right (mean=5.78). The two-way ANOVA also reveal significant main effects of Ss individual left-right predispositions on interpretation of the policies in left-right terms, F 2, 1887 = 2.77, p.063. H2 proposes that people should place the policies to the left (right) when they are described with left- (right)-laden persuasive words, independent of individual left-right predispositions. H3 propose that, when policies are described with universal words, people should place the policies to the left- or to the right in accordance with their individual left-right orientation. Hence, we should expect there to be some interactions between type of persuasive words and Ss left-right orientation. According to the ANOVA analyses, there seem to be no overall main interaction between effects of persuasive words and Ss left-right orientation, F 6, 1878 = 0.81, p.560, (see figure 1). 20

21 FIGURE 1. Two-way interaction of persuasive words and individual left-right predispositions, on interpretation of policies in left-right terms. Left-right interpretation Left Center Right Left-right predispositions Baseline L U R NOTES. Method: Two-way ANOVA. Predictions with 95% confidence intervals. Original scale for individual left-right predispositions: 1=far to the left, 2=somewhat to the left, 3=to the center, 4=somewhat to the right, 5=far to the right. However, there is still a possibility that, at some levels of the independent variables, there are significant single main interactions between persuasive words and Ss left-right predispositions. Therefore, in order to test the hypotheses, we need to look closer at effects of persuasive words at different levels of individual left-right predispositions; how do individuals to the left-, center- and right respond to different types of persuasive words when interpreting the policies in left-right terms? I proceed with pairwise comparison tests of the effects of the treatment, at each level of individual left-right orientation (results are presented in table 4). 21

22 TABLE 4: Interactions between effects of persuasive words and individual left-right orientation, on interpretation of policies in left-right terms: Pairwise comparison tests of statistical significance. Treatment: group vs group Group means Mean diff. HSD-test At individual predispositions: Left Baseline vs U Baseline vs L * Baseline vs R * Universal vs L * Universal vs R * Left vs R * At individual predispositions: Center Baseline vs U Baseline vs L * Baseline vs R Universal vs L * Universal vs R Left vs R * At individual predispositions: Right Baseline vs U Baseline vs L * Baseline vs R Universal vs L * Universal vs R Left vs R * NOTES. Significance level: * p<= Original scale for individual left-right predispositions: 1=far to the left, 2=somewhat to the left, 3=to the center, 4=somewhat to the right, 5=far to the right. Tukey HSD pairwise comparison tests for treatment variable; studentized range critical value (.05, 4, 1878) = I start with comparisons relevant for H2; that ideological persuasive words associated with left (right) should generate interpretation of proposed policies as left- (right)-laden, irrespective of individuals left-right predispositions. So far, we know that there is a main effect of persuasive right- and left-laden persuasive words; in general, Ss place the policies further to the left (right) in accordance with the ideological leaning of the persuasive words used to describe the policies. To determine whether H2 is correct however, we also need to be sure that there are no significant differences in where Ss to the left-, center- and right place 22

23 the policies on a left-right scale. Therefore, I compare means of Ss exposed to the left-laden and right-laden treatment to the means of Ss in baseline group, at each level of individual leftright predispositions. In table 4, we first see that both individuals of left-, right- and center orientation place the policies further to the left when they are exposed to left-laden persuasive words. Mean for individuals with center orientation exposed to left-laden persuasive words is 4.20 compared to 5.41 when exposed to the baseline treatment (hsd test = 7.36*), mean for individuals with left orientation exposed to left-laden persuasive words is 4.08 compared to 5.39 when exposed to the baseline treatment (hsd test = 8.13*), and mean for individuals of right orientation exposed to left-laden persuasive words is 4.18 compared to 5.21 when exposed to the baseline treatment (hsd test: 6.47*). Hence, effects of left-laden persuasive words support H2. In table 4 we can also see that the effects of the right-laden treatment are in accordance with expectations; both individuals of left-, right- and center orientation place the policies further to the right when they are described using right-laden persuasive words compared to when they are described in neutral terms (baseline version). However, here the effects of the treatment are significant only for individuals of left orientation (hsd test: 4.46*). In conclusion, although not all results are significant, the pairwise comparison tests overall support H2; Ss exposed to left-laden persuasive words place the policies further to the left than Ss in the baseline group, whereas Ss exposed to right-laden words place the policies further to the right, irrespective of individual left-right orientation. I now move on to comparisons relevant for H3, that universal persuasive words should generate interpretation of proposed policies in accordance with individuals ideological predispositions. To test this, I compare the means of the universal treatment group and the baseline group, at each level of individuals left-right predispositions. Since H3 23

24 propose that people should interpret the policies to be in accordance with their own ideological predispositions when the policies are described using universal persuasive words, Ss of left orientation are expected to place the policies further to the left, Ss of center orientation in the center, and Ss of right orientation should place them further to the right compared to Ss in baseline group. The pairwise comparison tests in table 4 reveal no support for this hypothesis. Instead, the tests show that, irrespective of ideological predispositions Ss placed the policies close to the center, and there are no significant differences in means between Ss exposed to the universal treatment and Ss in baseline group, neither for individuals of left orientation (hsd test = 0.38), center orientation (hsd test = 0.32), nor right orientation (hsd test = 0.24). I therefore have to reject H3, universal persuasive words do not generate interpretation of policies in left-right terms in accordance with individuals left-right orientation. H4 To test the last hypothesis H4, that persuasive words, individual left-right predispositions and interpretation of policies in left-right terms should interact in influencing peoples attitudes to policy pledges, a three-way ANOVA is employed. In table 5 we see that there are a significant main three-way interaction between effects of the three variables; F 12, 1891 = 1.74, p=.053. It also reveal significant single main effects of each of the three variables, persuasive words: F 3, 1891 = 2.53, p =.055, individual left-right orientation, F 2, 1891 = 10.47, p =.000, and interpretation of policies in left-right terms, F 2, 1891 = 23.47, p =

25 TABLE 5: Three-way interaction between effects of persuasive words, individual left-right predispositions and interpretation of policies in left-right terms, on attitudes to policies. Source Partial SS df MS F Corrected Prob > F F-ratio T LR:pre LR:int T*LR:pre T*LR:int LR:pre*LR:int T*LR:pre*LR:int LR:pre*LR:int at T Critical value for p<= LR:pre*LR:int at T= LR:pre*LR:int at T= LR:pre*LR:int at T= * 4.39 LR:pre*LR:int at T= * 4.39 LR:int at LR:pre (T=3) LR:int at LR:pre=1 & T= * 4.1 LR:int at LR:pre=2 & T= * 4.1 LR:int at LR:pre=3 & T= * 4.1 LR:int at LR:pre (T=4) LR:int at LR:pre=1 & T= * 4.1 LR:int at LR:pre=2 & T= * 4.1 LR:int at LR:pre=3 & T= * 4.1 Residual Total NOTES. Method: three-way ANOVA. Significance level: * = p <=0.05. Original scale for individual left-right predispositions: 1=far to the left, 2=somewhat to the left, 3=to the center, 4=somewhat to the right, 5=far to the right. A significant three-way interaction means that there is a two-way interaction that varies across levels of a third variable. In this model, the factor variable Treatment (T) has four levels (1=Baseline; 2=U; 3=L; 4=R). The first interaction variable individual left-right predispositions (LR:pre) has 3 levels (1=Left; 2=Center; 3=Right), and the second interaction variable interpretation of policies in left-right terms (LR:int) is recoded into three levels (1=Left; 2=Center; 3=Right). The dependent variable, attitudes to policy pledge is a scale ranging from 0=strongly dislike, to 10=strongly like. Because Stata uses the wrong error term when calculating the F-ratio, an error term based on just 449 degrees of freedom, and not on the 1856 in the original model, I use a corrected F-ratio based on the mean square residual from the original three-factor model. The formula for the correct computation of the F-ratio is: F(df between Ss: 2, df within Ss: 1856) = MS(X2: L-R predispositions * X3: L-R interpretation) /MS(residual) = Y (correct F-ratio). 8 Critical value is determined using the per family error rate method. 25

26 To get an understanding of the interaction effects, I first look at the two-way interactions between individual left-right orientation and interpretation of policies in left-right terms. Figure 2 illustrate means in attitudes for individuals of different predispositions and interpretation of policies, in each treatment group. As expected, the effects varies; in all treatment groups Ss of left orientation like the policies best when they interpret them as leftladen, Ss of center orientation like the policies best when they interpret them as center, and Ss of right orientation like the policies best when they interpret them as right-laden. FIGURE 2: Margins plots for interaction between effects of individual left-right predispositions and interpretation of policy in left-right terms. Baseline Treatment U Attitues to policy Attitudes to policy left center right Interpretation of policy left center right Interpretation of policy Left orientation Center orientation Left orientation Center orientation Right orientation Right orientation Treatment L Treatment R Attitudes to policy Attitudes to policy left center right Interpretation left-right left center right Interpretation left-right Left orientation Center orientation Left orientation Center orientation Right orientation Right orientation NOTES. Method: Two-way ANOVA. Predictions with 95% confidence intervals. Original scale for individual left-right predispositions: 1=far to the left, 2=somewhat to the left, 3=to the center, 4=somewhat to the right, 5=far to the right. 26

27 To find out in which treatment groups the two-way interactions are significant I use tests of simple main effects. ANOVA analyses are run for individuals left-right predispositions and interpretation of policies in left-right terms, for each treatment group (results are reported in table 5). A critical value for 95% significance level for simple main effects is determined using the per family error rate method 9, showing a critical value of When comparing the F-ratios 10 to this value, we see that the F-ratio for Ss exposed to left-laden words (9.17), and the F-ratio for Ss exposed to right-laden words (11.04), are statistically significant (above the critical value of 4.39). The F-ratio for Ss in the baseline group (3.51) and for Ss exposed to universal persuasive words (3.11) are however not significant (below the critical value). In other words, there are two-way interactions between left-right predispositions and interpretation of policies in left-right terms only when persuasive words are ideological (leftor right-laden). To further understand where exactly the interactions in the left- and the right-laden treatment occur, I proceed with testing differences in the levels of interpretation of policies in left-right terms (1=left, 2=center, 3=right) at each level of individual left-right predispositions (1=Left, 2=Center, 3=Right), in left- and right-laden treatment group, separately (see table 5). A per family error rate test show a critical value of 4.10, and when comparing the F-ratios 11 to this value, we see that all four F-ratios are above the critical value for 95% statistical significance. Hence, we can conclude that there are significant simple main interactions at 9 The method works by dividing 0.05 (the alpha level), by 4 (four tests of simple main effects), using the invfprob function in stata (see Kirk, 1995 for more information about the method). 10 Because Stata uses the wrong error term when calculating the F-ratio, an error term based on just 449 degrees of freedom, and not on the 1856 in the original model, I use a corrected F-ratio based on the mean square residual from the original three-factor model. The formula for the correct computation of the F-ratio is: F(df between Ss: 2, df within Ss: 1856) = MS(X2: L-R predisposotions * X3: L-R interpretation) /MS(residual) = Y (correct F-ratio). Correct F-values are reported in column 6 in table Like in the two-way interaction model for left-right predispositions and interpretation, this model uses the wrong error term. The F values have therefore been corrected (see column 6 in table 5). 27