TESTING FOR HIERARCHICAL STRUCTURE AND PRIMING EFFECTS AMONG INDIVIDUAL VALUE CHOICES

TESTING FOR HIERARCHICAL STRUCTURE AND PRIMING EFFECTS AMONG INDIVIDUAL VALUE CHOICES William G. Jacoby Department of Political Science Michigan State University 303 South Kedzie Hall East Lansing, MI 48824 jacoby@msu.edu A preliminary report on a study supported by the program for Time-Sharing Experiments in the Social Sciences February 2006

This paper is a preliminary report from a study of value choices within the American mass public. Longstanding psychological theories have stressed that individual value preferences are very stable, highly structured and broadly applicable to virtually all situational contexts. But, recent research suggests that value choices may be relatively malleable and susceptible to contextual effects. Furthermore, the apparent existence of value ambivalence raises questions about citizens abilities to differentiate among values in the first place. At the present time, it is not clear which of these two general perspectives provides the most accurate representation of core values. The program for Time-Sharing Experiments in the Social Sciences (TESS) provided support for a study that used some new items on an internet survey in order to to address the important issues raised by these potentially conflicting perspectives. The empirical results show that most people do make consistent pairwise choices among values. Furthermore, most individuals sets of pairwise choices are fully transitive, meaning that they can be assembled into hierarchical structures. Finally, value choices are largely impervious to priming effects. SUBSTANTIVE BACKGROUND For present purposes, values can be defined as an individual s abstract, general conceptions about the desirable and undesirable end-states of human life ( 1973). As such, values provide criteria for evaluating external stimuli and interacting with other elements of the social environment. They effectively define what is good and bad in the world. Human values have been the focus of an enormous amount of research across a variety of disciplines, including political science (Kuklinski 2001; Feldman 2003), sociology (Spates 1983), economics (Katona 1975), marketing (Kamakura and Mazzon 1991), and philosophy (Hansson 2001). However, the modern, empirical study of values really began in psychology, particularly with the pioneering work of Milton Rokeach (1973; 1979). Subsequent research has proceeded in a wide variety of directions (Seligman, Olson, Zanna 1996). But, there are two general ideas drawn from this research tradition that deserve particular emphasis: First, a critical distinguishing feature of values is their applicability across different contexts

(Schwartz and Bilsky 1987; Schwartz 1996). Values provide individuals with general evaluative standards that can always be applied, regardless of the specific circumstances under which the values are invoked. Second, people are rarely affected by single values, in isolation from other values. Instead, hierarchically-organized value structures are the key to understanding human behavior (e.g., Schwartz and Bilsky 1987; Schwartz 1992; 1996; Verplanken and Holland 2002). There is a clear consensus in the psychological literature regarding the contextual stability and hierarchically structured nature of individual values. However, recent research particularly, in political science has taken very different, and potentially conflicting, positions with respect to these value characteristics. For example, theories of value pluralism (Tetlock 1986), issue framing (Nelson, Clawson, Oxley 1997; Grant and Rudolph 2003), and campaign effects (McCann 1997) all hold that value choices are context-specific and changeable in different circumstances. In a somewhat different vein, theories of value ambivalence (Feldman and Zaller 1992; Alvarez and Brehm 2003) and indifference (Maio and Olson 1998) suggest that people may have trouble making and/or expressing reliable and consistent choices between values in the first place. The apparent variability and ambivalence in human values pose serious challenges for traditional psychological theories. If individual value preferences are changeable with contextual factors, then the transsituational (Schwartz 1996) nature of values is badly compromised (Seligman and Katz 1996). Similarly, ambivalence about values challenges the existence of transitive choices and, thereby, largely precludes the existence of individual value structures (Maio and Olson 1998; Jacoby 2002). Thus, it is critically important to determine whether the recent findings have merit, or whether the traditional theoretical understanding provides the more accurate depiction of individual values. The existing empirical evidence is ambiguous and largely indeterminate. For example, studies that seem to reveal framing effects usually examine differences in reactions to single values. In fact, preference orders across multiple values do not vary systematically across 2

issue frames (Nelson et al. 1997, p. 576). Similarly, most studies of ambivalence never test directly individual willingness or ability to make choices between specific values; instead, they examine indirect evidence based upon the supposed consequences of value ambivalence (Feldman and Zaller 1992; Alvarez and Brehm 2002). Finally, and perhaps most fundamentally, the very existence of hierarchical value structures has never been rigorously tested. Again, some studies simply focus on single values, taken separately from other values (e.g., Feldman 1988). In other cases, researchers ask subjects to rank-order a set of values (e.g., Rokeach 1973; Schwartz 1992). Both of these approaches bypass any direct assessment of consistency or transitivity in value choices. The one line of work that does attempt to construct preference orders from pairwise value choices (Jacoby 2002; 2006) is based upon a very limited dataset which precludes evaluation of the measurement error that undoubtedly exists in expressed statements of choices between values (Miethe 1985; McCarty and Shrum 2000). STUDY DESIGN AND DATA COLLECTION The problems outlined in the previous section are addressed by using a new set of items, administered as part of an internet survey that was supported by the program for Time- Sharing Experiments in the Social Sciences. The project focuses on individual choices among five important values: Liberty, economic security, equality, morality, and social order. These particular values were selected because of their salience in American political culture and their relevance to current policy controversies. The values were presented to the survey respondents under varying experimental conditions aimed at priming particular values and value conflicts. Respondents made choices among subsets of the values in a manner that enables empirical tests of hierarchical value structuring. Such tests are impossible with more traditional data collection strategies (i.e., rating single values or rank-ordering the full set of values). The overall plan of the study is laid out in Figure 1. Respondents were randomly assigned to one of eight conditions. Seven of these conditions involve two survey questions that either 3

emphasize a specific value (five conditions) or the conflict between a pair of values (two conditions). A control group was given two questions that do not mention values. The questions used in each of the conditions are shown in Part I of the Appendix. Next, all respondents were asked to place themselves along the liberal-conservative continuum using a relatively standard seven-point scale (Part II in the Appendix). This item served as a distraction task for respondents in the seven experimental conditions. The responses to this question are also intrinsically important because ideology is one of the factors that is hypothesized to affect value ambivalence (Feldman and Zaller 1992) and, hence, the degree of hierarchical structuring among individual value choices. The value choices were elicited next, using the method of triads (Coombs 1964; Weller and Romney 1988). Respondents were first shown a screen that introduced, and provided a brief definition for, each of the five values (Part IIIA in the Appendix). After that, they were shown a series of ten screens. On each screen, a distinct combination of three values (i.e., a triad ) was listed. For each triad, respondents were asked to indicate which one of the three values is most important, and which one of the three is least important. Given the nature of the internet survey, respondents could make the selections very easily, by clicking radio buttons with the computer mouse. The ten triads (i.e., all possible subsets of three values from the five) are shown in Part IIIB of the Appendix. The order of the triads was varied randomly, as was the order of the three values within each triad. The internet survey was conducted by Knowledge Networks, Inc.; it is designated TESS 37 Jacoby. Data collection was carried out from June 16 through June 21, 2005. A total of 1000 surveys were fielded, and 649 respondents completed the interview schedule. Thus, the response rate was about 65 %. CONSISTENCY AND TRANSITIVITY IN VALUE CHOICES Let us begin by considering choices among pairs of values. The individuals responses to each triad can be broken down into three pairwise choices. For example, assume a triad 4

containing values A, B, and C, with a respondent stating that A is most important and C is least important. This, in turn, implies that A is more important than B, B is more important than C, and A is more important than C. In the full set of ten triads, each pair of values appears in three different triads. Therefore, three replications of each respondent s choice between each value pair are recovered. This allows an assessment of consistency in pairwise choices (i.e., how many times does the respondent make the same choice for each pair?) and permits the determination of that person s dominant choice (i.e., the value chosen two or three times) for each value pair. Consistency in Pairwise Choices Table 1 presents basic descriptive data on the pairwise value choices. Each row of the table represents one of the ten distinct value pairs that can be derived from the set of five values. Beginning at the left side of the table, the first four columns represent the number of times the first-mentioned value in that row is chosen over the second-mentioned value, across the three replications for that pair. The rightmost column within each row summarizes the amount of consistency in choices involving that value pair; that is, the percentage of times respondents chose one or the other value consistently across all three replications of that particular value pair. The entries in Table 1 show that the respondents are highly consistent in their value choices. In every case, 70% or more make the same pairwise choice across all three replications. Stated somewhat differently, the percentage of respondents who exhibit some degree of inconsistency ranges from a high of 29.58% (for choices between equality and morality) or just under one-third of the sample, to a low of 19.57% (for choices between morality and social order) or just under one-fifth of the sample As a standard of comparison for the entries in Table 1, consider an admittedly unrealistic situation in which everyone is indifferent between two values (due to ambivalence, failure to recognize any potential conflict between the values, and so on). In that case, the probability 5

of choosing one value over the other on a single choice is 0.50. And, the probability of making consistent choices across all three replications is only 0.25. In other words, if people did not maintain real preferences between a given pair of values, then only one-fourth of the sample should be exhibiting consistent choices across the three replications of each choice. But, again, the empirical levels of consistency are much higher than this stringent standard. This, in turn, suggests that most people really do have crystallized preferences across the values examined in this study. Variability in Value Importance The replicated pairwise choices can be used to determine each person s dominant choice for each value pair; that is, the value that is chosen two or three times (i.e., a majority) across the replications. These dominant pairwise choices can, in turn, be used to determine each respondent s beliefs about the relative importance of the separate values. This is accomplished by assigning each value a score (for each survey respondent), showing the number of times that value is the dominant choice over other values in that person s full set of choices (Peffley, Knigge, Hurwitz 2001). As a simple example, consider three values: A, B, and C. Assume that these are presented to respondents in triads (probably along with other values), and that an arbitrary respondent has A dominating B (i.e., A was the selected over B on at least two of the three choices), A also dominating C, and B dominating C. For this person, A would receive a score of 2, since he/she has it dominating two other values (B and C); B would be scored 1 (it only dominates C); and C would receive a 0, since it never dominates any other values (even though it may have been chosen in one of the replicated pairwise choices). With the Knowledge Networks data, there are five values, so the scores for each value can range from zero to four. Table 2 shows the distribution of importance scores for each of the four values. The most striking feature of the table is the wide variability among the preferences. At the individual level, there does not seem to be general agreement about which values are most (or least) important. Morality received the largest number of high ( 4 ) scores but even so, only 6

about one-third of the respondents rated this value as more important than any other. And, very few people rated equality or social order very highly; less than eight percent of the respondents placed each of these values at the top of their rankings. At the other extreme, social order received the largest number of zero scores. But, again, only about 30% of the respondents ranked this value in last place. Overall, it is fair to say that each of the four values shows up in every possible position within the value hierarchies of a substantial number of people. The dot plot shown in Figure 2 summarizes succinctly the aggregate distribution of importance ratings for each value. Specifically, the horizontal position of the point plotted within each row corresponds to the mean importance score for that value; the solid bar around each point represents a 95% confidence interval. From the figure, it can be seen that morality, economic security, and liberty are considered to be the most important values. Their mean importance scores are 2.36, 2.32, and 2.29, respectively. However, as can be discerned from the overlapping confidence intervals in Figure 2, the differences between these means are not statistically significant. The remaining two values, equality and social order, have significantly lower mean importance scores, at 1.58 and 1.45, respectively. Again, the difference between these two means is not statistically significant. At the aggregate level, Americans do not seem to differentiate fully among the different values. Instead, public opinion sorts values into two distinct sets: One group that is relatively more important, composed of morality, liberty, and economic security and a second group that is usually viewed as less important, equality and social order. Are Individual Value Choices Transitive? For each survey respondent, the full set of dominant pairwise choices can be examined for transitivity. If transitivity holds in a respondent s value choices then, for any set of three values (again, say A, B, and C), choices on two of the pairs imply the choice that will be made on the third pair. So, if A is chosen over B, and B is chosen over C, then transitivity implies that A is also chosen over C. If an individual s choices among the values are fully 7

transitive (i.e., the transitivity relation holds for all possible subsets of three values), then each of the values will receive a distinct score. Therefore, it is possible to assemble that person s pairwise choices into a complete rank-ordering of the values. Figure 3 provides a simple example showing how the dominant pairwise pairwise choices are obtained from a hypothetical individual s responses to the triads, and how the dominant choices used to construct an overall preference ordering for that person. Intransitivity exists in a set of choices among three values whenever the third choice is not implied by the other two choices. For example, A is chosen over B, B over C, and then C is chosen over A. Here, each value is only selected over a single other value, so it is impossible to construct a rank-ordering of the three; hence, the three choices form an intransitive set. Thus, the relative prevalence of transitivity and intransitivity in the data constitutes a direct empirical test for the existence of individual value hierarchies. Table 3 provides the basic data on transitivity among individual value choices. The first row of the table shows that only 12.48% of the respondents exhibited any intransitivity among all of their dominant pairwise choices. Stated differently, nearly nine-tenths of the respondents maintain fully-ordered (i.e., completely transitive) value preferences. The remaining rows in the table give the proportion of intransitive choices on all subsets of three values. Parenthetically, note that some people are intransitive on more than one triad; hence the proportions intransitive on the particular triads sum to more than the 12.48% figure given above. The information in Table 3 shows a very obvious pattern: On any given triad, only a tiny minority of the respondents give intransitive choices among the values. The number of intransitivities never rises above about three percent on any value triad, and it is generally smaller than that. Clearly, the vast majority of the respondents have no difficulty providing a complete ordering of the five values. This, in turn, provides strong confirmation for the existence of fully-differentiated hierarchical value structures within the mass public. 8

CAN VALUE CHOICES BE PRIMED? The experimental manipulation in this study randomly assigned survey respondents to eight different conditions. Five of these conditions are each intended to prime one particular value. Specifically, respondents were asked to indicate the degree to which the Bush and Kerry presidential campaigns emphasized one of the values (See sections IA through IE in the Appendix for question wording). Two other conditions emphasized value conflict, by asking respondents to indicate the degree to which the presidential campaigns emphasized liberty versus social order (see section IF in the Appendix) and liberty versus equality (see section IG in the Appendix). Finally, the eighth condition did not prime any values; instead, respondents were asked about partisanship in the presidential campaigns (Appendix section IH). After the experimental manipulation, all respondents were asked to place themselves along a seven-point liberal-conservative scale. Then, they completed the battery of value choice items. If values are the transsituational phenomena argued by Rokeach (1973), Schwartz (1996) and others, then priming should have no discernible effect on pairwise value choices. Alternatively, if feelings about values can be manipulated through environmental conditions (e.g., Tetlock 1986; Nelson, Clawson, Oxley 1997; Maio and Olson 1998), then priming particular values should enhance the degree to which those values are chosen when they are juxtaposed against other values. The design of the current study enables multiple tests of the priming hypothesis. With five values, there are ten pairwise choices. For each pair, the dependent variable is the probability that one value is the dominant choice over the other. It is important to emphasize that our empirical measurement of these choices is particularly reliable, since dominant choices result from the choices made over the three replications of each value pair from the triad items. For each of the ten value pairs, there are two independent variables. These are dummies, scored one if one of the two values in the pair was primed, and zero otherwise. For example, assume that the dependent variable is the probability of choosing 9

equality over economic security. In that case, people who received the equality prime are scored one on the first predictor, and people who received the economic security prime are scored one on the other predictor. According to the priming hypothesis, the first predictor should increase the probability of choosing equality, while the second predictor should decrease that probability. Note that respondents in either of the two value conflict conditions are coded as having been primed for both values in the pair. Table 4 contains the results from this analysis. Each row of the table contains the maximum likelihood estimates for a logistic regression predicting a value choice using primes for the two values included in that particular pair. Notice first that the equations do not fit the data very well. All of the pseudo R-squared values are very small (i.e., all but one have zeroes out to at least the second decimal place). Turning to the individual value primes, a number of the coefficients have the wrong sign. That is, five of the coefficients for the first prime are negative (they should be positive) and three of the coefficients for the second prime are positive (they should be negative). Note, however, that only two of the coefficients are statistically different from zero at the 0.05 level (directional test): The equality prime increases the probability of choosing that value over economic security and also over social order. In all of the remaining instances, exposure to a value prior to a pairwise choice situation has no systematic impact on the probability of choosing that value over another value. Furthermore, even the two seemingly-significant coefficients in Table 4 should not be taken too seriously. Since each of the predictors in each of the equations constitutes a potential prime, there are actually 20 tests contained in the empirical results. The existence of these multiple tests should be taken into account when assessing statistical significance. The Bonferroni adjustment for performing n one-sided tests at the α level is α/n. Here, that produces a probability value of 0.0025; in other words, the latter value is the probability level that should be required to reject the null hypothesis at the 0.05 level, across 20 different tests. This would, in turn, require a coefficient that is 2.81 times larger than its standard 10

error. Neither of the significant coefficients in Table 4 meet this more stringent (but more valid) standard. Thus, there is virtually no reliable evidence in these results for priming effects in value choices. CONCLUSION The TESS Program provided a unique opportunity for an empirical analysis of of individual value choices. The preliminary results described in this report lead to several conclusions all, admittedly, somewhat tentative at this point. First, the successful completion of the data collection, itself, shows that the method of triads is a reasonable strategy for obtaining replicated pairwise comparisons among values. This is important because it provides reliable information on value choices. The latter can be used to test for individuals hierarchical value structures. Without this capability, the existence of such rank-ordered structures must remain an untested assumption. Second, the data from the internet survey show that people do, generally, make consistent choices among values. That, in turn, attests to the viability of values as a psychological construct. If values were only truisms, without strong cognitive and affective underpinnings, then we would expect people to move frequently from one to the other when asked to choose between them. But, that just did not happen with these data: The vast majority of the survey respondents made the same choice across the three replications of each value pair. So, this provides strong evidence against criticisms that have been leveled against the values concept in the recent literature. Third, the distribution of value preferences reveals both consensus and conflict in the ways that Americans think about fundamental principles like liberty and equality. There appears to be wide agreement that morality, liberty, and economic security comprise the most important values, with equality and social order falling at somewhat less salient positions in public esteem. But, these aggregate patterns emerge from enormous variability among individual choices among the values. All five of these values are highly important for some subsets of the public, and less so for others. 11

Fourth, individual value choices are usually transitive. This means that people really do differentiate among values. And, that is consistent with the existence of hierarchical value structures throughout the mass public. Fifth, the data show no evidence of priming effects in value choices. Experimental manipulations intended to enhance the accessibility of one value relative to another simply had no effect on the probability of choosing one value over the other. Such a finding is consistent with the view that values are transsituational phenomena, or judgmental standards that are relatively immune to external influences. In conclusion, the initial evidence from this study vindicates psychological theories and scholarly understandings that have been widely accepted, but seldom tested, over the years. Of course, further analysis is necessary in order to deal with questions like the following: What factors influence individual value choices? What causes the inconsistent and/or intransitive value choices that, although relatively rare, do arise in the data? Are there identifiable subsets of the public whose value choices can be affected by environmental factors? Of course, I intend to address all of these questions in the immediate future. But, even at this early point, it is clear that the TESS program s support of this study has enabled important insights regarding value choices in the American mass public. 12

REFERENCES Alvarez, R. Michael and John Brehm. (2002) Hard Choices, Easy Answers. Princeton, NJ: Princeton University Press. Coombs, Clyde H. (1964) A Theory of Data. New York: Wiley. Feldman, Stanley. (2003) Values, Ideology, and the Structure of Political Attitudes. In Oxford Handbook of Political Psychology. David O. Sears, Leonie Huddy, Robert Jervis (Editors). New York: Oxford University Press. Feldman, Stanley and John Zaller. (1992) The Political Culture of Ambivalence: Ideological Responses to the Welfare State. American Journal of Political Science 36: 268-307. Grant, J. Tobin and Thomas J. Rudolph. (2003) Value Conflict, Group Affect, and the Issue of Campaign Finance. American Journal of Political Science 47: 453-469. Hansson, Sven Ove. (2001) The Structure of Values and Norms. Cambridge, UK: Cambridge University Press. Jacoby, William G. (2002) Core Values and Political Attitudes. In Barbara Norrander and Clyde Wilcox (Editors), Understanding Public Opinion (Second Edition). Washington, DC: Congressional Quarterly. Jacoby, William G. (2006) Value Choices in the American Public. American Journal of Political Science. Forthcoming. Kamakura, Wagner A. and Jose Alfonso Mazzon. (1991) Value Segmentation: A Model for the Measurement of Values and Value Systems. Journal of Consumer Research 18: 208-218. Katona, George. (1975) Psychological Economics. New York: Elsevier. Kuklinski, James H. (2001) Introduction: Political Values. In Citizens and Politics: Perspectives from Political Psychology. James H. Kuklinski (Editor). Cambridge, UK: Cambridge University Press. Maio, Gregory R. and James M. Olson. (1998) Values as Truisms: Evidence and Implications. Journal of Personality and Social Psychology 74: 294-311. McCann, James A. (1997) Electoral Choices and Core Value Change: The 1992 Presidential Campaign. American Journal of Political Science 41: 564-583. McCarty, John A. and L. J. Shrum. (2000) Measurement of Personal Values in Research. Public Opinion Quarterly 64: 271-298. Miethe, Terance D. (1985) The Validity and Reliability of Value Measurements. Journal of Psychology 119: 441-453.

Nelson, Thomas E.; Rosalee A. Clawson; Zoe M. Oxley. (1997) Media Framing of a Civil Liberties Conflict and Its Effect on Tolerance. American Political Science Review 91: 567-583. Rokeach, Milton. (1973) The Nature of Human Values. New York: Free Press. Rokeach, Milton. (1979) Understanding Human Values: Individual and Societal. New York: Free Press. Schwartz, Shalom H. (1992) Universals in the Content and Structure of Values: Theoretical Advances and Empirical Tests in 20 Countries. In Mark P. Zanna (Editor), Advances in Experimental Social Psychology. Orlando, FL: Academic Press. Schwartz, Shalom H. (1996) Value Priorities and Behavior: Applying a Theory of Integrated Value Systems. In Clive Seligman, James M. Olson, Mark P. Zanna (Editors), he Psychology of Values: The Ontario Symposium, Volume 8. Mahwah, NJ: Lawrence Erlbaum Associates. Schwartz, Shalom H. and Wolfgang Bilsky. (1987) Toward a Universal Psychological Structure of Human Values. Journal of Personality and Social Psychology 53: 550-562. Seligman, Clive and Albert N. Katz. (1996) The Dynamics of Value Systems. In The Psychology of Values: The Ontario Symposium, Volume 8. Clive Seligman, James M. Olson, Mark P. Zanna (Editors). Mahwah, NJ: Lawrence Erlbaum Associates. Seligman, Clive; James M. Olson; Mark P. Zanna. (1996) The Psychology of Values: The Ontario Symposium, Volume 8. Mahwah, NJ: Lawrence Erlbaum. Spates, James L. (1983) The Sociology of Values. Annual Review of Sociology 9: 27-49. Tetlock, Philip E. (1986) A Value Pluralism Model of Ideological Reasoning. Journal of Personality and Social Psychology 50: 819-827. Verplanken, Bas and Rob. W. Holland. (2002) Motivated Decision Making: Effects of Activism and Self-Centrality on Values in Choices and Behavior. Journal of Personality and Social Psychology 82: 434-447. Weller, Susan C. and A. Kimball Romney. (1988) Systematic Data Collection. Newbury Park, CA: Sage. 14

APPENDIX ITEM LIST FOR INTERNET SURVEY I. Experimental Manipulation. The general objective is to encourage respondents to think about values without explicitly invoking their own value choices. To this end, the items ask respondents to assess the degree to which values were apparent in the 2004 major party presidential campaigns (this, of course, assumes that the survey would go into the field after the election). There are eight different conditions total: Five conditions prime a single value; two conditions prime conflict between two values; and one condition is a control which does not prime any values. In each condition, respondents are asked two questions. Note that the order in which the candidates are mentioned will be varied randomly. A. Liberty condition: In the recent presidential election campaign, the candidates and parties both emphasized certain themes and ideas. We are interested in your feelings about the extent to which the candidates emphasized the idea of liberty, or guaranteeing the widest freedom possible for everyone to act and think as they consider most appropriate. Please use the following scale to indicate how much you think the Bush campaign emphasized liberty. Just use the mouse to click on the appropriate scale value: 1 2 3 4 5 6 7 The Bush campaign did not emphasize liberty at all The Bush campaign placed a great deal of emphasis on liberty How about the Kerry campaign? How much did the Kerry campaign emphasize liberty? 1 2 3 4 5 6 7 The Kerry campaign did not emphasize liberty at all The Kerry campaign placed a great deal of emphasis on liberty

B. Equality condition: In the recent presidential election campaign, the candidates and parties both emphasized certain themes and ideas. We are interested in your feelings about the extent to which the candidates emphasized the idea of equality, or making sure that everybody has the same chance to get ahead in life. Please use the following scale to indicate how much you think the Bush campaign emphasized equality. Just use the mouse to click on the appropriate scale value: 1 2 3 4 5 6 7 The Bush campaign did not emphasize equality at all The Bush campaign placed a great deal of emphasis on equality How about the Kerry campaign? How much did the Kerry campaign emphasize equality? 1 2 3 4 5 6 7 The Kerry campaign did not emphasize equality at all The Kerry campaign placed a great deal of emphasis on equality 16

C. Economic Security condition: In the recent presidential election campaign, the candidates and parties both emphasized certain themes and ideas. We are interested in your feelings about the extent to which the candidates emphasized the idea of economic security, or guaranteeing a steady job and a decent income to everyone. Please use the following scale to indicate how much you think the Bush campaign emphasized economic security. Just use the mouse to click on the appropriate scale value: 1 2 3 4 5 6 7 The Bush campaign did not emphasize economic security at all The Bush campaign placed a great deal of emphasis on economic security How about the Kerry campaign? How much did the Kerry campaign emphasize economic security? 1 2 3 4 5 6 7 The Kerry campaign did not emphasize economic security at all The Kerry campaign placed a great deal of emphasis on economic security 17

D. Morality condition: In the recent presidential election campaign, the candidates and parties both emphasized certain themes and ideas. We are interested in your feelings about the extent to which the candidates emphasized the idea of morality, or people living according to the rules that constitute decent human behavior. Please use the following scale to indicate how much you think the Bush campaign emphasized morality. Just use the mouse to click on the appropriate scale value: 1 2 3 4 5 6 7 The Bush campaign did not emphasize morality at all The Bush campaign placed a great deal of emphasis on morality How about the Kerry campaign? How much did the Kerry campaign emphasize morality? 1 2 3 4 5 6 7 The Kerry campaign did not emphasize morality at all The Kerry campaign placed a great deal of emphasis on morality 18

E. Social Order condition: In the recent presidential election campaign, the candidates and parties both emphasized certain themes and ideas. We are interested in your feelings about the extent to which the candidates emphasized the idea of social order, or being able to live in a safe and peaceful society where the laws are respected and enforced. Please use the following scale to indicate how much you think the Bush campaign emphasized social order. Just use the mouse to click on the appropriate scale value: 1 2 3 4 5 6 7 The Bush campaign did not emphasize social order at all The Bush campaign placed a great deal of emphasis on social order How about the Kerry campaign? How much did the Kerry campaign emphasize social order? 1 2 3 4 5 6 7 The Kerry campaign did not emphasize social order at all The Kerry campaign placed a great deal of emphasis on social order 19

F. Liberty versus Social Order condition (note that the order in which liberty and social order are mentioned will be varied randomly across respondents in this condition): In the recent presidential election campaign, the candidates and parties emphasized the differences between certain themes and ideas. We are interested in your feelings about the how much the two candidates emphasized the idea of liberty (that is, guaranteeing the widest freedom possible for everyone to act and think as they consider most appropriate) versus the idea of social order (that is, being able to live in a safe and peaceful society where the laws are respected and enforced). Please use the following scale to indicate how much you think the Bush campaign emphasized liberty versus social order. Just use the mouse to click on the appropriate scale value: 1 2 3 4 5 6 7 The Bush campaign placed much more emphasis on liberty than on social order The Bush campaign emphasized liberty and social order just about equally The Bush campaign placed much more emphasis on social order than on liberty How about the Kerry campaign? How much did the Kerry campaign emphasize liberty versus social order? 1 2 3 4 5 6 7 The Kerry campaign placed much more emphasis on liberty than on social order The Kerry campaign emphasized liberty and social order just about equally The Kerry campaign placed much more emphasis on social order than on liberty 20

F. Liberty versus Equality condition (note that the order in which liberty and equality are mentioned will be varied randomly across respondents in this condition): In the recent presidential election campaign, the candidates and parties emphasized the differences between certain themes and ideas. We are interested in your feelings about the how much the two candidates emphasized the idea of liberty (that is, guaranteeing the widest freedom possible for everyone to act and think as they consider most appropriate) versus the idea of equality (that is, making sure that everybody has the same chance to get ahead in life). Please use the following scale to indicate how much you think the Bush campaign emphasized liberty versus equality. Just use the mouse to click on the appropriate scale value: 1 2 3 4 5 6 7 The Bush campaign placed much more emphasis on liberty than on equality The Bush campaign emphasized liberty and equality just about equally The Bush campaign placed much more emphasis on equality than on liberty How about the Kerry campaign? How much did the Kerry campaign emphasize liberty versus equality? 1 2 3 4 5 6 7 The Kerry campaign placed much more emphasis on liberty than on equality The Kerry campaign emphasized liberty and equality just about equally The Kerry campaign placed much more emphasis on equality than on liberty 21

E. Control Group, or Political Party condition: In the recent presidential election campaign, the candidates and parties both emphasized certain themes and ideas. We are interested in your feelings about the extent to which the candidates emphasized their ties to their own political parties. Please use the following scale to indicate how much you think the Bush campaign emphasized its ties to the Republican party. Just use the mouse to click on the appropriate scale value: 1 2 3 4 5 6 7 The Bush campaign did not emphasize the Republican party at all The Bush campaign placed a great deal of emphasis on the Republican Party How about the Kerry campaign? How much did the Kerry campaign emphasize the Democratic party? 1 2 3 4 5 6 7 The Kerry campaign did not emphasize the Democratic party at all The Kerry campaign placed a great deal of emphasis on the Democratic Party 22

II. Distraction Task. The idea here is to encourage respondents to think about something other than the value or values that were included in their version of the experimental manipulation. The latter should still remain accessible in short-term memory. But, this distraction is intended inhibit the degree to which their own expressed value choices (obtained immediately afterward) are based directly upon their responses to the experimental manipulation items. Many people use the labels liberal, moderate, or conservative to describe their own political views. Which one of the following labels best describes your own political ideas? Please use the mouse to click the check box next to the appropriate label: Q Extremely conservative Q Conservative Q Sightly conservative Q Moderate, middle of the road Q Slightly liberal Q Liberal Q Extremely liberal Q I don t generally think of myself in these terms. 23

III. Value Triads. The objective here is to elicit respondents value preferences in a manner that provides repeated measurement of each pairwise value comparison and does not presuppose the existence of a fully-ranked value hierarchy. A. Instruction Screen: We d like to ask you about some things that are important for our society, such as liberty, equality, economic security, morality, and social order. First, here is what we mean by these ideas: By LIBERTY we mean a guarantee of the widest freedom possible for everyone to act and think as they consider most appropriate. By EQUALITY we mean narrowing the gap in wealth and power between the rich and the poor. By ECONOMIC SECURITY we mean the gurarantee of a steady job and a decent income. By MORALITY we mean people living according to the rules that constitute decent human behavior. By SOCIAL ORDER we mean being able to live in an orderly and peaceful society where the laws are respected and enforced. All five of these ideas are important, but sometimes we have to choose between what is more important and what is less important. And, the specific choices we make sometimes depend upon the comparisons we have to make. On the next few screens, we will show you these ideas in sets of three. For each set, please use the mouse to indicate the idea that you think is most important of the three, and also the idea that you think is least important of the three. In some cases, you might think all three of the ideas are very important, but please try to indicate the ones you think are most and least important if you had to choose between them. If you absolutely cannot decide which ideas are most or least important, then just click the box at the bottom of the screen, and move on to the next set of three ideas. 24

B. Value triads. Each set of three values will appear on a separate screen. Note that the order of the triads, as well as the order of the three values within each triad, will be varied randomly. The MOST The LEAST Q Q Liberty Q Q Equality Q Q Economic Security The MOST The LEAST Q Q Equality Q Q Morality Q Q Social Order The MOST The LEAST Q Q Economic Security Q Q Morality Q Q Social Order 25

The MOST The LEAST Q Q Liberty Q Q Economic Security Q Q Morality The MOST The LEAST Q Q Liberty Q Q Morality Q Q Equality The MOST The LEAST Q Q Social Order Q Q Equality Q Q Economic Security 26

The MOST The LEAST Q Q Equality Q Q Social Order Q Q Liberty The MOST The LEAST Q Q Morality Q Q Economic Security Q Q Equality The MOST The LEAST Q Q Economic Security Q Q Social Order Q Q Liberty 27

The MOST The LEAST Q Q Liberty Q Q Morality Q Q Social Order 28

Table 1: Consistency in pairwise value choices. Number of times the first value is chosen over the second value across the three replications Zero One Two Three Percentage making consistent choices across all three replications Liberty versus equality 25.12% 11.56% 15.72% 47.61% 72.73% Liberty versus economic security 38.67% 12.02% 12.33% 36.98% 75.65% Liberty versus morality 41.91% 10.02% 10.63% 37.44% 79.35% Liberty versus social order 21.57% 10.48% 11.40% 56.55% 78.12% Equality versus economic security 52.39% 14.64% 10.02% 22.96% 75.35% Equality versus morality 56.09 10.02% 19.57% 14.33% 70.42% Equality versus social order 33.13% 12.79% 12.63% 41.45% 74.58% Economic security versus morality 44.68% 7.24% 12.48% 35.59% 80.27% Economic security versus social order 22.03% 11.71% 15.87% 50.39% 72.42% Morality versus social order 25.27% 8.32% 11.25% 55.16% 80.43% Note: In each row, the entries in the four leftmost columns indicate the percentage of respondents that selected the first-mentioned value zero, one, two, or three times across the three replications of the pairwise choice. The rightmost column gives the percentage of respondents that made fully consistent choices across all three replications. Sample size is 649.

Table 2: The distribution of importance scores for each value. Value Liberty Equality Economic Security Morality Social Order Zero: 14.48% 23.88% 9.09% 16.18% 30.20% Importance Score One: 15.10% 27.43% 20.80% 15.41% 25.89% Two: 20.96% 23.57% 22.65% 16.80% 19.72% Three: 26.19% 17.41% 23.88% 19.11% 16.80% Four: 23.27% 7.70% 23.57% 32.51% 7.40% Note: Cell entries give the percentage of respondents with each importance rating for each value. The number of observations is 649.

Table 3: Transitivity in pairwise value choices. Value triad Percentage whose choices across value pairs are intransitive Any intransitive choices across all ten value triads 12.48% Liberty, equality, economic security 2.47% Liberty, equality, morality 3.39% Liberty, equality, social order 1.85% Liberty, economic security, morality 0.77% Liberty, economic security, social order 2.16% Liberty, morality, social order 1.08% Equality, economic security, morality 2.16% Equality, economic security, social order 2.77% Equality, morality, social order 1.08% Economic security, morality, social order 1.69% Note: The entry in the first row shows the percentage of respondents with any intransitivity on any of the subsets of three values (i.e., triads). The entries in the remaining rows of the table give the percentage of respondents whose choices on that specific triad are intransitive. All percentages are based upon 649 observations.

Table 4: The impact of value priming on pairwise choices between values. Maximum likelihood logistic regression coefficients (standard errors in parentheses) Value choice: Prime for first value in pair Prime for second value in pair Pseudo R 2 Liberty over equality -0.068 (0.196) -0.086 (0.220) 0.000 Liberty over economic security 0.019 (0.187) 0.093 (0.242) 0.000 Liberty over morality 0.272 (0.187) -0.188 (0.250) 0.004 Liberty over social order -0.082 (0.203) -0.220 (0.223) 0.001 Equality over economic security 0.466* (0.215) 0.404 (0.249) 0.007 Equality over morality 0.340 (0.215) 0.235 (0.255) 0.004 Equality over social order 0.589* (0.220) -0.159 (0.209) 0.010 Economic security over morality -0.308 (0.242) -0.014 (0.244) 0.002 Economic security over social order -0.053 (0.254) -0.154 (0.216) 0.001 Morality over social order -0.133 (0.256) -0.083 (0.218) 0.000 Note: Table entries are maximum likelihood coefficients for a logistic regression equation predicting the value choice listed in the leftmost column. The number of observations is 649 in each equation. From the left, the first coefficient in each row is for a dummy variable indicating that the first-mentioned value is primed. The second coefficient is for a dummy variable indicating that the second-mentioned value is primed. According to the value priming hypothesis, the first coefficient should be positive, and the second coefficient should be negative. Coefficients marked with an asterisk are significantly different from zero at the 0.05 level in a directional test.

Figure 1: Overall Structure of Proposed Study EXPERIMENTAL MANIPULATION (Eight Conditions, Two Items in Each) Five conditions, each priming one specific value Two conditions, each priming a conflict between two specific values One control group receives no value priming DISTRACTION TASK (All Respondents, One Item) Liberal-conservative self-placement INSTRUCTIONS FOR TRIADS (All Respondents) One screen, providing a one-sentence definition for each value, and instructions for responding to the triad items VALUE TRIADS (All Respondents, Ten Items) Ten screens, each of which displays three values. Respondents select the most- and least-important value on each screen

Figure 2: Mean value importance scores. Morality Economic Security Liberty Equality Social Order 0 1 2 3 4 Mean importance score Note: Points in the dotplot represent mean importance scores (calculated across 649 internet survey respondents) for each value. The error bars around each point represent the 95% confidence interval for the mean.

Figure 3: A simple example showing how triads are used to obtain a full preference order (and, in so doing, test for transitivity). The example assumes only four values (A, B, C, and D) and follows a single, hypothetical, person s choices. A. Triads: All possible subsets of three values (i.e., triads ) are presented to the respondent. For each triad, the most-preferred value is underlined and the least-preferred value is crossed out. A B C A B D A C D B C D B. Dominant Pairwise Choices: With four values, each value pair occurs in two different triads (e.g., A and B appear in the first two triads). For all six possible value pairs, examine the choices made between them. B is chosen over A twice (in the first two triads), hence B dominates A C is chosen over A twice (in the first and third triads), hence C dominates A A is chosen over D twice (in the second and third triads), hence A dominates D B is chosen over C twice (in the first and fourth triads), hence B dominates C B is chosen over D twice (in the second and fourth triads), hence B dominates D C is chosen over D twice (in the third and fourth triads), hence C dominates D Based upon the preceding results, the hypothetical respondent is fully consistent in his/her pairwise choices (i.e., identical choices occur on both comparisons for each value pair). C. Obtain the Preference Order from the Dominant Choices: Count the number of times each value dominates another value. The resultant numeric scores give the preference order for the respondent. A dominates one other value (D), so it is assigned a 1 B dominates three other values (A, C, and D), so it is assigned a 3 C dominates two values (A and D), so it is assigned a 2 D dominates no other values, so it is assigned a 0 For this respondent, the preference ordering for the values (based upon the dominant pairwise choices) is B (most preferred), C, A, D (least preferred). If the scores assigned to the values are fully-ordered (as they are in this example), then all of the choices are transitive and a complete preference ordering for the values can be constructed. If there are any ties in the scores, then at least some of the pairwise choices are intransitive, and a full ranking of the values is impossible.