All data and do-files for the analyses contained in this study will be available online.

Transcription

1 Sorting the News: How Ranking by Popularity Polarizes Our Politics Yotam Shmargad Assistant Professor School of Government & Public Policy University of Arizona Samara Klar Associate Professor School of Government & Public Policy University of Arizona Abstract: Most Americans receive their political news from social media platforms. These platforms algorithmically determine the order in which news is presented, usually relying an article s popularity (i.e. number of likes, shares, and comments) to determine its ranking. To what extent does sorting the news by popularity influence people s attitudes toward politics and toward each other? With two large, nationally representative samples of adults and a novel experimental design, we find that ranking news articles by their popularity leads to heightened perceptions of media bias and increased reluctance to learn about and express political views on social media. With behavioral measures, we find that sorting by popularity exacerbates the tendency to like news confirming one s own viewpoint. Overall, our study holds important implications for how the curation of news on social media can change and even polarize our political preferences. Introduction 1 Media environments can influence how citizens engage with politics. As Prior explains, political science tends to treat ordinary people s political behavior as if it can be explained without reference to the media environment in which they live... yet as the environment changes, so might the behavior (Prior 2007, p. 3-4). It is impossible to consider today s media environment without thinking primarily of social media. According to recent Pew polling, 67 percent of adults now receive at least a portion of political news from social media among them, 45 percent rely exclusively on Facebook (Matsa and Shearer 2018). The news we see on social media inform our views and determine the relative salience of the issues of the day (Feezell 2017). Social media platforms are distinct from traditional news outlets in many ways. In this paper, we argue that one particularly consequential distinction is the manner in which these platforms re-arrange the order that news appears in real time. News articles, photos, and other content appear in a newsfeed a list of posted items that users can skim at their own pace. On most social media platforms, users can choose whether these items are arranged in the order that they were posted or ranked by popularity. On Facebook, for example, the default setting is called Top Stories. This setting presents the most popular posts from friends at the top of one s newsfeed. Popularity is determined, in part, by one s own social network of contacts. An article that a disproportionate number of one s contacts read and engage with will thus appear higher than an article with less engagement. Alternatively, Facebook users can change 1 All data and do-files for the analyses contained in this study will be available online.

2 the default setting to one called Most Recent Stories, which shows content from friends in the order in which it was posted; a chronological view of the news. The former setting ranking by popularity is the default setting on Facebook as well as on Twitter. Most users thus see articles that are most popular amongst their friends listed near the top of the newsfeed. While most users, regardless of their partisan preference, are typically exposed to the same overall selection of media coverage (Guess 2018), the composition of one s social network influences the order in which news stories appear. We test for the effects of ranking news articles by popularity with two nationally representative surveys and a novel experimental design. Our first survey presents research participants with sixteen news articles obtained from AllSides.com, a website that categorizes news articles according to their ideological leanings. We select 8 liberal-leaning articles and 8 conservative-leaning articles, in order to ensure that both sides of the political spectrum are equally reflected. We then present the sixteen articles to research participants in a newsfeed-like stream, with the order of the articles randomly determined for each participant, and provide a like button for each article that participants can click. We then use the like -counts of each article to create three experimental conditions for our second survey: a feed sorted by likes of Democratic participants, a feed sorted by of likes of Republican participants, and a control feed with articles ordered randomly. Participants of the second survey could then like the articles in their experimentally-manipulated newsfeed in a similar manner as in the first survey. After completing the newsfeed task, participants in the second survey were asked several questions that measure their perceptions of media bias, interest in engaging politically on social media, and affect towards members of their own party and the opposing party. We find that ranking news articles by popularity influenced which articles they chose to like, as well as their responses to the subsequent research questions. In particular, participants assigned to a newsfeed sorted by in-party members were more likely to like in-party articles, while participants in a newsfeed sorted by out-party members were more likely to like out-party articles (compared to those in newsfeeds that were randomly sorted). We also find that, compared to participants in the randomlysorted control newsfeed, those in the newsfeed sorted by out-party members were less likely to say that the articles represented their own views. Given that the same sixteen articles were included in each newsfeed condition, these perceptions of bias imply that the order in which people view has a distinct effect from the ideological bias of the articles presented. Finally, we find evidence that participants in a feed sorted by out-party members report less interest in engaging with politics online, both with respect to learning about politics as well as expressing their own political views on social media. Counter to our hypotheses, we find no significant effects of article ranking on affective polarization. The remainder of the paper is organized as follows. In the next section, we describe previous research on the effect of news article location in traditional media and online. We then build on these findings to formulate our hypotheses about the effect that popularity rankings should have on expressed support for in-party news, perceptions of media bias, as well as political engagement and polarization. In the third section, we discuss the two nationally-representative surveys that we administer and the novel experimental design embedded in the second survey. We describe the results of our experiment in the fourth section, and conclude with a discussion and directions for future work. 2

3 Theory and Background Studies of media curation show that the location of news stories matters. In newspapers, few readers miss the point that long front-page articles with banner headlines are more important than short inside stories (Althaus and Tewksbury 2002, p. 180). Front-page and above-the-fold articles exert an impact denied to stories appearing elsewhere (Clayman and Reisner 1998, p. 178). On television, lead stories are more influential than subsequent ones (Iyengar and Kinder 1987). Online readers are able to discern a story s relative importance to other news items using signals like location, length, headline size and more (Carlson 2007, p. 1016). On social media platforms, location is just as important for how influential a post can become. For example, Bakshy et al. (2015) find that a news article s position on Facebook matters more than ideological congruence; in other words, even when articles do not conform to a one s political views, there are strong tendencies to engage with content that is positioned toward the top of the News Feed (Bakshy et al. 2015, p. 21 supplemental material). What is unique about social media is the path dependence that can develop when certain stories receive more prominent placement. Top posts receive more attention and more opportunities to become more popular, and rankings based on popularity thus lead these stories to receive even more prominent placement. Both observational and experimental studies demonstrate how ranking by popularity can, in this way, become self-fulfilling. For example, as articles on news websites receive more clicks, online editors move them to more central locations on the website (Lee et al. 2014). Muchnik et al. (2013) experimentally manipulate up votes on online comments, and find that comments which were randomly assigned a single vote become 32 percent more likely to receive additional votes. When songs are ranked by popularity, those near the top are rated better and downloaded more often (Salganik et al. 2007; Salganik and Watts 2006). Ranking by popularity thus creates a self-fulfilling prophecy. This can be particularly problematic when it comes to political news, since motivated reasoning is made simpler by easier access to and validation of one s existing views. Popularity rankings inherently depend on the tastes of the group evaluating the information. Among a group of Democrats, popularity rankings should result in more liberal-leaning articles being propelled to the top, increasing the likelihood that they will be read and liked. Conversely, a group of Republicans will tend to prefer like-minded conservative news, helping to ensure that conservative news articles are placed more prominently. This leads to our first two-part hypothesis: H1a: Popularity rankings based on social feedback from in-party members will increase expressed support for like-minded news articles. H1b: Popularity rankings based on social feedback from out-party members will decrease expressed support for like-minded news articles. Rankings of news articles will do more than simply draw attention to stories placed near the top they should influence how people perceive the media environment as a whole. The most prominent stories will frame the manner in which people perceive a newsfeed. Garrett and Stroud (2014) explain that people tend to perceive ideologically like-minded information as unbiased and neutral, whereas the

4 ideologically incongruent information stands out and leads to perceptions of a hostile or biased media environment (Giner-Sorolla and Chaiken 1994). Arceneaux et al. (2012) also find that exposure to ideologically incongruent news induces hostile media perceptions; people become less likely to perceive the news as informative and more likely to view it as unfair. A newsfeed that prioritizes ideologically incongruent information should therefore frame how readers view the newsfeed in its entirety. We thus hypothesize that ranking by popularity will hold implications for perceptions of media bias: H2: Ranking news articles by popularity among a network of out-party members will lead people to believe that the news does not accurately represent their views. Ranking news articles by popularity according to an out-party draws cross-cutting media to the top of a newsfeed, making people more likely to see ideologically incongruent information. Existing work suggests that this type of cross-cutting exposure can disengage voters from politics, for (at least) two reasons. The first relies on ambivalence: exposure to ideologically incongruent information makes people ambivalent about the opinions they once held, and they can become less likely to act on those opinions (Dilliplane 2011). This type of disengagement is largely evident in behaviors like voting or discussing politics (Mutz 2002). Second, exposure to cross-cutting media can also disengage because of cognitive dissonance: the uncomfortable feeling one gets when exposed to information that contradicts their prior beliefs. To avoid this discomfort, a reasonable response is simply to tune out. Empirical work suggests that people are particularly likely to select into like-minded news coverage and away from disagreeable news when the topic at hand is political (Stroud 2008) and when high-speed Internet is available (Lelkes et al. 2015). We thus expect that exposure to a newsfeed that prioritizes disagreeable news coverage will discourage readers from engaging with politics on social media. H3: Ranking news articles by popularity among a network of out-party members makes people less interested in engaging with politics on social media. Finally, we consider how sorting news articles by popularity influences perceptions of people of the other party. Ranking political news articles by popularity dramatically increases the likelihood that people view one perspective over the other. How might exposure to such one-sidedness affect the way people think about Democrats and Republicans? Iyengar et al. (2012) suggest that selective exposure to like-minded news coverage can heighten in-party biases and discrimination against out-party members. Garrett et al. (2014) test this theory by exposing experimental subjects to a series of news articles from sources which either conform to or oppose their own political views (e.g. MSNBC for liberals and Fox News for conservatives are considered pro-party ). Indeed, they find that exposure to pro-party news sites is associated with more affective polarization, whereas exposure to opposing party sites reduces it. We expect that ranking by popularity should thus influence people s attitudes towards in-party and outparty members: 4

5 H4a: Ranking by popularity among a network of in-party members increases affective polarization. H4b: Ranking by popularity among a network of out-party members reduces affective polarization. We summarize our hypotheses in Table 1 below. Table 1: Summary of Hypotheses In comparison to random ranking: H1a: Ranking by popularity of in-party members increases expressed support for in-party articles. H1b: Ranking by popularity of out-party members increases expressed support for out-party articles. H2: Ranking by popularity of out-party members increases perceptions of media bias. H3: Ranking by popularity of out-party members decreases political engagement on social media H4a: Ranking by popularity of in-party members increases affective polarization. H4b: Ranking by popularity of out-party members reduces affective polarization. Procedure How does ranking political news by popularity influence people s evaluations of articles, news media, and partisans overall? To test our hypotheses, we designed a two-phase study: a survey followed by an experiment. The goal of the survey was to infer how news rankings differ depending on the set of people who engage with news articles. We then use rankings obtained from the survey to experimentally manipulate newsfeeds in the second phase of this study. We start by explaining the procedure and results for the first phase before turning to the second phase. (We present all question wording from both the survey and experiment in the Supplementary Appendix.) Phase 1 Although most Americans see roughly the same distribution of news articles (Guess 2018), algorithmic sorting implies that the order in which news appears can differ in systematic ways. Popularity rankings the default setting on Facebook and Twitter sort the news by how popular each article is among one s social network contacts. In the first phase of our study, we chose sixteen articles from the website AllSides.com. This website labels the ideological leaning of existing news articles, assigning each article a score from 1 to 5, where 1 is most liberal and 5 most conservative. 2 We chose eight articles rated as liberal and 8 rated as conservative. Using the Qualtrics survey platform, we 2 For more detail on the AllSides.com ranking methodology, see:

6 created a simulated newsfeed containing all sixteen articles. Below each article was a like button that subjects could click (and could unclick if they chose). 3 We recruited 200 adult participants through Survey Sampling International, a company that recruits survey participants and administers surveys. Each participant first provided demographic data about themselves, including their age, race, gender, and party identification. They then read the following instructions: On the following page, you will see snippets of several news articles. These are all articles that have appeared in the popular press within the last few months. Below each article is a like button the kind you might see on social media. Please take at least 5 or 10 minutes to review the articles that appear. If you like a particular article, please click the like button that appears below it. You can like as few or as many articles as you want. Participants then saw the newsfeed with the aforementioned news articles presented in random order that is to say, the articles were re-randomized for each participant, so that the average number of likes per article was unaffected by the order in which it was presented. Participants scrolled through the newsfeed, liking (or not liking) as many articles as they chose. Once they completed the newsfeed task, the survey was over. Phase 1 allowed us to rank the sixteen news articles by popularity among Democrats and among Republicans, just as an algorithm might do. Indeed, we found stark differences between Democrats and Republicans preferences for the different articles. For 10 of the 16 articles, the participant s partisanship was highly significant (p < 0.01) in predicting whether or not the respondent clicked like or not. For 2 articles, significance was marginal (hovering at around p=0.10), and there was no significant correlation between partisanship and liking 4 of the 16 articles. Ranking these articles by popularity thus produces different orderings depending on whether the sample of rankers were Democrats or Republicans. We use these different rankings as experimental treatments in the phase 2 of this study, to which we now turn. Phase 2 In the second phase of the study, we collect the data that allow us to test our hypotheses. Once again, we recruited research participants via Survey Sampling International this time a new sample of 1,200 adults and asked them to provide demographic data. They were then instructed to scroll through a newsfeed and to like as many or as few articles as they chose. We randomly assigned respondents to one of three different newsfeeds. All three newsfeeds contained the same set of sixteen articles, but differed in how the articles were sorted. One third of the participants saw the articles sorted according to their popularity among Democrats in phase 1. A second third saw the articles sorted according their popularity among Republicans in phase 1. The final third saw the articles simply presented in a random order that varied across participants in this condition. Beneath each article, the participants could also see the number of likes that the article received by participants in phase 1. Articles presented in the 3 The interactive newsfeeds will all be available at the authors websites links not provided here to preserve anonymity during the review process. 6

7 random condition each showed the number of likes they received by a random bipartisan sample. Figure 1 displays the top article in the Democratic-ranked newsfeed, the Republican-ranked newsfeed, and one of the articles that was included in the randomized newsfeed. Top article in Democraticranked newsfeed Top article in Republicanranked newsfeed Figure 1. Sample articles from newsfeed conditions Sample article from randomized newsfeed In this second experimental phase, participants read the following instructions before seeing the newsfeed: On the following page, you will see snippets of several news articles. These are all articles that have appeared in the popular press within the last few months. Below each article is a "like" button - the kind you might see on social media. You can also see the number of times the article was liked by respondents who previously took this survey. If you like a particular article, please click the "like" button that appears below it. This experimental design allows us to compare participants reactions to newsfeed that are sorted by popularity among in-party members, popularity among in-party members, and a random ranking. We measure our dependent variables with behavioral actions (i.e. liking articles) as well as self-reported survey questions. As we describe our empirical findings, we will explain how we operationalize each dependent variable. Results Before turning to our hypotheses, we first provide descriptive data to illustrate how participants behaved in the different newsfeeds. Again, each newsfeed contained an identical set of sixteen articles, randomized into different orderings depending on the condition to which participants were randomly assigned. The average participant liked just three articles. Thirty-six percent of our participants did not like any articles, and just 1% liked all 16 articles. Overall, there was no difference in the total number of likes across experimental condition. We also examine the importance of where an article appears in

8 the newsfeed across the three conditions. Recall that respondents were assigned to a condition in which the same sixteen articles were either sorted by popularity among Democrats, sorted by popularity among Republicans, or appeared in random order (where an article s popularity plays no role in where it was located). In Figure 2, we display the proportion of participants, across conditions, that liked the article in each of the sixteen positions in their newsfeed. Figure 2: Proportion of Participants who liked an Article by Condition, Position, and Party ID We display Democratic participants in blue (left panel) and Republican participants in red (right panel). 4 For each group, the leftmost column displays participants behavior in the newsfeed that aligns with their own partisanship (i.e. Democrats assigned to articles ranked by Democrats and Republicans assigned to articles ranked by Republicans). The middle column displays behavior in the random ordered newsfeed. The rightmost column displays behavior in the newsfeed where articles were ranked by the out-party (i.e. Democrats assigned to articles ranked by Republicans and Republicans assigned to articles ranked by Democrats). The y-axis illustrates the position of each article in the newsfeed, with the top-ranked articles at the top of the figure and the lowest-ranked articles at the very bottom. Darker shades indicate that a higher proportion of participants (within each condition) liked the article in that position. Lighter shades indicate that a lower proportion of participants liked the article. 4 Colors can be switched to gray-scale for publication. 8

9 We begin by focusing on how Democrats and Republicans behave in the random feed condition (i.e. the control group). Without considering the content of the articles themselves, we see a slight tendency for respondents to like articles that appear higher in the newsfeed (i.e. toward the top of the figure). Logit models confirm that, in the control group, the randomized position of an article influences whether or not it is liked. 5 Turning to the in-party condition (i.e. the left-most column in each panel), where like-minded articles cluster toward the top of the newsfeed and ideologically incongruent articles toward the bottom, we see that preferences for news at the top of the newsfeed becomes much stronger. This intuitively makes sense the top-ranked articles are both more visually prominent and also align with the respondent s own views, so that participants are likely to concentrate their likes near the top. Finally, we depict the out-party condition in the rightmost columns of both panels. Here, like-minded articles cluster toward the bottom of the newsfeed, with ideologically incongruent articles clustered near the top. Participants now tend to like articles closer to the bottom of their newsfeed more than either of the previous conditions, while still liking articles near the top with some regularity (despite their contradictory content). This figure demonstrates how the same sixteen articles garner dramatically different patterns of engagement depending on how they are arranged. Once we focus on the content of the articles themselves, we start to see differences in how Democratic respondents and Republican respondents in the random control group engage with the articles. Without considering the order of articles, we find that that, to no surprise, Democrats are more likely to like Democratic articles than are Republicans and vice versa. On average, Democrats like two Democrat-leaning articles and just fewer than one Republican-leaning article. Similarly, on average, Republicans like two Republican-leaning articles and one Democrat-leaning article. The tendency to like an article thus depends on whether it aligns with one s own views. Next, we turn to formal tests of our hypotheses by comparing key dependent variables across experimental conditions. Our study is designed to test how ranking news articles by popularity can influence people s online behavior and views about media and politics. Our first hypothesis states that ranking by popularity will exacerbate liking of articles near the top, even when these articles are not in line with people s viewpoints. Although we saw that, overall, people prefer articles that lean toward their own views, we base our first hypothesis on Bakshy et al. s (2015) observation that Facebook users appear to give greater attention to articles that are higher up on the newsfeed, regardless of their ideological congruence. We expected that ranking news articles by popularity of in-party members will increase expressed support for the preferences of the in-party, while articles ranked by popularity of out-party members with increase expressed support for the preferences of the out-party. For each group of participants (those in the in-party newsfeed, the out-party newsfeed, and the control group), we subtract the mean number of out-party likes from the mean number of in-party likes to create a measure of in-party preference. For example, if a Democratic participant liked 5 Democrat-leaning articles and 2 Republican-leaning articles, their in-party preference would be 3 (i.e. 5 minus 2). In line with Hypothesis 1, we expect that viewing the sixteen articles ranked by the in-party popularity should increase expressed bias in support of in-party articles. To compare the mean in-party bias score across the three groups, we simply regress the in-party bias score on dummy variablea for the in-party popularity ranking treatment and out-party popularity ranking treatment using the random 5 Results of this analysis are available from the authors upon request.

10 ranking control group as a baseline. Figure 3 displays the coefficients for each condition relative to the random control group. All analyses in this paper include two-tailed tests of significance. Figure 3: Effect of Ranking by Popularity on Preference for In-Party News Articles The results indicate that ranking political news by popularity of in-party members does indeed hold consequences for expressed support for the articles. When news articles are ranked by popularity among an agreeable social network (i.e. a network of co-partisans), like-minded articles are pushed toward the top and participants are significantly (β = 0.49; p < 0.01) more likely to like in-party articles than participants in the control group. On the other hand, when articles are ranked by popularity according to out-party members, participants reduce their bias for in-party articles compared to the control group, though the difference is marginally significant (β = -0.25; p = 0.07). These results support our first hypothesis: ranking the news by popularity changes the way people engage with news articles. This comports with what scholars have previously found in non-political settings: popularity rankings induce a self-fulfilling prophecy, whereby content at the top of a newsfeed stays at the top by getting more engagement (e.g. Salganik et al. 2007; Salganik and Watts 2006; Muchnik et al. 2013). Our second hypothesis states that popularity rankings will influence perceptions of the range of perspectives represented in the social media newsfeed. Recall that each experimental group sees the same selection of sixteen news articles. The only difference across conditions is the order in which these articles appear. To measure perceptions of bias, we asked participants the following question after they completed the newsfeed task: How well did the articles in the newsfeed represent your own political views? A response of 1 indicates very well, implying low media bias, while a response of 7 indicates very poorly, suggesting higher media bias. Our expectation, as stated in Hypothesis 2 above, is that when the articles are arranged by popularity of out-party members, participants will perceive that the selection of articles is less representative of their own views (i.e. is more biased). To test this, we regress the responses on dummy variables for the in-party and out-party popularity ranking treatments, again 10

11 using the control group as a baseline. Figure 4 displays the coefficients for each treatment relative to the control group. Negative coefficients indicate a perception of greater representativeness (i.e. less bias) while positive coefficients indicate a perception of poorer representativeness (i.e. more bias). Figure 4: Effect of Ranking by Popularity on Perceptions of Media Bias Our results confirm that ranking news articles by popularity holds significant consequences for perceptions of media bias. Compared to seeing the articles in random order, ranking by out-party popularity significantly increases participants perception that the selection of articles did not represent their views (β = 0.46; p < 0.01). Participants assigned to newsfeeds with articles sorted by out-party popularity thus perceived greater bias, even though they were exposed to the same set of sixteen articles as those assigned to the control group. We did not expect that ranking by in-party popularity would increase the perception of representativeness relative to the control group, since people tend to view news that conforms to their perspective as representative and neutral (Garrett and Stroud 2014); the coefficient in this case was negative (β = -0.17), indicating a move toward perceiving less bias, but does not reach the threshold of statistical significance (p = 0.16). We next examine the influence of popularity rankings on political engagement. We expect that ranking news articles by out-party popularity will make cross-cutting news more salient and, due to ambivalence (Mutz 2002; Dilliplane 2011) and cognitive dissonance (Stroud 2008; Lelkes et al. 2015), discourage participants from engaging with politics on social media. We test this hypothesis with two survey questions posed at the end of our study: How interested are you in using social media to learn about politics? and How interested are you in using social media to express your political views? In analyzing responses to these questions, we reverse-coded the response scale so that 1 indicates very uninterested and 7 indicates very interested. We again regress responses on dummy variables for the in-party and out-party popularity ranking treatments with the control group serving as the baseline. In

12 Figure 5, we depict responses to the question regarding participants interest in using social media to learn about politics. Figure 5: Effect of Ranking by Popularity on Interest in Learning About Politics Our first set of results support our hypothesis, and indicate that ranking news articles by outparty popularity, compared to random order, has a disengaging effect. Participants become significantly less likely to express interest in using social media to learn about politics (β = -0.33; p < 0.05). On the other hand, ranking the articles by popularity of in-party members has no effect when compared to random order (β = -0.09; p = 0.54). We find nearly identical results when it comes to participants desire to use social media to express their views. We display the results to this second question in Figure 6. Again, we find that ranking news articles by popularity of out-party members has a disengaging effect. Compared to the control group, participants are less interested in expressing their political views on social media when exposed to articles ranked by out-party popularity (β = -0.36; p < 0.05). In line with our first measure of political engagement (i.e. interest in learning about politics), the difference in political expression between those exposed to articles sorted by in-party popularity versus the control group is not statistically significant (β = -0.22; p = 0.15). Overall, these two measures confirm that ranking the news by popularity of out-party members discourages people from engaging with politics on social media. 12

13 Figure 6: Effect of Ranking by Popularity on Interest in Political Expression on Social Media Our final hypothesis addresses the influence of ranking by popularity on affective polarization. Given existing evidence that exposure to partisan news coverage can exacerbate in-party biases, we expect the ranking news articles by in-party popularity will influence affective polarization. As others have done (e.g. Iyengar and Westwood 2014; Klar, Krupnikov, and Ryan 2018), we measure affective polarization by asking participants to rate their feelings toward people who support the Democratic Party and Republican Party, respectively, on a scale from 0 to 10, where 0 indicates extremely unfavorable and 10 indicates extremely favorable. We follow previous work and calculate affective polarization by subtracting a participant s out-party rating from their in-party rating. The mean affective polarization in our sample is 4.58 and To test whether ranking by popularity resulted in changes in affective polarization, we regress these scores on dummy variables for the in-party and out-party popularity ranking treatments, with the control group serving as the baseline. Figure 7 displays our results. We do not find evidence that sorting news articles by popularity increases affective polarization. While ranking articles by out-party popularity increased affective polarization compared to the control group, the difference was not statistically significant (β = 0.31; p = 0.23). Ranking articles by in-party popularity slightly decreases affective polarization compared to the control group, but this difference was not statistically significant either (β = -0.03; p = 0.88). One important distinction between our study and previous studies that have investigated how media exposure influences affective polarization is that our participants all see the same selection of news articles across conditions no participant sees only like-minded or only cross-cutting coverage. We find that the mere ordering of the same news articles influences expressed support for certain articles (H1), perceptions of media representativeness (H2), and future online political participation (H3). It does not, however, appear to increase the already-large gap in affect between one s fellow partisans and those who support the other party.

14 Figure 7: Effect of Ranking by Popularity on Affective Polarization Discussion A majority of Americans report that they receive their political news, at least in part, and often in large part, from social media platforms such as Facebook. The manner in which content is presented on social media is wholly unique no other type of media platform updates how news items are sorted and presented, quickly and dynamically, in response to feedback from people s own social network contacts. In this paper, we argue and show that such popularity-based sorting of new articles has several important consequences for political media consumers. Ranking the news by popularity can influence the extent that we express support for news, perceive media bias, and engage politically online. For people who are surrounded primarily by like-minded peers, popularity-based sorting algorithms can exacerbate the asymmetric support that their own viewpoints already receive. For those surrounded primarily by outparty members, expressed support for the other party s views may increase, but this comes at a cost of heightened perceptions of ideological media bias and suppression of political engagement. Although social media platforms are often treated as ideologically like-minded echo chambers, there does exist considerable variation in the degree to which people connect to in-party and out-party members. In our own previous research, we surveyed a sample 277 undergraduate students at a large public university in the fall of 2017 (Authors Name Redacted). Among them, just under half reported that their social media networks are primarily composed of like-minded partisans. Meanwhile, slightly over half reported belonging to ideologically diverse social networks. A non-trivial 18% reported that their social media networks are primarily composed of out-party members, and 32%reported that their social networks are about evenly split between Democrats and Republicans. Guess and colleagues (forthcoming) find a high degree of accuracy in how people portray their own social media networks, giving us confidence that these numbers are reliable. They suggest that Americans are often not confined to social media echo chambers but instead are embedded in a variety of networks some ideologically 14

15 like-minded and others not at all. Indeed, Kamin (2016) and Guess et al. (2018) both report high degrees of cross-cutting media exposure online. Our results suggest that the composition of our social networks can hold important consequences for political news consumptions for a previously unexplored reason: news articles on social media most often appears in an order determined by their popularity among one s contacts. Our friends and virtual neighbors will shape which political viewpoints make it to the top of our feeds. This will perpetuate the bias in news articles that we engage with and endorse, further exacerbating the asymmetry in the feeds of our contacts. To our knowledge, our study is the first to examine how such popularity-based sorting of political information influences political evaluations and expressions, and attitudes about media more generally. We hope that this study provides a useful starting point for scholars who wish to build on the quickly expanding literature regarding how social media shape political attitudes. Of course, the simulated newsfeeds in our experiment do not perfectly recreate the social media environments that we see in the real world. Participants in our study did not personally know or form connections to each other, and could not see specifically who shared or liked each item in the newsfeed. These social cues could certainly influence how we engage with political content that we see online, and scholars are indeed finding that social endorsements increase engagement with online content (Messing and Westwood 2014). In addition, our simulated environment provides no opportunity for commenting, which is another feature that could shape how people read, evaluate, and engage with the news they see on social media. Indeed, recent work demonstrates, for example, that the order in which people read comments (Berry and Taylor 2017) and the tone of the comments themselves (Bakalar and Arceneaux 2018) and can influence how people engage online. Given that social media is an increasingly dominant way that people get their news, future research should investigate how other features of social media beyond popularity-based rankings shape political behaviors and attitudes. References Althaus, Scott L., and David Tewksbury. (2002). Agenda Setting and the New News: Patterns of Issue Importance Among Readers of the Paper and Online Versions of the New York Times. Communication Research 29:180. Arceneaux, Kevin, Martin Johnson, and Chad Murphy. (2012). Polarized Political Communication, Oppositional Media Hostility, and Selective Exposure. The Journal of Politics 74(1): Bakalar, Chloé, and Kevin Arceneaux. (2018). Does Social Media Undercut Deliberation? Paper presented at the Annual Meeting of the Midwestern Political Science Association. Bakshy, Eytan, Solomon Messing, and Lada A. Adamic. (2015). Exposure to ideologically diverse news and opinion on Facebook. Science 348 (6239): Berry, George, and Sean J. Taylor. (2017). Discussion Quality Diffuses in the Digital Public Square. Proceeding of the 2017 International World Wide Web Conference, April 3-7, Perth, Australia. Carlson, Matt. (2007). Order versus access: news search engines and the challenge to traditional journalistic roles. Media Culture Society 29(6):

16 Clayman, Steven E., and Ann Reisner. (1998). Gatekeeping in Action: Editorial Conferences and Assessments of Newsworthiness. American Sociological Review 1998, 63: Dilliplane, Susanna. (2011). All the News You Want to Hear: The Impact of Partisan News Exposure on Political Participation. Public Opinion Quarterly 75(2): Feezell, Jessica T. (2017). Agenda-Setting Through Social Media: The Importance of Incidental News Exposure and Social Filtering in the Digital Era. Political Research Quarterly, 71(2) Garrett, R. Kelly, and Natalie J. Stroud. (2014). Partisan Paths to Exposure Diversity: Differences in Pro- and Counterattitudinal News Consumption. Journal of Communication 64(4): Garrett, R. Kelly, ShiraDvir Gvirsman, Benamin K. Johnson, Yariv Tsfati, Rachel Neo, and Aysenur Dal. (2014). Implications of Pro- and Counterattitudinal Information Exposure for Affective Polarization. Human Communication Research 40: Giner-Sorolla, Roger, and Shelly Chaiken. (1994). The Causes of Hostile Media Judgments. Journal of Experimental Social Psychology 30(2): Guess, Andrew. (2018). (Almost) Everything in Moderation: New Evidence on Americans Online Media Diets. New York University. Guess, Andrew, Benjamin Lyons, Brendan Nyhan, and Jason Reifler. (2018). Avoiding the Echo Chamber About Echo Chambers: Why Selective Exposure to Like-Minded Political News is Less Prevalent than You Think. Miami: John S. and James L. Knight Foundation. Guess, Andrew, Kevin Munger, Jonathan Nagler, and Joshua Tucker. (Forthcoming). How Accurate are Survey Responses on Social Media and Politics? Political Communication. Iyengar, Shanto, and Donald R. Kinder. (1987). News That Matters. Chicago, IL: University of Chicago Press. Iyengar, Shanto, and Sean Westwood. (2014). Fear and Loathing Across Party Lines: New Evidence on Group Polarization. American Journal of Political Science 59(3): Iyengar, Shanto, Gaurav Sood, and Yphtech Lelkes. (2012). Affect, Not Ideology: A Social Identity Perspective on Polarization. Public Opinion Quarterly 76(3): Kamin, Julia. (2016). Cracks in the Echo Chamber. Manuscript presented at the 2016 Annual Meeting of the American Political Science Conference. Available: Klar, Samara, Yanna Krupnikov, and John Barry Ryan. (2018). Affective Polarization or Partisan Disdain?: Untangling a Dislike for the Opposing Party from a Dislike of Partisanship. Public Opinion Quarterly 82(2): Lee, Angela M., Seth C., Lewis, and Matthew Powers. (2014). Audience Clicks and News Placement: A Study of Time-Lagged Influence in Online Journalism. Communication Research 41(4):

17 Lelkes, Yphtach, Gaurav Sood, and Shanto Iyengar. (2017). The Hostile Audience: The Effect of Access to Broadband Internet in Partisan Affect. American Journal of Political Science 61(1): Matsa, Katerina Eva, and Elisa Shearer. (2018). News Use Across Social Media Platforms Pew Research Center Report. September 10, Messing, Solomon, and Sean J. Westwood. (2014). Selective Exposure in the Age of Social Media. Communication Research 41(8): Muchnik, Lev, Sinan Aral, and Sean J. Taylor. (2013). Social Influence Bias: A Randomized Experiment. Science 341: Mutz, Diana C. (2002). The Consequences of Cross-Cutting Networks for Political Participation. American Journal of Political Science 46(4): Prior, Markus. (2007). Post-Broadcast Democracy: How Media Choice Increases Inequality in Political Involvement and Polarizes Elections. New York, NY: Cambridge University Press. Salganik, Matthew J., Peter Sheridan Dodds, and Duncan J. Watts. (2007). Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market Science 311: Salganik, Matthew J., and Duncan J. Watts. (2006). Leading the Herd Astray: An Experimental Study of Self-Fulfilling Prophecies in an Artificial Cultural Market. Social Psychology Quarterly 71(4): Stroud, Natalie J. (2008). Media Use and Political Predispositions: Revisiting the Concept of Selective Exposure. Political Behavior 30: Supplementary Appendix This appendix includes the full question wording for the survey and experiment in Sorting the News. Phase 1: Survey Full Question Wording Thank you for participating in this study! On the following page, you will see snippets of several news articles. These are all articles that have appeared in the popular press within the last few months. Below each article is a "like" button - the kind you might see on social media. Please take at least 5 or 10 minutes to review the articles that appear. If you "like" a particular article,

18 please click the "like" button that appears below it. You can "like" as few or as many articles as you want. [NEWSFEED HERE] Before you leave, please tell us just a bit about yourself. How often do you use social media? Never A few times a month A few times a week Every day What is your gender? Male Female Other With which race do you primarily identify? (You may select more than one.) African American of Black Caucasian or White Hispanic or Latino Asian American of Pacific Islander Native American Other What is your age? Generally speaking, are you a Democrat, a Republican, or an Independent? Strong Democrat Weak Democrat Independent leaning to Democrat Independent Independent leaning to Republican Weak Republican Strong Republican When it comes to your political views, would you say you are liberal, conservative, or moderate? Very liberal Liberal Slightly liberal Moderate Slightly conservative 18

19 Conservative Very conservative Thank you! Those are all the questions we have for you today! Phase 2: Experiment Full Question Wording Thank you for participating in this study! Before you begin, we d like to ask a few things about you, just so we can get an idea of the types of people that filled out this survey. Please remember that all of your responses are anonymous and we will not collect any information that could allow us to identify you. What is your gender? Male Female Other What is your age? or more With which race do you primarily identify? (You may select more than one.) African American or Black White Hispanic or Latino Asian American or Pacific Islander Native American Other Generally speaking, are you a Democrat, a Republican, or an Independent? Strong Democrat Weak Democrat Independent Leaning to Democrat Independent Independent Leaning to Republican Weak Republican Strong Republican When it comes to your political views, would you say you are liberal, conservative, or moderate? Very Liberal Liberal

20 Slightly Liberal Moderate Slightly Conservative Conservative Very Conservative What is the highest level of education you have completed? High school Some college College degree Graduate degree How often do you use social media (sites like Facebook, Twitter, Instagram, and so on)? Every day A few times a week A few times a month A few times a year Never Great! Now let's get started. On the following page, you will see a newsfeed of recent articles. This newsfeed will look a bit like a newsfeed you might see on Facebook or Twitter --- it's a long series of articles for you to review as you scroll down the page. On your newsfeed, you will see a series of articles that have appeared in the popular press within the last year. Below each article is a "like" button - the kind you might see on social media. You can also see the number of times the article was liked by respondents who previously took this survey. If you like a particular article, please click the "like" button that appears below it. Please take as much time as you need to scroll through the articles that appear in the newsfeed. You can "like" as many articles as you want. Once you are done, you can click the blue arrow at the bottom of the page to continue. Please take as much time as you need to scroll through the articles that appear in this newsfeed. If you like a particular article, please click the "like" button that appears below it. You can "like" as many articles as you want. Once you are done, you can click the blue arrow at the bottom of the page to continue. [NEWSFEED HERE] How well did the articles in the newsfeed represent your own political views? Very well Moderately well Slightly well 20

21 Neither well nor poorly Slightly poorly Moderately poorly Very poorly How interested are you in using social media to learn about politics? Very interested Moderately interested Slightly interested Neither interested nor uninterested Slightly uninterested Moderately uninterested Very uninterested How interested are you in using social media to express your political views? Very interested Moderately interested Slightly interested Neither interested nor uninterested Slightly uninterested Moderately uninterested Very uninterested Generally speaking, how do you feel toward individuals who support the Republican Party on a scale of 0 to 10, where 0 means "Extremely Unfavorable" and 10 means "Extremely Favorable"? Please slide the widget along the scale to indicate the number that is right for you. Generally speaking, how do you feel toward individuals who support the Democratic Party on a scale of 0 to 10, where 0 means "Extremely Unfavorable" and 10 means "Extremely Favorable"? Please slide the widget along the scale to indicate the number that is right for you. Thank you! Your participation is complete.