MONOTONICITY FAILURE IN IRV ELECTIONS WITH THREE CANDIDATES: CLOSENESS MATTERS

Transcription

1 MONOTONICITY FAILURE IN IRV ELECTIONS WITH THREE CANDIDATES: CLOSENESS MATTERS Nicholas R. Miller Department of Political Science University of Maryland Baltimore County (UMBC) Baltimore, Maryland February 2012 Revised August 2013 Second revision January 2016 Abstract A striking attribute of Instant Runoff Voting (IRV) is that it is subject to monotonicity failure that is, getting more (first preference) votes can cause a candidate to lose an election and getting fewer votes can cause a candidate to win. Proponents of IRV have argued that monotonicity failure, while a mathematical possibility, is highly unlikely to occur in practice. This paper specifies the precise conditions under which this phenomenon arises in three-candidate elections, and then applies them to a number of large data sets in order to get a sense of the likelihood of IRV s monotonicity problem in varying circumstances. The basic finding is that the monotonicity problem is significant in many circumstances and very substantial whenever IRV elections are closely contested by three candidates. An earlier version of this paper was presented at the Second World Congress of the Public Choice Societies, Miami, March 8-11, This revision uses more extensive simulations, replaces tables with graphs, and corrects several errors, but it entails no substantial changes in results. The conference version of the paper has been cited in several subsequent works, which are in turn cited here. I thank Robert Z. Norman and Dan Felsenthal for helpful comments.

2 MONOTONICITY FAILURE IN IRV ELECTIONS WITH THREE CANDIDATES 1. Introduction It is generally recognized that ordinary Plurality Voting (or First-Past-the-Post) is problematic in elections with three or more candidates. Instant Runoff Voting (IRV) also known as the Alternative Vote, the Hare Rule, and Ranked Choice Voting is often proposed as an alternative voting rule. Under IRV, voters rank the candidates in order of preference. If one candidate has a majority of first preferences, that candidate is elected. Otherwise, the candidate with the fewest first preferences is eliminated and his or her ballots are transferred to other candidates on the basis of second preferences. This process is repeated until one candidate is supported by a majority of ballots and is elected. Here we consider only three-candidate contests, so IRV is limited to a single instant runoff and is formally equivalent to ordinary Plurality Voting with a runoff between the top two candidates in the event the leading candidate is not supported by a majority of votes. A striking feature of IRV is that getting more (first preference) votes can cause a candidate to lose an election (the more is less paradox ) and getting fewer votes can cause a candidate to win (the less is more paradox ). Voting rules that are never exhibit this anomaly are said to be monotonic. It might be thought that every voting system in actual use is monotonic and in fact most voting systems, including Plurality Voting, are. But many years ago Smith (1973) showed that so-called point-runoff systems that incorporate (actual or instant ) runoffs are non-monotonic put otherwise, that they are subject to monotonicity failure. Several years later, Doron and Kronick (1977; also see Straffin, 1980, p. 24) observed that Smith s class of point-runoff systems includes the Single Transferable Vote (STV), the single-winner variant of which is IRV, which is therefore subject to monotonicity failure. This finding attracted some attention among political scientists (especially Riker, 1982, pp ; Brams and Fishburn, 1983; and Fishburn and Brams, 1983). Recently Felsenthal and Tideman (2013 and 2014) have confirmed by examples that other runoff systems (including the Dodgson, Nanson, and Coombs methods) are subject to monotonicity failure indeed, to what in Section 5 of this paper is dubbed double monotoncity failure. Many proponents of IRV (and others) have argued that monotonicity failure, while a mathematical possibility, is highly unlikely to occur in practice. Thus Amy (2002, p. 55) says: While it is clear that nonmonotonicity can theoretically occur in an IRV election, most experts believe that the conditions needed for this paradox to occur are so special that it would be an extremely rare occurrence. One statistical study [Allard, 1995 and 1996] found that if IRV-like elections were to be held throughout the United Kingdom, a nonmonotonic result would occur less than once a century. In a hands-on assessment of STV based on his years of experience as the Chief Electoral Officer for Northern Ireland since the introduction of STV in 1973, Bradley (1995) reported that the experience of the use of STV in Northern Ireland over the past 22 years, involving a range of election types and sizes, reveals no evidence to support in practice the lack of monotonicity. A standard text on electoral systems (Farrell, 2001, p. 150) cites both Allard and Bradley to support the claim that there is no evidence that it [i.e., monotonicity failure under IRV] is a common occurrence. Fair Vote (2009), a U.S. electoral reform group that advocates IRV (under the name of Ranked Choice Voting), claims that, in terms of the frequency of non-monotonicity in real-world elections: there is no evidence that this has ever played a role in any IRV election Also see Poundstone (2008), pp

3 Monotonicity Failure under IRV page 3 Since monotonicity failure is a striking and counterintuitive phenomenon, it may be helpful to provide a (more or less) real-world example namely, a simplified version of the 2009 IRV election for mayor of Burlington, Vermont. 2 The Republican candidate was supported by 39% of the first preferences, the Democratic candidate by 27%, and the Progressive (left-of-democrat) candidate by 34%. Thus the Democrat was eliminated, with his ballots transferring to one or other surviving candidate on the basis of Democratic second preferences, which were 37% for the Republican and 63% for the Progressive, representing 10% and 17% respectively of the total electorate. Thus in the instant runoff, the Republican got 39% + 10% = 49% and the Progressive won the election with 34% + 17% = 51%. Now consider a wholly make-believe sequel. A third of the Republicans (13% of all voters) are so traumatized by the prospect of a Progressive mayor that they leave Burlington for more politically hospitable climes and are replaced by a like number of newcomers attracted by the prospect of a Progressive mayor. At the next election, all votes are cast exactly as before, except for the 13% of the electorate once made up of Republicans now replaced by Progressives. The Progressive candidate won a squeaker before, so with this augmented support he will surely win more comfortably this time. But in fact he doesn t win at all. The Republican candidate now has 26% of the vote, the Democrat 27%, and the Progressive 47%, so the instant runoff is between the Democrat and the Progressive candidates, which the Democrat wins handily by gaining the second preferences of the (remaining) Republican voters (who find the prospect of a Democratic mayor at least marginally more tolerable than a Progressive one). So the consequence of the Progressive candidate s first preference support being augmented by of 13% of the electorate is that he loses where before he won. After various preliminaries are set out in Section 2, Sections 3-5 specify the precise conditions under which variants of monotonicity failure arise in three-candidate elections. These conditions are relatively straightforward and have previously been stated by Lepelley et al. (1996) but are more fully explicated here. Sections 6-10 provide estimates of the likelihood of monotonicity problems in varying circumstances by applying these conditions to large sets of simulated IRV election data, including data that meet special conditions such as single-peakedness, and also to data derived from three-candidate English election results; in this respect, the results supplement the findings of Plassmann and Tideman (2014) and especially Norman and Ornstein (2014) and as well as earlier unpublished work by Ornstein (2010) and Smith (2010). 2. Preliminaries A three-candidate IRV ballot profile is a set of n rankings of candidates X, Y, and Z, one for each of n voters. We assume all voters rank all three candidates. Given a particular ballot profile B, the candidate with the most first preferences is the Plurality Winner, the candidate with the second most first preferences is the Plurality Runner-Up, and the candidate with the fewest first preferences 2 The description here is simplified in that there were other minor candidates, some voters cast truncated ballots (that did not to rank all candidates), and voter preferences as expressed on these ballots were not entirely single-peaked (i.e., not all Republicans ranked the Democrat over the Progressive and not all Progressives ranked the Democrat over the Republican). Detailed vote tallies may be found at org/burlington. html. Also see footnote 13.)

4 Monotonicity Failure under IRV page 4 is the Plurality Loser. Under ordinary Plurality Voting, the Plurality Winner is by definition elected. Let n (PW), n (P2), and n (PL) be the number of ballots that rank the Plurality Winner, the Plurality Runner-Up, and Plurality Loser first. 3 Given three candidates, it must be that n(pl) < n/3 < n (PW). If n/2 < n(pw), the Plurality Winner is also a Majority Winner. Note that all these definitions depend on the distribution of first preferences only. The IRV winner is the Majority Winner if one exists and otherwise is either the Plurality Winner or the Plurality Runner-Up, depending on the outcome of the instant runoff between them. If the Plurality Runner-Up wins the runoff, IRV and Plurality Voting produce different winners; in this event, the profile is critical. Given a three-candidate ballot profile B, candidates X, Y and Z have x, y, and z first preferences respectively, so x + y + z = n; x y is the number of voters who have a first preference for X and second preference for Y (and therefore a third preference for Z), and x z is the number who have a first preference for X and a second preference for Z, so x y + x z = x; and likewise for other candidates. Given a different ballot profile BN, the candidates have xn, yn, and zn first preferences respectively, and likewise for the other notation. Given a ballot profile B, if a majority of voters rank X over Y, i.e., if x + z x > y + z y, we say that X beats Y, and likewise for other pairs of candidates A Condorcet Winner beats both other candidates, and a Condorcet Loser is beaten by both other candidates. But it may be that X beats Y beats Z beats X (or X beats Z beats Y beats X), in which case there is no Condorcet Winner or Loser but instead a Condorcet cycle. Note that these definitions depend on the distribution of second and third, as well as first, preferences. A Majority Winner is always a Condorcet Winner but, with three or more candidates, a Plurality Winner may be a Condorcet Loser (as the Republican candidate in the 2009 Burlington election evidently was) and the Plurality Loser may be a Condorcet Winner (as the Democratic candidate evidently was). However, a Plurality Loser cannot be the IRV winner even it is a Condorcet Winner (again like the Democratic candidate in Burlington). Our aim is to specify conditions under which an IRV ballot profile B is vulnerable to monotonicity failure, that is: (a) (b) the IRV winner is X but X would lose under some other profile BN that differs from B only in that some voters rank X higher in BN than in B (Upward Monotonicity Failure or UMF), or X loses under IRV but X would win under some other profile BN that differs from B only in that some voters rank X lower in BN than in B (Downward Monotonicity Failure or DMF). Note that, in either event, every voter ranks Y and Z the same way under B and BN. Following Norman (2010), we call B and BN companion profiles. We round up some self-evident observations concerning companion profiles. 3 Like Lepelley et al. (1996), I assume that no ties occur, on the supposition that the number of voters is so large that ties almost never occur (as is true in the data used here); moreover, there is no standard way to break ties under either Plurality Voting or IRV.

5 Monotonicity Failure under IRV page 5 Lemma 1. In the event that profile BN differs from B only in that some voters rank X higher in BN than in B, the following relationships hold: (1) if X is a Majority Winner under B, X is also the Majority Winner under BN; (2) X is ranked first on no fewer, and Y and Z on no more, ballots under B than BN; (3) if X beats Y (or Z ) under B, X beats Y (or Z ) under BN; and (4) Y beats Z (or Z beats Y) under BN if and only if Y beats Z (or Z beats Y) under B. Profiles in which the Plurality Loser has at least one quarter of first preference support are especially significant. In this event, the profile is competitive, and we establish the following. Lemma 2. If a profile is competitive, n(p2) < 3n/8 < n(pw) < n/2, so there is no Majority Winner. If n(pl) > n/4, it follows that n(pw) + n(p2) < 3n/4, so n(p2) < (3n/4)/2 = 3n/8; moreover, n(p2) > n(pl) > n/4, so n(pw) < n/2. Given that Z is the Plurality Loser under profile B, two conditions must hold for B to be vulnerable to (Upward or Downward) Monotonicity Failure in the event that X is moved up or down in some ballot orderings. Condition 1. This condition pertains to the runoff pair and requires that the ballot changes that produce BN from B must deprive Y of enough first preferences (for UMF), or give Z enough additional first preferences (for DMF) that the runoff that had been between X and Y is now between X and Z. Condition 2. This condition pertains to the runoff outcome and requires that X, which won (for UMF) or lost (for DMF) the runoff against Y under B must lose (for UMF) or win (for DMF) the runoff against Z under BN. The conjunction of Conditions 1 and 2 is necessary and sufficient to make B vulnerable to (Upward or Downward) Monotonicity Failure. 3. Upward Monotonicity Failure Let X be the IRV winner and let Z be the Plurality Loser under ballot profile B (so X beats Y in any runoff). We make these observations: (a) (b) (c) ballot changes that move X upwards from third to second place cannot change the IRV winner, because (i) if X is already the Majority Winner, it remains so, (ii) if X is not the Majority Winner under B, X is still paired with Y in the runoff (because no first preferences have changed) and (iii) X still beats Y in this runoff; and likewise ballot changes that move X upwards from third or second place to first place on ballots that had Z in first place cannot change the IRV winner, because either (i) X becomes a Majority Winner and wins without a runoff or (ii) it remains true that X is paired with and defeats Y in the runoff; and therefore the essential difference between an initial ballot profile B and a companion ballot profile BN that produces UMF is that X is ranked first on some ballots in BN on which Y was ranked first in B.

6 Monotonicity Failure under IRV page 6 If profile B is vulnerable to UMF, Condition 1 requires that X can gain enough first preference ballots at Y s expense that two things are simultaneously true in the resulting ballot profile BN: (i) X is still not a Majority Winner, and (ii) Y becomes the Plurality Loser instead of Z. This can occur whenever the difference between half of the total vote and the first preference support for X exceeds the difference between the first preference support for Y and that for Z, i.e., n /2! x > y! z. Condition 2 requires that Z beat X under BN, i.e., zn + y z N > xn + y x N. Condition 1 can be simplified and, given Condition 1, Condition 2 can be restated in terms of the original profile B. Proposition 1. A three-candidate ballot profile B in which X is the IRV winner and Z is the Plurality Loser is vulnerable to Upward Monotonicity Failure if and only if: Condition 1U: z > n /4; and Condition 2U: z + y z > x + y x. 4 Condition 1 requires that n /2! x > y! z. Substituting (n! y! z) into this expression in place of x, removing parentheses, and simplifying gives n/2! n + y + z > y! z, which further simplifies to Condition 1U. Thus z > n /4 puts Z, rather than Y, into the runoff with X under BN. Condition 2 requires that Z beat X in the runoff under BN. From Lemma 1(c), this implies that Z also beats X under B. Therefore, Condition 2U is necessary for UMF, but it needs to be shown that it is also sufficient. In the event that y x $ y! z, all the first preference ballots that X must gain at Y s expense to make Y the Plurality Loser under BN can come from the y x ballots that would in any case transfer to X in a runoff with Z under B, so Z beats X by the same margin under BN as under B. If y x < y! z, it is evidently more difficult for Z to beat X under BN than under B because, to the extent that y! z exceeds y x, Z loses and X gains [(y! z)! y x ] transferred ballots from Y in the runoff. Therefore, it must be that Suppose to the contrary that z + y z! [(y! z)!y x ] > x + y x + [(y! z)!y x ]. z + y z! [(y! z)!y x ] # x + y x + [(y! z)!y x ]. Removing parentheses and rearranging terms, we get 4 Proposition 1 is essentially identical to Proposition 1 in Lepelley et al. (1996, p. 136); similar conditions are used by Ornstein and Norman (2014). As noted in footnote 3, it is assumed that ties do not occur. Three types of ties may occur under IRV with three candidates. First, if the number of voters is even, (i) the Plurality Winner may receive exactly half the votes or (ii) or the instant runoff may be tied. Second, whether the number of voters is even or odd, candidates may be tied for Plurality Runner-Up status. Event (i) causes no problem if we suppose that the runoff proceeds. In the event of (ii) or (iii), it not clear what happens under IRV; if such ties are broken by coin flips, the conditions for monotonicity failure become rather messy.

7 Monotonicity Failure under IRV page 7 3z # x + y. Substituting (n! z) for (x + y) and further simplifying, we get z # n/4, contradicting Condition 1U. 5 Proposition 1 can be stated more transparently as follows. Proposition 1'. A three-candidate ballot profile B is vulnerable to Upward Monotonicity Failure if and only if (1) the profile is competitive; and (2) the Plurality Loser beats the IRV winner. Thus UMF can occur only in relatively closely contested elections such that all three candidates receive between 25% and 50% of the first preference votes and either the Plurality Loser is the Condorcet Winner or the profile is cyclical. Both critical and non-critical profiles may be vulnerable to UMF. 4. Downward Monotonicity Failure Let Y be the IRV winner and Z be the Plurality Loser under ballot profile B (so X is beaten by Y in any runoff). We make these observations: (a) (b) (c) ballot changes that move X downwards from second to third place cannot change the IRV outcome, because (i) if Y is already the Majority Winner, it remains so, or (ii) if Y is not the Majority Winner, X is still paired with Y in the runoff (because no first preferences have changed), and (iii) Y beats X in this runoff; and likewise ballot changes that increase Y s first preferences by moving X downwards cannot make X the IRV outcome because (i) if Y is the Majority Winner, it remains so, or (ii) X will no longer make it into the runoff with Y or (iii) X is still paired with and beaten by Y in the runoff, and therefore the essential difference between the initial ballot profile B and a companion ballot profile BN that produces DMF is that X is ranked second and Z first on some ballots in BN on which X was ranked first in B. If profile B is vulnerable to DMF, Condition 1 requires that it is possible for X to lose enough first preference ballots in favor of Z that two things are simultaneously true in the resulting profile BN: (i) Z is no longer the Plurality Loser, and (ii) Y, rather than X, becomes the Plurality Loser. This can occur whenever the difference between first preference support for X and for Y exceeds the difference between first preference support for Y and for Z, i.e., x! y > y! z. Furthermore, in order for Z to gain these first preferences rather than Y, these (y! z) new first preference ballots for Z must all come from the x z ballots that initially ranked Z rather than Y second, so that it must be that x z > y! z. Condition 2 requires that X beat Z under BN, i.e., xn + y x N > zn + y z N. These conditions can be simplified and restated in terms of the original profile B. 5 Note that this does not mean that the Condition 1U is by itself sufficient for UMF. Rather it means that Condition 1U implies that, if Z beats X under B, Z also beats X under BN and therefore implies UMF.

8 Monotonicity Failure under IRV page 8 Proposition 2. A three-candidate ballot profile B in which Y is the IRV winner and Z is the Plurality Loser is vulnerable to Downwards Monotonicity Failure if and only if: Condition 1D: (a) y < n /3 and (b) x z > y! z; and Condition 2D: y + y z < n/2. 6 Condition 1 requires that x! y > y! z. Rearranging and substituting n! y for x + z gives n! y > 2y. Rearranging further gives Condition 1D(a). The further requirement of Condition 1 is directly stated by Condition 1D(b). Condition 1D puts Z, rather than Y, in a runoff with X under BN. Condition 2 requires that X beat Z in the runoff under BN. By Lemma 1(c), this implies that X must beat Z under B, i.e., x + y x > z + y z ; moreover, X must still beat Z after (y! z) first preference ballots shift from X to Z; that is, it must be that x!(y! z) + y x > z + (y! z) + y z. Removing parentheses, rearranging, and cancelling gives x + z + y x! y > y + y z. Substituting n! y for x + z and!y z for y x! y, this expression simplifies to Condition 2D. DMF. The following corollary states more transparent necessary (but not sufficient) conditions for Corollary 2.1. A three-candidate ballot profile B under is vulnerable to Downward Monotonicity Failure only if (1) the profile is critical; (2) the Plurality Loser is supported by the first preferences of more than one sixth of the voters; and (3) the Plurality Loser is not a Condorcet Winner. Since it has less than a third of first preference support, the IRV winner cannot be the Plurality Winner, so only critical profiles are vulnerable to DMF. Since the IRV losers together have more than two-thirds of the first-preference support and since the Plurality Winner cannot be a Majority Winner, the Plurality Loser has more than one-sixth of such support, so elections vulnerable to DMF may be less closely contested than those vulnerable to UMF. Given the labelling of the alternatives as in Proposition 1, Y beats X and X beats Z, so either Y is the Condorcet Winner or the profile is cyclical. 5. Double Monotonicity Failure There is an obvious connection between Upward and Downward Monotonicity Failure. Consider a ballot profile B that is vulnerable to UMF with respect to profile BN as specified by Proposition 1. Then profile BN is clearly vulnerable to DMF with respect to profile B as specified by 6 Proposition 2 is essentially identical to Proposition 3 in Lepelley et.al. (1996, p. 139).

9 Monotonicity Failure under IRV page 9 Proposition 2. That is to say, profiles that are vulnerable to monotonicity failure come in companion pairs, one vulnerable to UMF and the other to DMF. In this sense, UMF and DMF are the same phenomenon. Note, however, that this does not mean that profiles pair off as unique companions or that equal numbers of profiles are vulnerable to each type of monotonicity failure. Proposition 1' and Corollary 2.1 together imply the following pertaining to companion profiles. Corollary 2.2. Given three-candidate companion profiles B and BN such that B is vulnerable to UMF and BN to DMF, the IRV winner under BN beats the IRV winner under B. A further question is whether a single profile B can be simultaneously vulnerable to both UMF and DMF. Call this vulnerability to Double Monotonicity Failure (2MF). Proposition 1' and Corollary 2.1 together imply the following. Corollary 2.3. A three-candidate ballot profile is vulnerable to 2MF only if it is competitive, critical, and cyclical. 7 Corollary 2.3 does not itself establish that ballot profiles vulnerable to 2MF actually exist. Such a profile must satisfy Conditions 1U and 1D and also Conditions 2U and 2D. That such a ballot profile can exist is shown by the following example with n = 100, in which Z is the Plurality Loser and X is the IRV winner: Y X Z Z Y X X Z Y The profile is vulnerable to UMF because, if 9 of the 38 YZX voters move X to the top of their ballots, Y becomes the Plurality Loser instead of Z, and Z then loses to Y in the runoff, so Z becomes the IRV winner. At the same time, it is vulnerable to DMF because, if 3 of the 38 YZX voters drop Y to second or third preference, Y remains the Plurality Winner but X becomes the Plurality Loser, and Y then beats Z in the runoff, so Y becomes the IRV winner Monotonicity Failure with Random Ballot Profiles As noted earlier, it has been claimed that monotonicity failure, while mathematically possible, is highly unlikely to occur in practice. Therefore, we examine three large and diverse samples RAN1, RAN2, and RAN3, each with 256,000 randomly generated ballot profiles for about 30 million voters; each of the three candidates has on average the first preference support of about 10 million voters but, in any particular profile, they are likely to have very different levels of (first and second 7 Cyclicity follows because UMF implies the Plurality Loser beats the IRV winner but DMF implies it cannot be a Condorcet Winner. However, Felsenthal and Tideman (2014) have shown that cyclicity is not required for 2MF with five or more candidates. 8 Lepelley et al. (1996, p. 141) note that such profiles may exist.

10 Monotonicity Failure under IRV page 10 preference) support. 9 The three samples differ with respect to the variability of first preference support for each candidate across profiles; RAN1 has a standard deviation of about 1.2 million for each candidate, RAN2 of about 2.4 million, and RAN3 of about 3.3 million. 10 Table 1 shows, for each sample, the percent of profiles are vulnerable to Upward, Downward, and Double Monotonicity Failure, as well as Total Monotoncity Failure (TMF = UMF + DMF!2MF). It also shows similar percentages for competitive, critical, and cyclical profiles, as well as those that meet the conditions given in Corollary As can be seen, the rate of vulnerability to monotonicity failure varies considerably among the three samples but is substantial in all of them. However, Figures 1 and 2 reveal that the factor fundamentally at work is election closeness; RAN1 has on average the closest elections and RAN3 the least close and, once we control for closeness, the three samples exhibit very similar rates of Monotonicity Failure. In Figure 1, election closeness is measured by the percent of first preference support for the Plurality Loser. Profiles are stratified into intervals one percentage point wide with respect to closeness (except that the highest category extends from 32% to 33a%); note the distribution of profiles over intervals in shown for each sample. Consistent with Condition 1U, no instance of vulnerability to UMF (or 2MF) occurs when this percent is less than 25% but, once this threshold is crossed, about 10% of profiles are vulnerable to UMF, and this percentage increases steadily thereafter approaching 50% in the closest contests. In contrast, vulnerability to DMF does not become evident once the logical threshold of about 17% is crossed but does appear when the Plurality Loser wins about 20% of the vote and increases slowly but steadily thereafter, reaching about 15-20% in the closest contests. Once the 25% threshold is crossed, about half of the profiles vulnerable to DMF are also vulnerable to UMF, and thus 2MF, as well. When elections are most closely contested, more than 50% of profiles are vulnerable to some kind of monotonicity failure. Figure 2 is similar to Figure 1 but measures closeness by the difference between the percent of first preference support for the Plurality Winner and that for the Plurality Loser. This measure of closeness entails no specific threshold for UMF; both UMF and DMF begin to appear gradually 9 Thus, these profiles are not drawn from what social choice theorists call an impartial culture, which we instead take up in the next section. 10 Specifically, each sample RAN1, RAN2, and RAN3 was generated by drawing the number of first preferences for candidate X from a normal distribution with a mean of 10 million and standard deviation of 1.2 or 2.4 or 3.3 million, subject to the constraint that x $0, and rounding to the nearest integer. Then the number of such ballots ranking Y second was drawn from a normal distribution with a mean of x/2 and a standard deviation of x/6, subject to the constraint that 0 # x y # x, with Z ranked second in the remaining x z = x! x y ballots. The numbers for the other rankings were determined in like manner. Given the large number of votes, no ties occur in any of these samples. 11 The conference version of this paper presented more detailed tables (though based on different smaller and less varied simulated data sets) that reported the frequency with which profiles met the individual conditions 1U, 2U, 1D(a), 1D(b), and 2D. This version of the paper is available at ELECTSYS.html.

11 Monotonicity Failure under IRV page 11 (though DMF appears earlier) as closeness increases. Otherwise, the Figure 2 presents a picture that is very similar to Figure 1. In summary, in this diverse and perhaps most representative set of simulated profiles, vulnerability to Monotonicity Failure (especially Upward) in three-candidate IRV elections is hardly a rare event, increases with election closeness, and occurs more than half the time in the most closely contested elections. 7. Monotonicity Failure in an Impartial Culture Many social choice analyses assume an impartial culture in which each voter is equally likely to cast a ballot ranking the three candidates in each of the six possible ways. Thus, if the number of voters is very large, x y. x z.y x.... z y. n/6 and x. y. z. n/3 in almost every ballot profile. While the assumption that profiles are drawn from an impartial culture is, in a sense, the most neutral one that can be made and provides the basis for many probability calculations in social choice and voting power theory, it also implies that almost all elections with many voters are extraordinarily close and in this respect at least is empirically implausible. Nevertheless, given its prominence in social choice analyses, it is worthwhile to examine IRV ballot profiles drawn from an impartial culture for vulnerability to monotonicity failure. We examine a single sample IC of 256,000 ballot profiles drawn from an impartial culture with 30 million voters. 12 Since we have seen that vulnerability to monotonicity failure occurs in more half of the closest random elections, we might at first blush expect impartial profiles to exhibit vulnerability at a similar very high rate. But in fact this is not the case. Table 2 is set up in the same manner as each panel of Table 1 and displays percentages that are generally similar, despite the fact that all the elections are extraordinarily close. Indeed, in no profile does the Plurality Loser have less than % of first preference support, and in no profile is the margin separating first-preference support for Plurality Winner and Loser greater 0.036%. Nevertheless, on a veritably microscopic scale, we can still distinguish among different levels of election closeness, and Figures 3 and 4 appear to be quite similar to Figures 1 and 2 until we note the scale on the horizontal dimension. The reason that vulnerability to monotonicity failure among impartial culture profiles is no greater than among to random profiles appears to be this. Given random profiles, the various conditions required for vulnerability on average fall considerably short of the thresholds required for monotonicity failure, but there is considerable dispersion about these averages, so the required thresholds are quite often met. Given impartial culture profiles, the conditions on average fall only slightly below the required thresholds, but there only very slight dispersion about these averages, so the required thresholds are met no more frequently. Put another way, impartial culture profiles resemble those in a sample such as RAN1 in which each quantity x y, etc., is augmented by a vastly greater constant. While this makes every profile extraordinary close in relative (percentage) terms but 12 The number of voters with each of the six ballot rankings was drawn from a normal distribution with a mean of 5 million and a standard deviation equal to the square root of 1.25 million or 1118, i.e., the normal approximation of the binomial distribution with p = 0.5, rounded to the nearest integer. While the 256,000 profiles include 15 with some sort of tie, these few cases can be ignored without affecting any of the results reported here.

12 Monotonicity Failure under IRV page 12 it has no impact on the plurality, Condorcet, or IRV winner status of the candidates and therefore little impact on their vulnerability to monotoncity failure 7. Monotonicity Failure with Single-Peaked Preferences Given three candidates, suppose that one is generally perceived as centrist in his ideological or policy positions relative to the other two candidates, who in turn are perceived as (relatively) extreme but in opposite directions (e.g., one to the left and the other to the right of the centrist candidate). If the centrist candidate is universally seen as a compromise between the two more extreme one, no voters will have the centrist as their third preference. Such a preference profile is said to be single-peaked; since one candidate is never ranked lowest, such preferences are also said to be bottom-restricted. Lemma 2 rounds up some elementary propositions concerning such preferences. Lemma 3. If voter preferences expressed on a ballot profile are single-peaked: (1) a cycle cannot occur; (2) an extreme candidate is a Condorcet Winner if and only if he is a Majority Winner, and (3) otherwise the centrist candidate is a Condorcet winner; (4) the centrist candidate is never the Condorcet Loser; (5) the IRV winner is the Condorcet Winner unless the Condorcet Winner is the Plurality Loser (necessarily the centrist candidate), (6) in which case the IRV winner is the extreme candidate that beats the other extreme candidate. Given Lemma 3, Proposition 1' restricted to the special case of single-peaked preferences reduces to the following. Proposition 3. If the voter preferences expressed on a three-candidate ballot profile B are singlepeaked, B is vulnerable to Upward Monotonicity Failure if and only if the centrist candidate (1) is the first preference of more than 25% of the voters, but (2) is nevertheless the Plurality Loser. Given single-peakedness, the conditions for Downward Monotonicity specified in Proposition 2 cannot be simultaneously fulfilled. Proposition 2 stipulates that Z is the Plurality Loser and Y is the IRV winner. If Y is an extreme candidate, Y is the IRV winner only if Y is a Majority Winner, contradicting the requirement that y < n/3. Therefore, Y must be the centrist candidate. But this cannot be true either, because single-peakedness then implies that x z = 0, contradicting the requirement that x z > y! z. Thus we have the following. Proposition 5. If the voter preferences expressed on a three-candidate ballot profile B are singlepeaked, B cannot be vulnerable to Downward (or Double) Monotonicity Failure Cf. Lepelley et al., p.146. Proposition 5 might seem to be contradicted by the fact that a singlepeaked profile B may be vulnerable to UMF with respect to a companion profile BN, so BN must be vulnerable to DMF. This is true, but the implication is that profile BN is not single-peaked, not that Proposition 5 is contradicted. This consideration raises a consideration that was sidestepped in presenting the make-believe sequel to the Burlington mayoral election. In order to meet the definition of monotonicity failure, the 13% of the electorate

13 Monotonicity Failure under IRV page 13 We now examine three large samples of single-peaked ballot profiles. Sample SP1 was generated in the same manner as RAN2, except that all ballots with X or Z ranked first were assigned Y as the second preference (making Y the centrist candidate). In the other two samples, preferences are restricted in the same way, but one candidate is average less popular than the two others, with an average of 6 million first preferences while the other two averaged 12 million (with standard deviations scaled proportionately). Sample SP2 has a weak centrist candidate Y and SP3 has a weak extreme candidate Z. Table 3, which is set up in a more condensed fashion than the preceding tables because only one type of monotonicity failure can occur, shows the rate of vulnerability to UMF for each of the three samples of single-peaked profiles. Perhaps the most striking feature of the table is the high incidence of vulnerability among competitive profiles with a weak center candidate, which results because the center candidate is very likely to be the Plurality loser; nevertheless, the overall rate of vulnerability is lower than in the symmetric case, since many fewer profiles are competitive in the first place. Figures 5 and 6 shows, for each of the single-peaked samples, the rate of vulnerability to UMF by election closeness and they display, for SP1 and SP2 rather different patterns from other types of profiles. Proposition 4 makes Figure 5, in which closeness is measured by support for the Plurality Loser, particularly easy to understand. In the symmetric case, each candidate is equally likely to be the Plurality Loser, so the centrist candidate has this status in one third of the profiles (with a little sampling fluctuation) and the these profiles are vulnerable to UMF if and only if they are competitive. A weak centrist candidate is typically but not always the Plurality Loser, so most competitive profiles are vulnerable to UMF; however, as elections become closer and support for the Plurality Loser increases, the center candidate is less likely to have this status, so vulnerability actually declines. However, the opposite pattern applies when an extreme candidate is weak. The same considerations account for the general pattern displayed in Figure 6, in which closeness is measured by the difference in support for the Plurality Winner and Loser. 8. Monotonicity Failure with Clone Candidates Consider a three-candidate election in which two candidates C 1 and C 2 have similar policy positions or otherwise appeal to same group of voters, while a third candidate X has a distinctive policy position or otherwise appeals to a different group of voters. Thus there are two sets of voters with substantially opposed preferences: those who prefer both C 1 and C 2 to X and those who prefer X to both C 1 and C 2 ; however, voters in both groups may have either preference between C 1 and C 2. Following Tideman (1987), C 1 and C 2 are clones, while X is the exceptional or (extreme) candidate whom no one ranks second. Therefore, as with single-peakedness, profiles include only four of the six possible rankings of the three candidates but, whereas single-peaked preferences are bottomrestricted, these preferences are middle-restricted. composed of the newly arrived Progressives who replaced the fleeing Republicans would have the preference ordering P > R > D (instead of the more plausible P > D > R), thereby violating single-peakedness.

14 Monotonicity Failure under IRV page 14 The case in which X supporters are a large minority, and particular when n/3 < x < n/2, is of special interest. If the C supporters are sufficiently equally divided between the two clones with respect to their first preferences, candidate X is then the Plurality Winner (and would be elected under Plurality Voting) even though X is also the Condorcet Loser; thus the C supporters constitute a majority vulnerable to vote splitting either clone can win if the other is not a candidate but, if both are candidates, each spoils the other s chance of election. One appeal of IRV is that it resolves this spoiler problem to the advantage of the majority of voters favoring the clone candidates, because at least one clone must get into the instant runoff, in which it defeats X and thereby becomes the IRV winner. But this advantage of IRV comes at some cost, namely vulnerability to (upward) monotonicity failure. First, we round up several well-known or self-evident points in the form of another lemma. Lemma 4. If voter preferences expressed on a ballot profile are middle-restricted: (1) a cycle cannot occur; (2) the IRV winner is the extreme candidate X if and only if X is the Majority Winner; and (3) otherwise X is the Condorcet Loser and the IRV winner is the Condorcet Winner, namely the clone candidate that beats the other clone candidate. Given Lemma 4, Proposition 1' restricted to the special case of middle-restricted preferences gives the following. Proposition 6. If the voter preferences expressed on ballot profile B are middle-restricted, B is vulnerable to Upward Monotonicity Failure if and only if (1) it is competitive, and (2) the Plurality Loser is the clone candidate that beats the other clone. Note that the Plurality Loser must be disproportionately favored as the second preference of X supporters in order for condition (2) to hold. Given middle-restricted preferences, the conditions for Downward Monotonicity specified in Proposition 2 cannot be simultaneously fulfilled. Proposition 2 stipulates that Z is the Plurality Loser and Y is the IRV winner. Given middle-restricted preferences without a Majority Winner, the IRV winner Y must be one of the two clone candidates. If X is the other clone and Z is extreme, x z = 0; but, given that Z is the Plurality Loser, y! z > 0, so Condition 1D(b) cannot hold. If Z is the other clone and X is extreme, y z = y, so Condition 2D becomes 2y < n/2 or y < n/4. But Z is the Plurality loser, so z < y < n/4 and x > n/2, making X the Majority Winner and contradicting the stipulation that Y is the IRV winner. Thus we have the following. Proposition 7. If the voter preferences expressed on a three-candidate ballot profile B are middlerestricted, B cannot be vulnerable to Downward (or Double) Monotonicity Failure. We now examine two large samples of ballot profiles with clone candidates. Sample CL1 was generated in the same manner as RAN2, except that no ballots with X or Z ranked first were assigned Y as the second preference (making Y is the extreme candidate), whereas in sample CL2, the

15 Monotonicity Failure under IRV page 15 extreme candidate has on average the first preference support of 12 million voters compared to 9 million for each of the clones (with standard deviations scaled proportionately), so as to make the spoiler problem under Plurality Voting especially prominent. Table 4, set up in the same manner as Table 3, suggests that vulnerability to UMF is about as prevalent under middle-restricted as bottom-restricted preferences, though a heightened potential for spoiler effects under Plurality Voting somewhat increases its prevalence. However, Figures 7 and 8, showing the relationship with election closeness, clarifies the matter. Figure 7 shows that, once profiles become competitive, profiles drawn from the sample with a strong extreme candidate are considerably more likely to be vulnerable to UMF than those from the symmetric sample; as competitiveness increases, the incidence of vulnerability increases in both samples but in the latter more rapidly, so that the rate converges at about 30% in the most competitive elections. Using the other measure of election closeness, Figure 8 shows that the rates for both samples increase more or less in tandem Monotonicity Failure in English General Elections: Finally, we examine samples based on constituency-level data from the five U.K. general elections from 1992 through Only data from English constituencies is used, because virtually all elections in England during this period were essentially three-party (Labour, Liberal Democrat, and Conservative) affairs, while those in Wales, Scotland, and Northern Ireland almost always included strong (and often winning) candidates of nationalist parties as well (and in 2015 those in England included significant UK Independence Party candidates). The handful of English constituencies that did not closely fit a three-candidate pattern were also excluded. 16 The five general elections give us a sample of 2642 three-candidate elections. An obvious problem is that these elections were conducted under Plurality Voting and therefore the election data provides us only with (what we take to be) the first-preferences of voters, while analysis of IRV elections requires that voters rank the three candidates in (what we would take to be) order of preference. This problem is addressed by allocating second preferences in each district and each year in proportion to second preferences nationwide, as determined by surveys that provide 14 Single-peaked (bottom-restricted) and middle-restricted preferences are examples of value-restricted preferences (Sen, 1966). Clearly there is a third category of value-restriction, namely top-restricted (or singlecaved ) preferences, i.e., profiles in which there is a candidate that no one ranks first. But if only two candidates are ranked first, one or the other is a Majority Winner, so no such profiles are vulnerable to monotonicity failure. 15 This data come from Pippa Norris s Shared Datasets website ( Data/Data.htm). I am extremely grateful to Professor Norris for making this valuable data readily available. For an ingenious attempt, using somewhat informal methods, to identify instances of vulnerability to monotonicity failure in actual (Irish) IRV elections, see Gallagher (2013). 16 In 2010 these included the Speaker s constituency (by tradition the Speaker is not opposed by majorparty candidates) and one district won by a fourth-party (Green) candidate.

16 Monotonicity Failure under IRV page 16 individual level data about second preferences for each of the five elections. 17 These surveys indicate that English voter preferences were partially single-peaked that is, most but not all Labour ( leftof-center ) voters had the Liberal Democrats (the centrist party) as their second preference as did most but not all Conservative ( right-of-center ) voters, while Liberal Democrat voters had more evenly divided second preferences that typically leaned somewhat in the Labour direction but with the proportion varying considerably from election to election. In addition to this sample (designated ENG1), I also generated two other samples with different second (and third) preferences: in sample ENG2, preferences are strictly single-peaked; in sample ENG3, second preferences are assigned randomly in the same manner as the RAN samples. Table 5 is set up in the same manner as Tables 1 and 2. The most obvious and striking feature of the English data is that considerably fewer profiles (on the order of 1-2%) are vulnerable to monotonicity failure than in any of the wholly simulated data sets. This might suggest that the simulated data (and preceding analysis) are largely irrelevant and misleading once we look at (more or less) real electoral data, the problem of monotonicity failure under IRV almost disappears perhaps not to the vanishingly low level first claimed by Allard (1996) but to a very low level indeed. However, we have repeatedly seen that the primary determinant of vulnerability to monotonicity failure is election closeness, and very few of these English elections were closely contested three-candidate contests (perhaps because they were actually conducted under Plurality Voting, not IRV). In fact, a large majority (60%) of all English ballot profiles had a Majority Winner, and in very few (4.2%) did the Plurality Loser get as much as 25% support. Table 5 shows, among the small number of competitive English elections, vulnerability to monotonicity was fairly common. The underlying similarity between the English data and the most pertinent simulated data i.e., the random and single-peaked profiles, is evident when we look at Tables 6 and 7, which show monotonicity failure for the English data by election closeness in the same manner as the preceding figures (but in table format with collapsed categories of closeness because of the small number of cases in each closeness bracket.) Controlling for election closeness and taking account of unreliability of the rates due to the small number of cases, vulnerability to monotonicity failure appears to be approximately as common in the English data as in the simulated data in particular, approaching 50% in the most closely contested elections. In sum, when we control for election closeness, the English data is consistent with conclusions reached on the basis of simulated data. 10. Concluding Remarks This paper has set out precise conditions under which vulnerability to monotonicity failure arises in three-candidate IRV elections, and it has applied these conditions to a number of varied large sets of ballot profiles in order to get a sense of the severity of IRV s monotonicity problems in 17 Data for 1992 through 2005 comes from Curtice (2009), which in turn comes from the British Election Study (post-election) for 1992 and 1997 and from ICM/BBC (pre-election) for 2001 and Data for 2010 comes from Ritchie and Gardini (2012), which in turn was taken a (pre-election) poll conducted for ITV News and The Independent newspaper. Survey respondents who gave a nationalist or other fourth-party second preference, or who did not express a second-preference, were excluded in these calculations, and proportions were calculated on the basis of Labour plus Liberal Democrat plus Conservative second preferences only.