Evidence-based elections: Beyond the rigging debate IN DETAIL

Transcription

1 IN DETAIL Gino Santa Maria/Bigstock.com Evidence-based elections: Beyond the rigging debate Claims of rigged voting made headlines during the 2016 US presidential election campaign. But while there is no convincing evidence of fraud, there is also a lack of strong evidence to say that US election systems produce the right results. Mark Lindeman explains the vote verification problem 18 SIGNIFICANCE February The Royal Statistical Society

2 I m telling you, November 8, we better be careful, because that election is going to be rigged. (Donald Trump, 1 August 2016) Were this year s deviations from pre-election polls the results of a cyberattack? Probably not [But the] only way to know is to closely examine the available physical evidence paper ballots and voting equipment in critical states (J. Alex Halderman, 23 November 2016) Throughout the 2016 US presidential campaign, talk of alleged past, ongoing, and future election fraud flew thick and fast first on social media, then on news blogs, and then in the mainstream media. Much of the mainstream reporting sought to evaluate cyber threats to voting machines and other election systems. Headlines ranged from How to hack an election in 7 minutes to Voting experts call fears of election hacking overblown. Other stories took on Donald Trump s repeated predictions of a rigged election which turned out to hinge on lurid scenarios of people voting 10 times, maybe, who knows?. With Trump s unexpected victory, the furore redoubled. Green Party presidential candidate Jill Stein requested recounts in the key states of Michigan, Wisconsin and Pennsylvania to look for widespread fraud or error. None of these recounts proceeded as far as she had requested; none found major errors. For some observers, revelations about Russia s role in promoting Trump s chances made foreign intervention in the vote count only too plausible. As this article went to press, no convincing evidence of systematic miscounting has been found, but some people continue to vociferously challenge the results challenges that others dismiss as the whining of sore losers. For election integrity experts, the spectacle has been hard to watch. I am part of the volunteer leadership of the Election Verification Network, a group of people in the USA and beyond who bring widely varied expertise to the task of implementing accurate elections that citizens can trust. While we do not agree about everything (and the opinions expressed here are my own), most of us do agree that better equipment and procedures could largely forestall such controversies. In particular, many US election systems have poor verification properties. Although there is no evidence that they often produce incorrect election outcomes, the evidence that election results are right is not as strong as it should be and can be. Mark Lindeman is a political scientist whose research includes public opinion, political behaviour, and election verification issues. His work to simplify, explain, and promote post-election vote tabulation audits includes co-authoring several methodological papers; serving as executive editor of the 2012 white paper Risk- Limiting Audits: Why and How on behalf of a multidisciplinary working group; and advising officials and advocates in several states about audit implementation. He is co-author of Public Opinion (Westview Press) of contests at once) and, more recently, the mandate to make voting accessible to people with various disabilities. The 2002 Help America Vote Act provided $3 billion in federal funding to help local jurisdictions purchase new voting equipment. Unfortunately, this equipment often has massive security flaws. DREs essentially are special-purpose computers. As such, they typically can be reprogrammed to do almost anything as when two computer scientists hacked a Sequoia AVC Edge voting machine to play the video game Pac-Man (see Figure 1). Optical scanners generally cannot play Pac-Man, but many are as insecure as DREs. Two statewide studies, in California (bit.ly/2hwqifh) and Ohio (bit.ly/2hwfij4), found serious security flaws in every voting system reviewed. Some hacks require physical tampering with individual machines before an election, although often not much. Other hacks do not require direct contact: malware can inadvertently be spread by election officials or by voters. Some machines could even be hacked from a distance using Wi-Fi. The election management software that aggregates results from individual machines also is subject to subversion or innocent error. Many election officials and other observers have argued that these vulnerabilities are not very important in practice because election administration is highly decentralised, and because procedural safeguards protect the systems. I largely agree that widespread hacking would be very difficult. Nevertheless, these two points are somewhat in conflict: in part because elections are administered by thousands of disparate local jurisdictions, we cannot tell how well protected the systems are. FBI director James Comey s statement that the US voting system is hack-resistant because it is clunky as heck is not altogether reassuring. Many jurisdictions have The basic verification problem Around the world, it is common to find countries conducting elections using hand-counted paper ballots. Most US jurisdictions, however, use electronic equipment: either paper ballots counted by optical scanners, or votes cast on directrecording electronic (DRE) systems, usually with touch-screen or push-button interfaces. In part, the use of voting machines reacts to a rich historical legacy in the USA of ballot-stuffing and other forms of fraud using paper ballots. It also reflects the complexity of US elections (where voters may vote in dozens FIGURE 1 Pac-Man running on a Sequoia AVC Edge voting machine. Picture courtesy of J. Alex Halderman and Ariel J. Feldman (jhalderm.com/pacman) February 2017 significancemagazine.com 19

3 instituted routine procedures, such as conducting logic and accuracy tests before each election, that provide some degree of assurance. Unfortunately, no set of procedures ever can confirm that complex software is performing to specification. No matter how much election officials trust their procedures, they have no credible way to convince the public. This is not a small problem. As computer scientist Dan Wallach has said, the point of elections is not just to choose a winner, but to convince the losers that they lost. Not everyone will be convinced even by objectively strong evidence but if there is no such evidence, the very premise of representative democracy is undermined. Thus, we need what Philip Stark and David Wagner call evidence-based elections: elections that provide convincing evidence that the winners actually won (bit.ly/2hsjxd2). Many voting systems do not meet Stark and Wagner s evidence-based standard. It is literally impossible to know how accurately they are recording people s votes; they can fail unobservably. We should fix that, and we can. Outlining the basics Evidence-based elections have three central elements. 1. Record the votes on a trustworthy audit trail. For the moment, that means paper records that voters themselves can verify: no more paperless electronic voting, whether in person or online. (So far, there is no complete, verifiable internet voting solution, although some progress has been made.) 2. Curate the audit trail: ensure that no records have been added, lost or changed, and implement compliance audits that provide evidence that this is so for instance, that all ballots are accounted for. Good curation entails that it is not possible for members of one political party to access voted ballots unilaterally, for example. 3. Check the audit trail: use risk-limiting audits that provide a high level of assurance that any tabulation errors are too small to affect the outcome, or that lead to full recounts that correct wrong outcomes. (The risk limit is the maximum chance that, if an outcome is wrong, it will not be corrected after the audit.) Public observers can watch the audit and, in some designs, even participate in the selection of the audit sample. The 2016 election shows how routine risk-limiting audits would improve upon current recount laws. These laws, which vary widely across the states, sometimes enable a recount when the reported margin of victory is very small, when candidates credibly challenge the result, and/or when candidates are willing to pay for the recount. Hillary Clinton declined to request any recounts, and drew some criticism Gino Santa Maria/Bigstock.com 20 SIGNIFICANCE February 2017

4 for purportedly abandoning her voters. Contrariwise, some people saw Stein s recount requests and fundraising efforts as political grandstanding and Michigan courts eventually ruled that she did not meet the legal standard to obtain a recount. Mandatory audits likely would have been less fraught and more effective. Statistics is especially important in designing these risklimiting audits. Some people advocate 100% hand counts in every election. Full hand counts are fine, but I think it is very unlikely that most jurisdictions will adopt them. So we need audits that use the paper records to confirm (or to correct) outcomes efficiently. For instance, audits of landslide contests generally can be smaller than audits of very close ones, because greater error rates can be tolerated. Full hand counts are needed only in the closest contests. Risk-limiting audits: the nitty-gritty One approach to risk-limiting audits is called a ballot-polling audit. This approach basically relies on simple random sampling, rather like an opinion survey but with the advantage that ballots cannot refuse to participate! If the initial sample does not provide strong evidence that the reported winner indeed got the most votes, the audit continues. (Since the sample is random, one cannot know in advance exactly how large it has to be to satisfy the risk limit.) Ballot-polling audits do not rely on any knowledge about the original vote counts, although they do require a full accounting of how many ballots there are, and where. They can be surprisingly efficient in large races that are not very close. Figure 2 uses California turnout data from the 2008 presidential election, with ballot-polling audits shown as the blue dotted curve. Here, it can be seen that a contest won by a margin of 4% would require an audit of approximately 4000 ballots which is around 0.03% of the votes cast. The expected sample size falls sharply as the winning margin increases. The other, more common approach is a comparison audit: manually inspecting one or more ballots, and comparing the manual interpretation to the machine interpretation of the same ballot(s). So far most audits have been (nonrisk-limiting) comparison audits at the precinct level: some precincts are randomly selected, and in each of them, hand counts of all the ballots cast are compared to the machine counts of the same ballots. Most California counties report vote-by-mail and election-day vote counts separately, so the state has almost twice as many auditable batches as there are precincts. Batch-level audits can be rather efficient, especially in jurisdictions (like California) with lots of batches. California presently audits simple random samples of 1% of batches in each county. These samples are large enough, statewide, to yield strong evidence about even fairly close election outcomes. That said, a more efficient approach the one shown in Figure 2 as the solid black curve is to select batches with probability proportional to error bound: in effect, to give more attention to batches with more reported votes for the Expected # of ballots audited apparent winner. Although these tend also to be larger-thanaverage batches, the total number of ballots to be audited can be substantially smaller. However, far more efficient yet is to conduct ballot-level comparison audits: to choose a random sample from among the ballots, and compare each ballot to how the machine interpreted its vote(s). Ballot-level comparison audits (the red dashed curve in Figure 2) require far less counting than ballot-polling audits especially at small margins because they estimate the error rate, which should be close to zero, instead of vote shares. Their advantage over batch-level audits is even larger: auditing, say, 100 randomly selected ballots provides almost as much information about the overall error rate as auditing 100 entire batches. Regrettably, most currently deployed optical-scan voting systems do not support ballotlevel audits; there is no way to determine how the system interpreted a particular ballot. However, the technology to support ballot-level audits is already available. I sometimes hear that risk-limiting audits are unappealing or elitist because ordinary people cannot understand the statistics. This strikes me as a red herring, especially if the alternative is blind reliance upon the voting system. Many implementations of risk-limiting audits require only multiplication, division, addition, and subtraction, and the basic principles are easy to explain (for more details, see Winning margin FIGURE 2 Counting estimates for three different audit methods at various winning margins, using California turnout data for the 2008 presidential election. The risk limit is 10%: in other words, if the reported outcome was wrong, these audit methods would have at least a 90% chance of leading to a full recount that would correct the outcome. The risk is figured using worst-case assumptions; the actual correction rates should be even higher February 2017 significancemagazine.com 21

5 bit.ly/2hzop91). The root principle is simplest of all: it is better to check the results than not. Evidence-based elections with risk-limiting audits show great promise. Colorado already has a statute requiring risk limiting audits (the implementation is still in progress). Los Angeles County and Travis County (Austin), Texas, are both working on state-of-the-art voting systems with excellent auditability and usability properties, and commercial vendors are offering improved support. At the same time, jurisdictions still face incentives not to bother with manual audits. It will likely require tenacious advocacy to obtain the evidence-based elections that we deserve. n Crisis? What crisis? Rebutting cumulative vote tally analysis By Mark Lindeman 1photo/Bigstock.com Election integrity discussions often feature striking claims about statistical evidence of fraud. For instance, in the summer of 2016, a report called An Electoral System in Crisis claimed to present convincing evidence that US electronic voting equipment in many areas throughout the country is not counting the votes accurately. In particular, Crisis asserted that in the 2016 Democratic primary election, Hillary Clinton had been given unearned votes in state after state. The report (bit.ly/2hslpsm), building on work by other authors, uses cumulative vote tally (CVT) graphs to support its allegations of probable fraud. These graphs which, to my knowledge, have never been used in the academic election forensics literature bear some explanation. Figure 3 is an example of a CVT graph, depicting votes cast in the 2016 Louisiana Democratic primary. To prepare this graph, first the 3921 precincts in Louisiana are sorted from the smallest (i.e., the fewest votes cast in the Democratic primary) to the largest (the most votes cast). Then, for each precinct in order from smallest to largest, the cumulative sum of votes cast is reckoned, along with the cumulative sums for each candidate. From these, the cumulative vote shares are calculated. The lines show how these vote shares change as more and more votes are included. For instance, statewide, 840 precincts contained 20 or fewer Democratic primary votes. These precincts contained just 8766 votes, of which Clinton received 4089, or about 47% (near the low point on the solid red line). As larger precincts are added, Clinton s cumulative vote share increases. When all the precincts are included, her overall vote share is above 70%. Meanwhile, Bernie Sanders s vote share decreases as larger precincts are added, although not as sharply. So what? In a nutshell, Crisis suggests that the increase in Clinton s cumulative vote share, as ever larger precincts are added, is caused by fraud. The conjecture is that in the larger precincts, votes have been taken from Sanders (and other candidates) and given to Clinton. How might these rising and falling lines evince fraud? Crisis offers the notion that votes are being shifted in large precincts because it is easier to disguise the differences than it would be in smaller 22 SIGNIFICANCE February 2017

6 precincts. However, a conventional scatterplot of the data, in which each point represents a precinct s results in the Democratic primary, suggests that if there was hacking, the hackers did not discriminate: Clinton s average vote share steadily increases with size, beginning even with the very smallest precincts. Indeed, the increase is steepest among smaller precincts (from 1 to 100 votes), tending to level off as size continues to increase. A far better explanation The Crisis report emphatically declares that demographics cannot account for the observed patterns in the CVT graphs. In many cases this claim probably is untestable, because precinct-level demographic data are not available. Louisiana, however, publishes precinct-level registration data by party and race. Crisis explicitly asserts that in Louisiana, a demographic explanation seems to evaporate when the data is carefully investigated. But this is not the case. In Figure 4 we examine the cumulative share of (self-identified) blacks among registered Democrats, alongside Clinton s cumulative share of votes. (This figure uses the same sort by primary votes cast, not number of registered Democrats for both shares.) It turns out that the smallest precincts in the primary also tended to be those with the smallest share of black voters among registered Democrats. The black share of registered Democrats increases in parallel with Clinton s vote share, but somewhat faster. Clearly the large precincts are very different than the small ones. Indeed, because Louisiana reports turnout by race, we can examine the relationship between the black share of Democratic primary voters and Clinton s share of the vote (see Figure 5). Linear regression analysis indicates that this one demographic variable can completely account for the apparent relationship between the number of votes cast and Clinton s vote share. There is no reason to postulate hacking. The Crisis report presents other reasons to be suspicious of the results, but none can withstand sceptical scrutiny. In short, there is no basis for using CVT graphs to analyse the legitimacy of vote counts. The Election Forensics Toolkit (bit.ly/2hsk6e0) is one place to explore more robust methods although no statistical test offers conclusive results. We would be far better off scrutinising physical evidence of voter intent, recorded on paper ballots. n Cumulative vote shares Cumulative votes cast FIGURE 3 Cumulative vote shares, small to large, in the 2016 Louisiana Democratic Primary. Adapted from Figure 9 in the Crisis report Cumulative shares Cumulative votes cast Clinton share Black share of Dems FIGURE 4 Clinton vote share and black share of registered Democrats in the the 2016 Louisiana Democratic primary Editor s note In the interests of transparency, we would like to point out that the Crisis report makes reference to an article published in June 2015 on the Significance website (bit.ly/2hsp5uv). The article, by Beth Clarkson, described how analysis of CVT graphs inspired by earlier work by Choquette and Johnson (bit.ly/2hsr0j6) had led her to suspect fraud in some US state elections. It also described her legal efforts to obtain the data necessary to conduct her own audits. Although Crisis refers to Significance as a journal, we would like to make clear that articles including Clarkson s are not formally peer-reviewed. Significance does not in any way endorse Clarkson s analysis, nor share in its interpretations. Here, Lindeman makes a strong case for why CVT graphs are not appropriate fraud detectors, and we have added a note to the Clarkson piece to point readers to this article. Brian Tarran Black share of Democratic primary voters FIGURE 5 Clinton vote share by black share of Democratic voters in the 2016 Louisiana Democratic primary. For readability, this graph excludes the smallest precincts, with ten or fewer votes February 2017 significancemagazine.com 23