A risk-limiting audit (RLA) is a post-election tabulation auditing procedure which can limit the risk that the reported outcome in an election contest is incorrect. It generally involves (1) storing voter-verified paper ballots securely until they can be checked, [1] and (2) manually examining a statistical sample of the paper ballots until enough evidence is gathered to meet the risk limit. [2] [3]
Advantages of an RLA include: samples can be small and inexpensive if the margin of victory is large; [2] there are options for the public to watch and verify each step; [1] and errors found in any step lead to corrective actions, including larger samples, up to a 100% hand count if needed. Disadvantages include: the sample needs to be a large fraction of all ballots to minimize the chance of missing mistakes, if any contest is close; and it is hard to check computer totals publicly, except by releasing computer records to the public. [4] If examining sampled ballots shows flaws in ballot storage, the usual approach cannot recover correct results, [5] and researchers recommend a re-vote if the number of ballots held in flawed storage is enough to change winners. [6] An alternative to re-votes is to create and verify backups of the paper ballots soon after they are voted, so there is an alternative to flawed storage of the original ballots.
As with other election audits, the goal is to identify not only intentional alterations of ballots and tallies, but also bugs in election machines, such as software errors, scanners with blocked sensors [7] or scanners skipping some ballots. The approach does not assume that all ballots, contests or machines were handled the same way, in which case spot checks could suffice. The sample sizes are designed to have a high chance of catching even a brief period when a scratch or fleck of paper blocks one sensor of one scanner, or a bug or hack switches votes in one precinct or one contest, if these problems affect enough ballots to change the result.
Comparisons can be done ballot-by-ballot or precinct-by-precinct, though the latter is more expensive. [8]
There are three general types of risk-limiting audits. [8] Depending on the circumstances of the election and the auditing method, different numbers of ballots need to be hand-checked. For example, in a jurisdiction with 64,000 ballots tabulated in batches of 500 ballots each, an 8% margin of victory, and allowing no more than 10% of any mistaken outcomes to go undetected, method 1, ballot comparison, on average, needs 80 ballots, method 2, ballot polling, needs 700 ballots, and method 3, batch comparison, needs 13,000 ballots (in 26 batches). [8] The methods are usually used to check computer counts, but methods 2 and 3 can also be used to check accuracy when the original results were hand-counted. [1] The steps in each type of risk-limiting audit are:
All methods require:
The last three items are hard in one-party states, where all participants may be swayed by the ruling party.
Hand-checking ballots (method 1) identifies bugs and hacks in how election computers interpret each ballot, so computer processing can be improved for future elections. Hand-counting ballots (methods 2 and 3) bypasses bugs and hacks in computer counts, so it does not identify exactly what mistakes were made. Independently totaling cast vote records (method 1) or batch totals (method 3) identifies bugs and hacks in how election computers calculate totals. Method 2 does not need this independent totaling step, since it has a large enough sample to identify winners directly.
Colorado uses method 1 in most counties, and method 2 in a few counties which use election machines which do not record and store "cast vote records". Colorado uses no audit method in two counties which hand-count ballots in the first place. [10]
Risk-limiting audits are a results audit to determine if votes were tabulated accurately, not a process audit, to determine if good procedures were followed. [11]
The process starts by selecting a "risk limit", such as 9% in Colorado, [12] meaning that if there are any erroneous winners in the initial results, the audit will catch at least 91% of them and let up to 9% stay undetected and take office. [8] [2] Another initial step is to decide whether to audit: all contests; a random sample of contests, allowing a known risk that erroneous winners will take office; or a non-random sample, so no statistical confidence is available on the non-audited contests. Based on a formula, a sample size is determined for each contest being audited. [13] [8] [14] The size of the sample depends primarily on the margin of victory in the targeted contest.
A random starting point (seed) is chosen by combining information from multiple independent people, [9] to create a series of random numbers identifying specific ballots to pull from storage, such as the 23rd, 189th, 338th, 480th ballots in precinct 1, and other random numbers in other precincts.
When storage is opened, records are checked to see if each sampled precinct still has the same number of ballots recorded during the election, if correct numbers appear on seals, if machines or containers have been tampered with in any way, and/or other methods to check if ballots have avoided intrusion. [15] [8] [2] If ballots have not been stored successfully, advocates of risk-limiting audits say there should be a re-vote, [6] [8] or no result should be declared, [16] which usually requires a re-vote, or results can be declared if "the number of questionable or missing audit records is small enough that they cannot alter the outcome of the contest." [15] [8] However, if storage or records are flawed, laws may require initial results to be accepted without audit. [17] To provide an alternative to a re-vote, seven Florida counties back up the paper ballots by copying them the day after they are voted, with machines independent of election machines. [18] While any copy can have flaws, comparing cast vote records to these independent backup copies would give an alternative to re-voting or skipping the audit when storage is not trustworthy. Florida does not hand-check this backup, which would be required by a risk-limiting audit. Instead Florida machine-audits 100% of votes and contests. They have found discrepancies of 1-2 ballots from official machines. [19] Maryland has a less safe alternate approach. Maryland's election machines create and store ballot images during the election, separate from the cast vote records. Most election machines do so. [20] Maryland compares cast vote records to these ballot images from the same election machines. [21] Unlike Florida, this approach is not an independent backup or check. A hack or bug in the election machine can alter, skip, or double-count both image and cast vote record simultaneously. [22] Maryland's semi-independent checking is better than no checking, since it has found and resolved discrepancies, such as folded ballots leaving fold lines on the images, which computers interpreted as write-in votes; sensor flaws which left lines on the images, interpreted as overvotes; and double-feeds where two ballots overlap in the scanner, and one is uncounted. [23]
When an audit produces the same result as initial election results, the outcome is confirmed, subject to the risk limit, and the audit is complete. If the audit sample shows enough discrepancies to call the outcome into question, a larger sample is selected and counted. This process can continue until the sample confirms the original winner, or a different winner is determined by hand-counting all ballots. [8]
Sample sizes rise rapidly for narrow margins of victory, with all methods. In a small city or county, with 4,000 ballots, method 1, ballot comparison, would need 300 ballots (300–600 minutes, as discussed in Cost below) for a contest with a 2% margin of victory. It would need 3,000 ballots (50-100 staff hours in the city or county) for a 0.1% margin of victory. Method 2 or 3, ballot polling or batch comparison, would need a full hand count of the 4,000 ballots (70-130 staff hours). Margins under 0.1% occur in one in sixty to one in 460 contests.
Large numbers of contests on a ballot raise the chances that these small margins and large samples will occur in a jurisdiction, which is why no place does risk-limiting audits on all contests, leaving most local government races unaudited, though millions of dollars are at stake in local spending [24] and land use decisions. [25] Colorado picks contests with wider margins to avoid large samples. [13] California's rules for 2019–2021 require any RLA to audit all contests, [26] and no election offices have chosen to use RLAs under these rules.
The power of the sample also depends on staff expanding the audit after any discrepancy, rather than dismissing it as a clerical error, [27] or re-scanning problematic ballots to fix just them. [28]
When Maryland evaluated audit methods, it noted that local boards of elections could not budget, or plan staffing, for risk-limiting audits, since the sample "is highly dependent on the margin of victory in any given audited contest... A very close margin of victory could... require days of staff work, possibly compromising the local certification deadline." [21]
An alternative to large samples is to audit an affordable sample size, and let the risk limit vary instead of the sample size. [22] For a fixed sample, closer margins of victory would have more risk of letting erroneous winners take office, but any substantial sample would still have a known substantial chance of catching errors. Election managers would announce the level of confidence provided by the sample, and would have procedures to follow up if the sample finds one or more errors. [22]
The sample sizes presented will be enough to confirm a result, subject to the risk limit, when the apparent winner is the actual winner. If the sample does not confirm the win, more ballots are sampled, up to a 100% hand count to confirm a different winner. [8]
Ballots are at risk when being transported from drop boxes and polling places to central locations, and may be protected by GPS tracking, [29] guards, security systems, [30] and/or a convoy of the public. [31]
No US state has adequate laws on physical security of the ballots. [32] Security recommendations for elections include: starting audits as soon as possible after the election, regulating access to ballots and equipment, [33] having risks identified by people other than those who design or manage the storage, using background checks and tamper-evident seals. [34] [8] However seals on plastic surfaces can typically be removed and reapplied without damage. [35] [36]
Experienced testers can usually bypass all physical security systems. Security equipment is vulnerable before and after delivery. Insider threats and the difficulty of following all security procedures are usually under-appreciated, and most organizations do not want to learn their vulnerabilities. [37]
Method 1 requires the ballots to be kept in strict order so one can compare the computer interpretations of sampled ballots with those exact physical ballots. [38] If the correct ballots are present, but out of order, method 2 can be used. Maryland, like other states, randomizes the order of paper ballots and cast vote records to protect ballot secrecy, so method 1 cannot be done there, since paper ballots and cast vote records cannot be compared. [21]
All the methods, when done for a state-wide election, involve manual work throughout the state, wherever ballots are stored, so the public and candidates need observers at every location to be sure procedures are followed. [1] However, in Colorado and most states the law does not require [ broken anchor ] any of the audit work to be done in public. [39]
All methods are designed to be independent of the election software, to ensure that an undetected error in the election software can be found by the audit. The audit in practice is dependent on its own software, separate from the election system. Election staff examine ballots and enter staff interpretations into an online software tool, which is supposed to handle the comparison to the voting system interpretation, report discrepancies, and tell staff whether to sample further. [40] [41] It is also hard to prepare the list of ballots to sample from (ballot manifest) without using information from the election system.
Method 1, ballot comparison, requires a second step, besides checking the sample of ballots: 100% of the computer interpretations of ballots ("cast vote records") need to be re-tabulated by computers independent of the original election computers. [2] This re-tabulation checks whether election computers tallied the cast vote records correctly. Like any computer step this independent tally is subject to hacks and bugs, especially when voting rules are complex, such as variations in the number of candidates from different districts to vote for. The reason for the re-tabulation step is that independently programming a different kind of machine provides an independent check on official election machines.
While all methods require physical security on the paper ballots, method 1 also requires enough security on the cast vote records so no one can change them. This can be accomplished by computer-calculating, storing and comparing a hash code for each file of cast vote records: [42] (a) right after the election, (b) when independent tabulation is done, and (c) when ballot comparison is done. [22]
Colorado says it has a system to do the independent count of cast vote records, but it is not yet publicly documented, [43] so the chance of bugs or hacks affecting this independent computer at the Secretary of State's office along with one or more of the election machines is unknown.
California's process for risk-limiting audits omits the step of independent totals. [44] When it did a pilot, independent totals were calculated by a student on a university computer. [45]
Cost depends on pay levels and staff time needed, recognizing that staff generally work in teams of two or three (one to read and one or two to record votes). Teams of four, with two to read and two to record are more secure [46] and would increase costs.
Each minute per vote checked means 25 cents per vote at $15/hour, or $250 per thousand votes. Checking random ballots can take more time: pulling individual ballots from boxes and returning them to the same spot. It is relevant to methods 1 and 2.
[ needs update ]
As of early 2017, about half the states [ broken anchor ] require some form of results audit. [39] Typically, these states prescribe audits that check only a small flat percentage, such as 1%, of voting machines. As a result, few jurisdictions have samples large or timely enough to detect and correct tabulation errors before election results are declared final. [47] [48]
In 2017, Colorado [49] became the first state to implement ballot comparison audits, auditing one contest, not randomly chosen, in each of 50 of its 64 counties, [50] several days after the election. Following the 2018 General Election, Colorado will conduct audits in the 62 of its 64 counties that use automated vote counting equipment (the two remaining counties hand count the ballots).
Rhode Island passed legislation [51] requiring that state's Board of Elections to implement risk-limiting audits beginning in 2018. Individual jurisdictions elsewhere may be using the method on the local election clerks' initiative.
In 2018 the American Statistical Association, Brennan Center for Justice, Common Cause, Public Citizen and several election integrity groups endorsed all three methods of risk-limited audits. Their first five criteria are: [1]
In 2014, the Presidential Commission on Election Administration recommended the methods in broad terms:
By selecting samples of varying sizes dictated by statistical risk, risk-limiting audits eliminate the need to count all the ballots to obtain a rapid test of the outcome (that, is, who won?), while providing some level of statistical confidence.
In 2011, the federal Election Assistance Commission initiated grants for pilot projects to test and demonstrate the method in actual elections. [53]
Professor Phillip Stark of the University of California at Berkeley has posted tools for the conduct of risk-limiting audits on the university's website. [54]
A voting machine is a machine used to record votes in an election without paper. The first voting machines were mechanical but it is increasingly more common to use electronic voting machines. Traditionally, a voting machine has been defined by its mechanism, and whether the system tallies votes at each voting location, or centrally. Voting machines should not be confused with tabulating machines, which count votes done by paper ballot.
Electronic voting is voting that uses electronic means to either aid or take care of casting and counting ballots including voting time.
Electoral fraud, sometimes referred to as election manipulation, voter fraud, or vote rigging, involves illegal interference with the process of an election, either by increasing the vote share of a favored candidate, depressing the vote share of rival candidates, or both. It differs from but often goes hand-in-hand with voter suppression. What exactly constitutes electoral fraud varies from country to country, though the goal is often election subversion.
Electronic voting is the standard means of conducting elections using Electronic Voting Machines (EVMs) in India. The system was developed for the Election Commission of India by state-owned Electronics Corporation of India and Bharat Electronics. Starting in the late 1990s, they were introduced in Indian elections in a phased manner.
Vote counting is the process of counting votes in an election. It can be done manually or by machines. In the United States, the compilation of election returns and validation of the outcome that forms the basis of the official results is called canvassing.
Voter verifiable paper audit trail (VVPAT) or verified paper record (VPR) is a method of providing feedback to voters who use an electronic voting system. A VVPAT allows voters to verify that their vote was cast correctly, to detect possible election fraud or malfunction, and to provide a means to audit the stored electronic results. It contains the name and party affiliation of candidates for whom the vote has been cast. While VVPAT has gained in use in the United States compared with ballotless voting systems without it, hand-marked ballots are used by a greater proportion of jurisdictions.
A DRE voting machine, or direct-recording electronic voting machine, records votes by means of a ballot display provided with mechanical or electro-optical components that can be activated by the voter. These are typically buttons or a touchscreen; and they process data using a computer program to record voting data and ballot images in memory components. After the election, it produces a tabulation of the voting data stored in a removable memory component and as printed copy. The system may also provide a means for transmitting individual ballots or vote totals to a central location for consolidating and reporting results from precincts at the central location. The device started to be massively used in 1996 in Brazil where 100% of the elections voting system is carried out using machines.
Punchscan is an optical scan vote counting system invented by cryptographer David Chaum. Punchscan is designed to offer integrity, privacy, and transparency. The system is voter-verifiable, provides an end-to-end (E2E) audit mechanism, and issues a ballot receipt to each voter. The system won grand prize at the 2007 University Voting Systems Competition.
End-to-end auditable or end-to-end voter verifiable (E2E) systems are voting systems with stringent integrity properties and strong tamper resistance. E2E systems use cryptographic techniques to provide voters with receipts that allow them to verify their votes were counted as cast, without revealing which candidates a voter supported to an external party. As such, these systems are sometimes called receipt-based systems.
An optical scan voting system is an electronic voting system and uses an optical scanner to read marked paper ballots and tally the results.
An election recount is a repeat tabulation of votes cast in an election that is used to determine the correctness of an initial count. Recounts will often take place if the initial vote tally during an election is extremely close. Election recounts will often result in changes in contest tallies. Errors can be found or introduced from human factors, such as transcription errors, or machine errors, such as misreads of paper ballots.
Quick count is a method for verification of election results by projecting them from a sample of the polling stations.
The Voluntary Voting System Guidelines (VVSG) are guidelines adopted by the United States Election Assistance Commission (EAC) for the certification of voting systems. The National Institute of Standards and Technology's Technical Guidelines Development Committee (TGDC) drafts the VVSG and gives them to the EAC in draft form for their adoption.
Electronic voting by country varies and may include voting machines in polling places, centralized tallying of paper ballots, and internet voting. Many countries use centralized tallying. Some also use electronic voting machines in polling places. Very few use internet voting. Several countries have tried electronic approaches and stopped because of difficulties or concerns about security and reliability.
An election audit is any review conducted after polls close for the purpose of determining whether the votes were counted accurately or whether proper procedures were followed, or both.
The Verified Voting Foundation is a non-governmental, nonpartisan organization founded in 2004 by David L. Dill, a computer scientist from Stanford University, focused on how technology impacts the administration of US elections. The organization's mission is to "strengthen democracy for all voters by promoting the responsible use of technology in elections." Verified Voting works with election officials, elected leaders, and other policymakers who are responsible for managing local and state election systems to mitigate the risks associated with novel voting technologies.
Voatz is a for-profit, private mobile Internet voting application. The stated mission of Voatz is to "make voting not only more accessible and secure, but also more transparent, auditable and accountable." The company is headquartered in Boston, Massachusetts.
Electronic voting in the United States involves several types of machines: touchscreens for voters to mark choices, scanners to read paper ballots, scanners to verify signatures on envelopes of absentee ballots, and web servers to display tallies to the public. Aside from voting, there are also computer systems to maintain voter registrations and display these electoral rolls to polling place staff.
The 2021 Maricopa County presidential ballot audit, commonly referred to as the Arizona audit, was an examination of ballots cast in Maricopa County during the 2020 United States presidential election in Arizona initiated by Republicans in the Arizona State Senate and executed by private firms. Begun in April 2021, the audit stirred controversy due to extensive previous efforts by former president Donald Trump and his allies to overturn the election, and due to assertions of rule violations and irregularities in the conduct of the recount, leading to claims that the audit was essentially a disinformation campaign. In June 2021, Maggie Haberman of The New York Times and Charles Cooke of National Review reported that Trump had told associates that based on the results of the audit, he would be reinstated as president in August 2021.
VotingWorks is a nonprofit organization that creates and sells open-source voting systems in the U.S. They currently have three products: one for casting and counting ballots, another, named Arlo, for risk-limiting audits (RLAs), and a third for accessible at-home voting.
{{cite web}}
: CS1 maint: multiple names: authors list (link){{cite web}}
: |first=
has generic name (help)CS1 maint: multiple names: authors list (link){{cite journal}}
: CS1 maint: multiple names: authors list (link){{cite web}}
: CS1 maint: multiple names: authors list (link){{cite book}}
: CS1 maint: location missing publisher (link){{cite web}}
: CS1 maint: multiple names: authors list (link){{cite web}}
: CS1 maint: multiple names: authors list (link){{cite web}}
: CS1 maint: multiple names: authors list (link){{cite web}}
: CS1 maint: multiple names: authors list (link){{cite journal}}
: CS1 maint: multiple names: authors list (link)