Checking It Twice
Election officials have had no practical way to guarantee a correct ballot count—until now
Counting is hard. Neither people nor machines seem to be able to do it reliably. And that’s a nightmare for election officials who need an accurate ballot count to decide elections.
Eighteen states require officials to double-check the machine counts by hand for a portion of the ballots. But election officials have had little guidance on what to do with the recount results. If the election is close and the recount finds a few errors, should a registrar call for a larger recount or go ahead and certify the result? Most laws left it to their discretion.
Now Philip Stark, a statistician at the University of California, Berkeley, has developed a recount method that guarantees a 99 percent chance that the result is the same as it would be with a full hand count. Several counties in California plan to try out the method on ballot measures during the presidential primaries this year. If this trial and others go smoothly, California could adopt the method statewide.
The idea behind hand recounts is that even though people are not necessarily more accurate than machines, their mistakes tend to be different. Machine counts have problems because of software errors, because the voters used a kind of ink the optical scanners couldn’t read, or because a memory card or stack of ballots never made it to be counted. Hand counts go wrong when someone loses track of the count, a paper jam disrupts the printing of the paper ballots, or a box of paper ballots disappears. Counting the ballots both ways can uncover the errors in each method.
Recounting by hand is expensive, though, and if an election is a landslide, a full recount probably isn’t necessary. In that case, a random check of a few precincts to catch large-scale fraud or foul-ups will suffice. But in a close race, a few miscounted ballots could decide the outcome.
As part of a complete review of California’s election procedures, election officials turned to Stark to develop a method to determine just how big a recount is necessary.
“No flat percentage, short of 100 percent, gives high confidence in all circumstances,” Stark says. The appropriate percentage depends on the number of precincts and ballots, the size of the apparent margin of victory, and the number of mistakes the recount finds. A very close race with only a few ballots and lots of mistakes will require a full recount. For many races, though, a recount of 1 percent of precincts, randomly chosen—the number currently required by California law—will be enough to generate 99 percent confidence.
“There are going to be mistakes,” he says. “There are always mistakes.” The question is just whether there are few enough mistakes in the 1 percent sample to suggest that the election results are very likely right. He first figures out how many mistakes there would have to be in all the ballots to change the election result. Then he counts the number of mistakes in his sample. If he sees very few errors in his sample, it is possible but unlikely that all the ballots would have had enough mistakes to change the election result. Standard techniques from statistics quantify just how unlikely that is. If the probability that the outcome is correct is less than 99 percent chance, Stark keeps counting, including additional randomly chosen precincts. He stops only when he’s generated sufficient confidence or he’s counted all the ballots.
Stark’s method addresses some of the concerns about the trustworthiness of electronic voting machines. “Regardless of how arcane, esoteric, or just wrong the machine’s mistakes might be—whether it is a simple programming bug or deliberate fraud—you can guarantee that you have good chance of finding it, if it’s big enough to alter the outcome” with this method, Stark says. “That assumes, though, that the mistake doesn’t simultaneously affect the audit trail.” He also points out that it also necessitates that the machines reliably produce such a paper trail, which they often don’t.
The 2000 presidential race was so close in Florida, Stark says, that a full statewide recount would almost certainly have been necessary to guarantee the correct outcome with high confidence. He points out that that race was extraordinarily close, as shown by a thought experiment. He imagined that every person in Florida who voted in that election had decided how to vote by tossing a coin. For that scenario, he then calculated that there would have been an 82 percent chance that the margin between the two candidates would have been larger than it was in real life.
A federal bill (H.R. 811) is currently under review in the House of Representatives that would require audits of federal elections and mandate that all electronic voting machines produce a paper trail for audits. In a close race, the bill would require a larger percentage of ballots to be checked. But because it doesn’t take into account how many errors have been found, it can’t generate a 99 percent confidence rate, as Stark’s method does.
If you would like to comment on this article, please see the blog version.
