Diebold Was Aware of Shortcomings Early
Company knew that vote-counting software was flawed
By Ian Hoffman
Tuesday, April 20, 2004 - Attorneys for Diebold Election Systems Inc. warned in late November that its use of uncertified vote-counting software in Alameda County violated California election law and broke its $12.7 million contract with Alameda County.
Soon after, a review of internal legal memos obtained by ANG Newspapers shows, Diebold's attorneys at the Los Angeles office of Jones Day realized the McKinney, Texas-based firm also faced a threat of criminal charges and exile from California elections.
Yet despite warnings from the state's chief elections officer, Diebold continued fielding poorly tested, faulty software and hardware in at least two of California's largest urban counties during the Super Tuesday primary, when e-voting temporarily broke down and voters were turned away at the polls.
Other documentation obtained by ANG shows that the latest approved versions of Diebold's vote-counting software in this state cast doubt on the firm's claims elsewhere that it has fixed multiple security vulnerabilities unearthed in the last year.
"In California those issues can be addressed," said Diebold spokesman David Bear. "They were addressed in Maryland, and they could be changed in California."
California elections officials said they are perplexed that Diebold apparently has not changed practices since a December audit revealed uncertified software running in every county that it serves.
"Diebold may suffer from gross incompetence, gross negligence. I don't know whether there's any malevolence involved," said a senior California elections official who spoke on condition of anonymity. "I don't know why they've acted the way they've acted and the way they're continuing to act. Notwithstanding their rhetoric, they have not learned any lessons in terms of dealing with this secretary (of state)."
The memos show that for months, Diebold attorneys at Jones Day have been exploring ways to keep the nation's second-largest electronic voting provider from losing an eighth of the national market.
Jones Day partner Daniel D. McMillan declined to comment on the content of the documents except to confirm they were internal papers from his office. He warned against drawing conclusions from the firm's memos.
Diebold's legal team appears to have been exploring whether California Secretary of State Kevin Shelley has the power to investigate the company's practices. The memos reflect an argument that the regulations by which California approves voting equipment for elections may never have been properly codified and are unenforceable.
Diebold's Bear said his company is cooperating with Shelley's office.
"I've been working with the SOS, and we're hopeful we can move forward and the advantages of electronic voting can be continued to be offered to the citizens of California," he said. "We will continue to work with state and local elections officials to address any and all elections issues."
The law firm's memos reflect a corporate defense firm on a $500,000-a-month campaign to protect Diebold.
It is a critical moment for Diebold, for electronic voting in California and for at least some of the 19 counties statewide that purchased Diebold voting systems for more than $50 million.
On Wednesday, state elections officials begin debating their advice to Shelley on whether to disallow some or all Diebold voting systems, or all touch-screen voting machines, from the November elections.
What Shelley decides will be a test of state authority over makers of the computers that will determine the electoral votes in California and other states. His decision also could send two of California's largest counties -- Alameda and San Diego -- scrambling for other ways to count votes six months from now.
Voting experts say the industry's factories and printing plants probably can handle the extra demand for replacement voting machines and paper ballots, given at least three months' notice. But Shelley's decision also could unleash a barrage of lawsuits that could mire orders of equipment and ballots in legal wrangling over who will pay for them.
At the center of those battles will be Jones Day. The firm's internal memoranda show its attorneys considered the idea of calling a new bit of uncertified voting software "experimental." State rules say local governments can use entire experimental voting systems without state approval.
The lawyers also presented California officials who were seeking documents from Diebold with sweeping confidentiality agreements designed to hide flaws in Diebold software as much as its intellectual property.
In drafts of a Feb. 13 letter to state regulators, Diebold's attorneys declared that Diebold makes no changes to electronic devices that the company and its predecessor have been programming for at least five years.
The drafts show they staked out a firm position that a critical piece of Diebold's voting system -- its voter-card encoders -- did not need national or state approval because they were commercial off-the-shelf products, never modified by Diebold.
But on the same day the letter was received, Diebold-hired techs were loading non-commercial Diebold software into voter-card encoders in a West Sacramento warehouse for shipment to Alameda and San Diego counties.
"They were still crunching and working on that software in the middle of February," said James Dunn, who worked as an assembly technician in Diebold's Sacramento warehouse.
More than 600 of the devices froze or displayed unfamiliar screens and error messages on the morning of Election Day, for failure rates of 24 percent in Alameda County and about 40 percent in San Diego County.
Diebold Elections executives were told in October by state officials to ensure every piece of its voting systems was fully tested and approved by national and state authorities.
But Diebold resisted, arguing that the encoders did not need testing and approval because they were a "peripheral" device on its voting systems and that the devices were common, commercial products.
That was true for the hardware. But not the software.
In fact, Diebold engineers were writing and rewriting the software at DESI headquarters in Texas and in Sacramento, supplying the latest versions two weeks before the encoders failed at high rates in the Super Tuesday presidential primary.
Diebold eventually sent a sample of the encoders to an outside laboratory, but it did not have time for more than cursory testing.
The encoders were the only way that poll workers were trained to create cards that let voters call up digital ballots on Diebold's touch-screen machines at more than 2,000 polling places in Alameda and San Diego counties. Dunn says he is not surprised.
As he and other techs raced to assemble the encoders out of tablet-PC screens, batteries and card-writing bases shipped to Sacramento from factories in Asia, Diebold officials kept supplying new versions of the software.
In addition, the hardware components often failed to mate well, resulting in frozen screens. And when the batteries lost power, the devices lost their internal clock and operating settings, often Diebold's software as well.
Dunn blames Diebold's rush to get the devices into the March2 elections and the lack of standard quality controls in assembling and configuring them. No instructions, no checklists, no tracking system.
An outspoken tech complained about the poor quality controls and the failure of the devices when sapped of power.
"He was gone. They fired him," Dunn said. "The attitude among the others there was, 'I don't care how screwed up these things are, I'm going to keep quiet. I'm not going to get fired.'"
A Diebold software engineer pressed her superiors to allow testing of all the devices before they were shipped to Oakland, San Diego and elsewhere, but the tests -- successful creation of voter cards -- were performed only on the last 10 percent to 15 percent of the devices, Dunn said.
"I got the feeling that the whole thing was rushed, that the products were brought to market too fast, and they did it because they had to get products to these counties before the election and they weren't ready," he said. "It wasn't fully developed. It was still prototyped, and they were out of time."
Alameda County had paper provisional ballots on hand at polling places for use in lieu of the disabled touch screens. At least 14 polling places ran out and turned away voters. San Diego County relied on one of Diebold's latest features, electronic provisional ballots, so larger numbers of voters were turned away at the polls.
Diebold's claims to California elections officials, through its attorneys, that it does not modify the encoder software is blatantly untrue, according to Dunn and electronic-voting opponent Jim March.
"That's a lie," March said.
Last year, Seattle-based journalist Bev Harris found nine versions of Diebold encoder software on an unsecure Internet site. Software engineers such as March have been marveling at their multitude since.
"When you vote, you are inserting a memory card containing up to 128k of God-only-knows what. With no oversight, the 'smart cards' could contain some very stupid stuff indeed, or even deliberate subversion," he said.
Contact Ian Hoffman at firstname.lastname@example.org .
add a comment on this article