“You should write a blog post about the Iowa Caucus app.” My mom told me over the phone.
She didn’t understand what went wrong with the app and if the Iowa Democratic Party had a backup count on paper, why we didn’t have the results yet. I won’t try to guess why we don’t have results yet of the caucus. The app’s failures however, intrigued me.
I’m not sure what I expected. From the news and online, it sounded like conspiracy theory backdoor deals to spin the results through technology. But after looking further into the debacle, I think the IowaReporterApp is a simple case of development failure.
While the app itself fell to the classic project development traps, I think it’s important to discuss the other management faux pas surrounding the project.
First, what the heck is a caucus and how does it work?
In the Iowa caucus there are no ballots or voting booths. Instead, “caucus-goers aren’t technically voting for candidates but rather for delegates who support a given candidate.” They do this by moving to the side of the room the delegate you want to support is.
However, there’s a catch. A candidate needs at least 15% of the people in the room in their corner to be locked in. If a candidate has less than 15%, those supporters need to participate in a process called “realignment”.
If you’re involved in realignment, you have a few options: Join a viable candidate’s group, convince other people from non-viable groups to support your candidate to hit the viability threshold, or go join another non-viable candidate other than the one you supported to hit the viability threshold.via CBS News
In realignment, people can:
- Join a different candidate’s group who has at least 15% viability.
- Convince other people from a non-viable group to join theirs.
- Join another non-viable candidate’s group to hit their viability
Once that is settled, the precinct leader how many delegates each candidate wins based on the support in the room.
Shadow Inc. and the Iowa Caucus App
The Iowa Caucus app was developed by the company, Shadow Inc. Shadow’s mission is to “build political power for the progressive movement by developing affordable and easy-to-use tools for teams and budgets of any size.” According to their website, they implemented technology for, “Hillary for America, Obama for America, Google, Kiva, Apple, the AFL-CIO, and the DNC.”
The app was designed to be simple. Precinct leaders log in and enter caucus counts. Those counts are sent to a Google server which are then sent to the Iowa Democratic Party (IDP). The IDP then ran the data through an “accuracy and quality check”. I was not able to find out what that meant.
Essentially, for the app’s part, you could tally precinct and delegate counts the same way on a spreadsheet.
|Delegate 1||Delegate 2||Delegate 3|
What went wrong
Early reports said a number of confusing things. We heard, “The underlying data and paper trail is sound”. The problem, we were told was in the reporting. The party “found inconsistencies in the reporting of three sets of results”.
If the app’s data was correct and uncompromised how could it be a reporting issue?
It turns out to be a simple case of failure in integration testing. Somewhere between sending the good data from Shadow’s Google and the IDP’s “quality check”, some of the data was not accepted.
“One of the main reasons there was so much confusion on caucus night, Niemira said, is because of a problem that occurred when Shadow tried to move the results it collected onto a verification system controlled by the Iowa Democratic Party (IDP). During that process, a data-formatting error caused it to not be accepted.”via Vice.com
According to Shadow CEO Gerard Niemira, the IowaReporterApp had faulty code that put the caucus data into a format that IDP couldn’t read during the independent verification step.
“we had some code that would look at our results database and then move that over to the IDP’s quality control check environment. In the process of doing that, we had some faulty code that took the data and put it into a format that made it fail the checks by the IDP.”Niemira via Vice
“We started our engagement with the IDP in August and began requirement gatherings and beginning to develop the app at that point, so we basically had the month of August, September, October, November, and December to do it, though requirements gathering takes a long time, so we didn’t have a final production version of this until pretty close to caucus time,” Niemira said.via Vice.com
Timeline of Events
Shadow Inc. and IDP’s engagement begins
August – December 2019
Shadow builds the app.
January 14, 2020
Reporters share their concerns about the IDP’s smartphone app and lack of security details.
Monday, February 3 2020
7 p.m. CT, the Iowa Caucus takes place.
Caucus chairs are unable to log into the app.
Tuesday, February 4, 2020
News of inconsistencies in reporting is released. It’s reported. “The underlying data and paper trail is sound”.
Wednesday, February 5, 2020
The name of the app, IowaReporterApp, and development company, Shadow Inc. is released.
Thursday, February 6, 2020
Cybersecurity experts report on the code the analyzed from the IowaReporterApp.
The IowaReporterApp and the IDP are a great lesson in why IT projects fail. The failure in reporting the results itself is something that Shadow was able to fix with code. Shadow and the IDP are not able to fix the faith voters lost in an system they already had doubts in.
IT projects on the outset always seem achievable. Everyone seems to be on the same page that technology is there to help them. I think Niemira understood that too in creating the Iowa Caucus App. He said an interview with Vice, “The point of this app was to help temporary precinct chairs do the math and get good results in the room and speed up the process, help them basically.”
IT projects that start out with simple designs and good intentions face the onlaought of budget restraints, time constrictions, and resources. It seems like Shadow is no different.
As we determined, the app is a tool to help temporary precinct chairs tally counts. “For something like this, you don’t want to introduce complexity where there doesn’t need to be any,” Niemera said to Vice. He says the IowaReporterApp, “is a relatively simple function, it’s basically a calculator, so that’s the approach we took to it.”
According to cyber security experts, “the app was clearly done by someone following a tutorial.” Said Kasra Rahjerdi. “It’s similar to projects I do with my mentees who are learning how to code,” he said.
Other design flaws:
- Many caucus chairs were unable to log into the app. It required multiple PIN numbers, logins and text messages to get set up. Caucus members were seen taking photos of the pins to text for help, some even Tweeting them out.
- Other precinct chairs were able to send in their numbers through the app, it just went slowly.
While the evidence suggests Shadow Inc. definitely shipped a buggy app, the Iowa Democratic Party screwed up too.
I recently finished reading Bad Blood – the story behind Theranos and CEO Elizabeth Holmes. Holmes promised a multitude of blood tests and real time results with only a finger prick of blood. She promised this with her machines the Edison and minilab.
Holmes raised millions of dollars and was the first self-made woman billionaire. She had an impressive board with people like General Mattis, Henry Kissinger, and George Shultz.
However, the company was shrouded in mystery. Holmes didn’t let anyone see how her lab or machines actually worked. Even many of Theranos’ own employees never saw the inside of one of the machines.
I couldn’t help but think of this when I read about the Iowa Democratic Party. No one was sure how the app worked because the DNC kept everything secret. The DNC claimed the secrecy was for cybersecurity reasons but when it was clear the Iowa Caucus was in trouble, no one knew the name of the app or the company that built it to find out what went wrong.
Anyone outside of IDP leaders had no way to verify it was “just a reporting issue”. This compounded with the news that Pete Buttigieg’s campaign donated to Shadow Inc. cast the results into even murkier waters.
Those aware of the apps development warned the DNC that transparency apps like these are essential for user trust. Users are even more important in this case, as they are the American people voting in what they believe is a just system (although for some that trust is shaken).
“We were really concerned about the fact there was so much opacity. I said over and over again trust is the product of transparency times communication. The DNC steadfastly refused to offer any transparency. It was hard to know what to expect except the worst,” Greg Miller, cofounder of the Open Source Election Technology Institute, which publicly warned the IDP against using the app weeks ago, told Motherboard.via Vice.com
When there is a new piece of election technology, people want to know the general security details behind it. Miles Parks, who runs election security for NPR, said in an interview, “We know very little about the specifics of this app. We don’t know who developed it or who wrote the code.” They didn’t even know what types of security tests were run on it.
This lack of transparency, was on purpose as the Democratic Party believed giving up this information would help hackers. “But”, said Parks, “experts actually say that that secrecy doesn’t help against hacking at all.”
Betsy Cooper, a cybersecurity policy expert at the Aspen Institute, agrees with Parks. She said in the same interview, “Basic transparency around how [the app] was built, how up-to-date the security of the app is and how it’s been tested all could be made publicly available with little cost to the DNC.” Unfortunately, the DNC or IDP never shared their security test results.
After the news broke and the IowaReporterApp was unveiled, cybersecurity companies looked to get a copy of the app to see what went wrong.
Blue Hexagon is one such company. In an interview with CNET, Irfan Asrar (Blue Hexagon’s head of cyberthreat intelligence and operations), counted many problems. Asrar believes the problems are results of “the app being rushed into production,”.
Asrar said, “The larger concern is that the app was so easy to obtain, which means anyone could access the infrastructure supporting it and potentially cause damage,”.
“Our message is that apps like this should be developed in the sunlight, and part of an open bug bounty.”Former Facebook chief security officer Alex Stamos via Vice.com
I called my mom back with the information I discovered. She said it made a little more sense but still left her uneasy. She said, “if The Voice can do a voting app why can’t IDP do it? Maybe they should have hired The Voice to do their app.”
After reading about the the IowaReporterApp and what is was supposed to do, I can’t help but wonder, did the IDP ever need an app at all?
Coffee = Blog Fuel
If you find joy and value in what I write, please consider donating by “buying me a coffee”.