I left The Miami Herald for Arizona State University more than four years ago. But in November, for the first time, I wished I were back working in The Herald newsroom. The reason was the dead-heat Florida presidential election. This political nightmare was a journalist's dream, and I knew that computer-assisted reporting would play a big role in the coverage. As the media frenzy began to build, I sent a 'gee, if there's anything I can do to help, just yell' e-mail to Herald city editor (and IRE chairman) Judy Miller.
My eagerness to have a piece of this story must have been evident. A few days later, Judy called. We talked about different angles The Herald troops were pursuing, including an analysis underway by CAR specialist Geoff Dougherty indicating that hundreds of felons had voted in violation of state law. But in light of the evident confusion in so many Florida polling places, the burning problem at the moment was 'discerning the mind of the voter' - particularly the clouded minds of the 185,000 or so Floridians who had gone to the polls but had failed for whatever reason to cast a valid ballot.
I did have an idea, albeit unformed, about how to attack the question. I had already been intrigued by how a smart statistician at Carnegie Mellon had demonstrated convincingly how vanishingly small was the probability that so many voters in Palm Beach County, home of the infamous 'butterfly' ballot, would deliberately vote for Pat Buchanan. We should gather precinct-level returns to see what they tell us.
I didn't have a very clear idea of what I might do with such data, but it somehow seemed useful at the time. It never occurred to me how staggeringly difficult it might be for The Herald to canvass all 67 counties, which were using all manner of voting methods and at least one of which was still using paper ballots. However, one thing Herald editors excel at is getting bright young reporters to work really hard when yet another 'Story of the Century' breaks in South Florida.
And so it happened with this one. Two reporters, Anabelle de Gale and Lila Aruza, had worked long hours on the phones wheedling elections departments across the state, pulling together what I so casually asked for. Sometimes they got it as a nice, neat e-mail attachment. More often, it came as a fax from technologically backward - or deliberately obstructive - officials that had to be tediously retyped into a spreadsheet.
Making a table As they completed each county, Anabelle and Lila e-mailed me the spreadsheets and I stitched those into one master Excel table that would cover the state's 5,885 precincts. For each precinct, the table showed the total vote for Bush, Gore, the combined other presidential candidates, and the total number of people who had cast ballots. By the middle of the week after Thanksgiving, the pressure had shifted to me and they wanted a story for that weekend.
And I didn't have anything to say. My favorite [reaction] came from the Bush spokesman who dismissed it all as 'voodoo statistics'. My initial plan had been to merge the results table with voter demographics from other sources, in the hope that we could make some definitive statements about who had been most affected by the ballot problems. But the demographic merge turned out to have huge problems caused by the varying ways each county named its precincts.
It was a story that would be done later by Geoff, but the necessary cleanup put it out of reach for that weekend. For a while, all I could envision was some sort of squishy look at such trivialities as the number of uncounted votes in each precinct. Eureka! Finally, inspiration struck. Years of working with both elections and census data taught me that people behave pretty much like their neighbors do.
A precinct, almost by definition, is a neighborhood. So it dawned on me to take the voting behavior of those who had successfully voted in each precinct, and just apply it proportionally to the uncounted votes. Thus, if 60 percent of the vote in a given precinct went to Bush, then I would give Bush 60 percent of the uncounted votes, and so on. At that point, the analysis became a straightforward spreadsheet exercise, simple enough that I'm going to have my students duplicate it this semester.
I did the formulas, copied them down the columns, hit the recalculation button, and saw the lead: Gore would have won by about 24,000 votes, instead of losing by about 500, if all those uncounted ballots had been cast correctly. Reporter Curtis Morgan began bouncing our results off different sources for reaction. My favorite came from the Bush spokesman who dismissed it all as 'voodoo statistics'; I think I'm gonna have a T-shirt made with that slogan.
Although we were still refining the findings, Curtis managed to craft a nicely readable account with these initial interviews. He also came up with the clever analogy of this analysis to a public opinion survey. Normally, a small sample is used to gauge the opinion of a large group. In this case, I was using the demonstrated behavior of the large group to project the behavior of a small subset. Some critics retorted that a perfect election is impossible, and that therefore my analysis was meaningless.
Others argued strenuously about the so-called 'protest vote' of those who go to the polls but deliberately don't vote in a race because they don't like any candidate.