The data entry ordeal
What to do when you don't have unfailingly accurate slave labor
By Deirdre Shesgreen MICAR
Is the task of putting a few hundred thousand records on computer stopping you from cracking that great campaign finance story?
Data entry is not most investigative journalists' idea of a good time - but often it comes with the territory.
Hiring a data entry service is a solution, but they come with problems of their own.
The Kansas City Star learned the hard way while investigating the abuse of Missouri's "Second Injury" worker's compensation fund.
"We tried a so-called professional typist to do this, and it was disaster," said Rich Hood, the Star's political correspondent. "Among other things, she left out the contribution amounts. She went like a bat out of hell, but it was unusable."
the Star spent hundreds of hours entering the data itself. "We checked and rechecked the accuracy of the thing so we could stand behind the story," Hood said.
If the government wants to create a database, they can go to the nearest prison and enlist inmates to do the tedious task.
"Two ways that government inputs info into computers is through civil servants and prison inmates," said John Roure, who runs Optic Solution, a company that builds databases for law-enforcement agencies, unions, PACs and other associations.
Inmates at the Texas Department of Criminal Justice have done data entry for workers compensation commission reports, records from the Department of Transportation and other agencies, said Phillip Anderson, industrial supervisor of the Huntsville prison.
But the government mainly uses prisoner labor for non-sensitive data entry, Roure said.
Unfortunately, newspapers don't have access to the same human or financial resources. So enthusiastic and dedicated reporters end up spending months in front of a computer screen, coming away with a great story but glazed eyes.
One source of cheap labor newspapers do have access to is off-shore data entry companies. Though using a plant in Mexico that pays its employees $4 a day may raise some ethical eyebrows, it is cheaper than the U.S. firms.
Martin Horan, who runs Horan Data Services Inc., in Cincinnati, just lost a bid to an off-shore data entry company.
"We really can't compete with their prices," Horan said.
A local firm wanting to keep its business in the United States asked Horan to give them his best price. Unfortunately, Horan said, "the price they got from the off-shore place was half the best price I could offer."
But the offshore data services' accuracy is a problem. Horan said that the local company is frustrated with the quality of the off-shore service.
"They don't have the luxury of handling questions about the data on a timely basis, so they usually don't handle them at all."
When the St. Petersburg Times investigated abuse of a Florida law that allowed people charged with a crime to erase all records of their arrest, they had to go through hundreds of thousands of pages of information, said Times reporter David Barstow.
Where information is easily understandable, they could have sent the data to a data entry service, said Barstow. But with some types of documents, the chance of error, or of missing a crucial clue, was too great.
And as you sort through the piles of paper, other things can come up. "You can't send it to someone who doesn't know what you're thinking," he said.
"We did consider trying to get journalism students to do it because it is pure drudgery - mindless work - but the only way to be absolutely confident about the data is to enter it yourself," Barstow said.
Barstow said the St. Petersburg Times did consider giving campaign contribution data to an off-shore data entry service.
"We explored the possibility of farming out to Jamaica, but it was still enormously expensive and it was not a possibility in terms of hardcore reporting. Too many things can go wrong."
In the United States, data entry is an expensive endeavor. Getting 100,000 typed records with 16 fields put into a database could cost $27,000 - $54,000 if you want the information double-checked, said Gayle Myers of International Data Corp. in St. Louis. And it could take on average four weeks, depending on other factors.
International Data estimates costs on a per-record basis. The rate is higher for smaller jobs because they still charge a set-up fee. For 1,500 typed records with 16 fields, the bill would be between $250 and $425.
"They (data entry services) don't handle small jobs. They're used to doing things for IBM. If they do a few thousand records they charge exorbitant rates," Barstow said.
But for the Cleveland Plain Dealer, the $10,000 tag on the database was worth keeping their reporters reporting. The Plain Dealer paid 18 cents per record.
"There's a trade off with concerns about accuracy and judgment calls. But if you do data entry you're not doing reporting," said Dave Davis, investigative reporter for the Plain Dealer. Davis said that if it doesn't take reporters away from reporting for very long, they would prefer to do data entry in house.
"But it's not worth it if it's something that's going to take lot of time," Davis said. Time pressures added incentive to hire outside help.
So the Plain Dealer went to a company they knew and had worked with before.
"We created the record layout and hired people to go through the records and input the data. We were looking over their shoulder the whole time."
"I told them not to guess and gave them my home number. They called a lot."
Another possible solution is illustrated by a cooperative effort among state legislative reporters and a professor at Pennsylvania State University.
An organization of full-time Pennsylvania state government reporters funded a project to computerize all the campaign contributions to the candidates for governor and state legislature in 1989 and 1990.
"The reason I got interested in the project was because I spent about a year tracking contributions from lobbyists - just doctors, lawyers and insurers. But there were 253 legislators," said David Morris, then-president of the Pennsylvania Legislative Correspondence Association and a state reporter for The Associated Press.
"I built my own database with index cards. Then I decided it was time to move into the 20th century," said Morris, who is now head of the AP bureau in Sacramento.
He teamed up with James Eisenstein, a Penn State professor of political science who has studied campaign finance since 1976. With help of work-study students, Eisenstein computerized almost 35,000 contributions in all, plus another 2,000 records of summary information.
"We computerized all contributions except individual contributions between $50 to $250," said Eisenstein.
The project cost about $20,000. "What we are trying to do should be done by the state," Eisenstein said.
Eisenstein has tried to keep the project going by forming a coalition of all people who would be interested in the information.
The Pennsylvania Newspaper Publishers' Association was the largest financier, giving about $5,500.
In return, three versions of the information were made available to all members of PNPA.
The first - the blue book - was an index of all contributions by name of candidate, type of contribution and alphabetical listing of contributor in each category.
The second - the red book - gave all PAC contributions sorted by the name of the PAC and then the name of the recipient.
The third - a database application - allowed users to make specific inquiries by candidate name, by contributor name and SIC code. The searches could be specified by chamber, party, election outcome, and date.
"So, for instance, you could find out all contributions that went to Republican senate losers in the primary between March 2 and April 2 in 1990. It is a very powerful program," said Eisenstein.
By including the SIC codes, "it does one thing that no other data base does," according to Eisenstein. It automatically classifies PACs by industry.
The date functions lets the reporter see if there is a relationship between the timing of a contribution and the legislative agenda.
But the project is petering out. The coalition only came up with $8,250 to computerize 1992 campaign contributions, not enough to get past the primary election reports.
But Eisenstein said he hasn't given up on the idea yet.
"About 20 organizations got at least one version of the reports," Eisenstein said. "We had a good, but not massive, response."
Using the database "became pretty routine," said Morris. Now that he's moved to Sacramento, Morris said he misses the luxury of using the database.
"It's a pain in the butt because you have to go through all this paper by hand. You have to go to the secretary of state, find the paper and take notes or make copies."
Morris is trying to get some news organizations interested in starting up a California database, but "it's a long, cumbersome process," he said.