State elections account for hundreds of millions of dollars in activity every two years, but tracking the money that fuels those races can be a cumbersome process. Nearly every state has its own method of reporting campaign finances and most don't offer the same kind of data that's available at the federal level.
State data comes in a variety of formats, but electronic access has been increasing lately. Some states provide data in electronic format or display the records on a Web site in a manner that could be imported into a database, while other states provide filings in Adobe Portable Document Files or other image files, or on paper.
During a recent project on state political parties, the Center for Public Integrity found at least 17 states that offered campaign finance data in a database, HTML or ASCII text format. Among the best are Ohio, which has comma-delimited text files for individual candidates and for all party organizations, and New York, which offers a single file containing contributions and expenditures during a given filing period. Washington, Idaho and Colorado also provide downloadable files.
Several other states are going the route of the searchable Web database, where a user can enter search terms or date ranges and get back text files that can easily be put into Microsoft Excel or Microsoft Access. States such as Florida and Maryland do this. Others offer data on a filing-by-filing basis - not exactly the best when you want to download everything - but it's better than nothing. Pennsylvania, Illinois and Rhode Island are examples of the latter. Typically, such files can be batch-downloaded if you know the filenames.
States that offer HTML versions of their data that can be put into a database with a little bit of work include Arizona, North Dakota, North Carolina and Nebraska. In some instances, you can simply save the HTML and use Access import utility, which will convert HTML tables into Access pretty easily. In other cases, it may make sense to develop a script that retrieves the pages or automatically "scrapes" the information from them.
Other states offer what can be called "pseudo-Web disclosure," providing only summary information but not the itemized records.
It should come as no surprise that such practices are wildly inconsistent in terms of the information provided and whether electronic records match paper filings. Maine, for example, offers text filings on its Web site purporting to contain campaign finance information, but they very often are missing thousands of records from party committees. Before revamping its system, Indiana offered downloads that included all filings by certain committees, meaning that amended records were simply piled on top of the original data with the potential for significant overlap.
Expenditure data, while often overlooked, can be a valuable resource. But it can have pitfalls, particularly when it comes to the issue of "subvendors," the breakdown of payments made by credit card, for example. Check with your state to see if it requires such itemization on its reports and whether any data available makes it easy to separate the lump sum payments from the itemized records (in most cases, you want the latter).
Here are some tips for handling state campaign finance data:
Know what the files, and thus the data, are supposed to look like. It makes sense to grab a copy of a blank report, even if you get all-electronic data.
Remember that a number of states have some form of public financing, and those records often appear in the general contributions. While it may be technically accurate to list the state government among the largest contributors to a candidate, you probably will want to remove those records from your query results.
If you are interested in the activities of political parties be sure to check your state's laws to see what kinds of accounts parties can maintain. A handful of states permit parties to have operating accounts for political use and administrative accounts for overhead and generic expenses. Sometimes these administrative accounts have little or no disclosure. South Carolina's parties will begin disclosing contributions to their administrative accounts at the end of the year. Other parties have building funds or housekeeping accounts that file separate reports, so be sure to check or you might be missing some of the picture.
Don't forget the small stuff: states treat small-dollar contributions very differently. Some states require lump sum reporting of contributions below a certain threshold, while others, like Nebraska, don't require any reporting of contributions that are not itemized. In smaller states, such figures could seriously affect your numbers.
Contact Derek Willis by e-mail at derek@thescoop.org